steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/bundle/config/mutator
History
Gleb Kanterov 3d91691f25
PythonMutator: propagate source locations (#1783) 
## Changes
Add a mechanism to load Python source locations in the Python mutator.
Previously, locations pointed to generated YAML. Now, they point to
Python sources instead. Python process outputs "locations.json"
containing locations of bundle paths, examples:

```json
{"path": "resources.jobs.job_0", "file": "resources/job_0.py", "line": 3, "column": 5}
{"path": "resources.jobs.job_0.tasks[0].task_key", "file": "resources/job_0.py", "line": 10, "column": 5}
{"path": "resources.jobs.job_1", "file": "resources/job_1.py", "line": 5, "column": 7}
```

Such locations form a tree, and we assign locations of the closest
ancestor to each `dyn.Value` based on its path. For example,
`resources.jobs.job_0.tasks[0].task_key` is located at `job_0.py:10:5`
and `resources.jobs.job_0.tasks[0].email_notifications` is located at
`job_0.py:3:5`, because we use the location of the job as the most
precise approximation.

This feature is only enabled if `experimental/python` is used.

Note: for now, we don't update locations with relative paths, because it
has a side effect in changing how these paths are resolved

## Example
```
% databricks bundle validate

Warning: job_cluster_key abc is not defined
  at resources.jobs.examples.tasks[0].job_cluster_key
  in resources/example.py:10:1
```

## Tests
Unit tests and manually
 		2025-01-22 15:37:37 +00:00
..
paths Enable gofumpt and goimports in golangci-lint (#1999) 2024-12-12 10:28:42 +01:00
python PythonMutator: propagate source locations (#1783) 2025-01-22 15:37:37 +00:00
apply_presets.go Added support for Databricks Apps in DABs (#1928) 2025-01-13 16:43:48 +00:00
apply_presets_test.go Handle `${workspace.file_path}` references in source-linked deployments (#2046) 2025-01-08 12:43:56 +00:00
apply_source_linked_deployment_preset.go fix: Detailed message for using source-linked deployment with file_path specified (#2119) 2025-01-20 16:16:51 +00:00
apply_source_linked_deployment_preset_test.go Added support for Databricks Apps in DABs (#1928) 2025-01-13 16:43:48 +00:00
capture_schema_dependency.go Patch references to UC schemas to capture dependencies automatically (#1989) 2025-01-16 13:27:00 +00:00
capture_schema_dependency_test.go Patch references to UC schemas to capture dependencies automatically (#1989) 2025-01-16 13:27:00 +00:00
compute_id_compat.go Enable gofumpt and goimports in golangci-lint (#1999) 2024-12-12 10:28:42 +01:00
compute_id_compate_test.go Added support for creating all-purpose clusters (#1698) 2024-09-23 10:42:34 +00:00
configure_dashboard_defaults.go Add support for AI/BI dashboards (#1743) 2024-10-29 09:11:08 +00:00
configure_dashboard_defaults_test.go Enable testifylint and fix the issues (#2065) 2025-01-02 12:03:41 +01:00
configure_volume_defaults.go Add default value for `volume_type` for DABs (#1952) 2024-12-04 11:05:54 +00:00
configure_volume_defaults_test.go Add default value for `volume_type` for DABs (#1952) 2024-12-04 11:05:54 +00:00
configure_wsfs.go Extract functionality to detect if the CLI is running on DBR (#1889) 2024-11-14 16:10:45 +00:00
configure_wsfs_test.go Extract functionality to detect if the CLI is running on DBR (#1889) 2024-11-14 16:10:45 +00:00
default_queueing.go Enable job queueing by default (#1385) 2024-04-22 10:36:39 +00:00
default_queueing_test.go Enable testifylint and fix the issues (#2065) 2025-01-02 12:03:41 +01:00
default_target.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_target_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_workspace_paths.go Add resource path field to bundle workspace configuration (#1800) 2024-10-02 13:55:40 +00:00
default_workspace_paths_test.go Add resource path field to bundle workspace configuration (#1800) 2024-10-02 13:55:40 +00:00
default_workspace_root.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_workspace_root_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
environments_compat.go Compare `.Kind()` instead of direct equality checks on a `dyn.Value` (#1520) 2024-06-27 13:28:19 +00:00
environments_compat_test.go Enable testifylint and fix the issues (#2065) 2025-01-02 12:03:41 +01:00
expand_pipeline_glob_paths.go Track multiple locations associated with a `dyn.Value` (#1510) 2024-07-16 11:27:27 +00:00
expand_pipeline_glob_paths_test.go Enable gofumpt and goimports in golangci-lint (#1999) 2024-12-12 10:28:42 +01:00
expand_workspace_root.go Enable perfsprint linter and apply autofix (#2071) 2025-01-07 10:49:23 +00:00
expand_workspace_root_test.go Always prepend bundle remote paths with /Workspace (#1724) 2024-10-02 15:34:00 +00:00
initialize_urls.go Enable gofumpt and goimports in golangci-lint (#1999) 2024-12-12 10:28:42 +01:00
initialize_urls_test.go Add error checking in tests and enable errcheck there (#1980) 2024-12-09 13:56:41 +01:00
initialize_variables.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
initialize_variables_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
initialize_workspace_client.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
load_git_details.go Set WorktreeRoot to sync root outside git repo (#2197) 2025-01-22 10:50:13 +00:00
merge_apps.go Added support for Databricks Apps in DABs (#1928) 2025-01-13 16:43:48 +00:00
merge_apps_test.go Added support for Databricks Apps in DABs (#1928) 2025-01-13 16:43:48 +00:00
merge_job_clusters.go Compare `.Kind()` instead of direct equality checks on a `dyn.Value` (#1520) 2024-06-27 13:28:19 +00:00
merge_job_clusters_test.go Bump github.com/databricks/databricks-sdk-go from 0.36.0 to 0.37.0 (#1326) 2024-04-03 10:39:53 +00:00
merge_job_parameters.go Merge job parameters based on their name (#1659) 2024-08-06 16:12:18 +00:00
merge_job_parameters_test.go Merge job parameters based on their name (#1659) 2024-08-06 16:12:18 +00:00
merge_job_tasks.go Compare `.Kind()` instead of direct equality checks on a `dyn.Value` (#1520) 2024-06-27 13:28:19 +00:00
merge_job_tasks_test.go Enable testifylint and fix the issues (#2065) 2025-01-02 12:03:41 +01:00
merge_pipeline_clusters.go Compare `.Kind()` instead of direct equality checks on a `dyn.Value` (#1520) 2024-06-27 13:28:19 +00:00
merge_pipeline_clusters_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
mutator.go Move loadGitDetails mutator to Initialize phase (#1944) 2024-12-02 09:49:32 +00:00
noop.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
override_compute.go Enable gofumpt and goimports in golangci-lint (#1999) 2024-12-12 10:28:42 +01:00
override_compute_test.go Show an error when using a cluster override with 'mode: production' (#1994) 2024-12-11 14:57:31 +00:00
populate_current_user.go Move utility functions dealing with IAM to libs/iamutil (#1820) 2024-10-10 13:02:25 +00:00
populate_current_user_test.go Add short_name helper function to bundle init templates (#1167) 2024-02-01 16:46:07 +00:00
prepend_workspace_prefix.go Enable perfsprint linter and apply autofix (#2071) 2025-01-07 10:49:23 +00:00
prepend_workspace_prefix_test.go Do not prepend paths starting with ~ or variable reference (#1905) 2024-11-15 15:03:59 +00:00
process_target_mode.go Encourage the use of root_path in production to ensure single deployment (#1712) 2025-01-13 12:19:12 +00:00
process_target_mode_test.go Added support for Databricks Apps in DABs (#1928) 2025-01-13 16:43:48 +00:00
resolve_resource_references.go Fix lost diags across different mutators (#2057) 2024-12-31 14:01:45 +00:00
resolve_resource_references_test.go Add error checking in tests and enable errcheck there (#1980) 2024-12-09 13:56:41 +01:00
resolve_variable_references.go Resolve variables in a loop (#2164) 2025-01-16 14:39:54 +00:00
resolve_variable_references_test.go Migrate TestResolveComplexVariableWithVarReference (#2156) 2025-01-15 17:52:17 +01:00
rewrite_sync_paths.go Rename `RootPath` -> `BundleRootPath` (#1792) 2024-09-27 10:03:05 +00:00
rewrite_sync_paths_test.go Rename `RootPath` -> `BundleRootPath` (#1792) 2024-09-27 10:03:05 +00:00
rewrite_workspace_prefix.go Enable gofumpt and goimports in golangci-lint (#1999) 2024-12-12 10:28:42 +01:00
rewrite_workspace_prefix_test.go Enable testifylint and fix the issues (#2065) 2025-01-02 12:03:41 +01:00
run_as.go Added support for Databricks Apps in DABs (#1928) 2025-01-13 16:43:48 +00:00
run_as_test.go Added support for Databricks Apps in DABs (#1928) 2025-01-13 16:43:48 +00:00
select_default_target.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
select_default_target_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
select_target.go Encourage the use of root_path in production to ensure single deployment (#1712) 2025-01-13 12:19:12 +00:00
select_target_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
set_variables.go Enable gofumpt and goimports in golangci-lint (#1999) 2024-12-12 10:28:42 +01:00
set_variables_test.go Enable testifylint and fix the issues (#2065) 2025-01-02 12:03:41 +01:00
sync_default_path.go Add paths field to bundle sync configuration (#1694) 2024-08-21 15:33:25 +00:00
sync_default_path_test.go Rename `RootPath` -> `BundleRootPath` (#1792) 2024-09-27 10:03:05 +00:00
sync_infer_root.go Enable gofumpt and goimports in golangci-lint (#1999) 2024-12-12 10:28:42 +01:00
sync_infer_root_internal_test.go Add paths field to bundle sync configuration (#1694) 2024-08-21 15:33:25 +00:00
sync_infer_root_test.go Rename `RootPath` -> `BundleRootPath` (#1792) 2024-09-27 10:03:05 +00:00
translate_paths.go Default to forward slash-separated paths for path translation (#2145) 2025-01-17 09:38:01 +00:00
translate_paths_apps.go Allow artifact path to be located outside the sync root (#2128) 2025-01-14 08:34:55 +00:00
translate_paths_apps_test.go Added support for Databricks Apps in DABs (#1928) 2025-01-13 16:43:48 +00:00
translate_paths_artifacts.go Allow artifact path to be located outside the sync root (#2128) 2025-01-14 08:34:55 +00:00
translate_paths_artifacts_test.go Default to forward slash-separated paths for path translation (#2145) 2025-01-17 09:38:01 +00:00
translate_paths_dashboards.go Allow artifact path to be located outside the sync root (#2128) 2025-01-14 08:34:55 +00:00
translate_paths_dashboards_test.go Default to forward slash-separated paths for path translation (#2145) 2025-01-17 09:38:01 +00:00
translate_paths_jobs.go Allow artifact path to be located outside the sync root (#2128) 2025-01-14 08:34:55 +00:00
translate_paths_pipelines.go Allow artifact path to be located outside the sync root (#2128) 2025-01-14 08:34:55 +00:00
translate_paths_test.go Default to forward slash-separated paths for path translation (#2145) 2025-01-17 09:38:01 +00:00
validate_git_details.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
validate_git_details_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
verify_cli_version.go Enable gofumpt and goimports in golangci-lint (#1999) 2024-12-12 10:28:42 +01:00
verify_cli_version_test.go Ignore CLI version check on development builds of the CLI (#1714) 2024-08-23 10:13:21 +00:00