steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/bundle/config/mutator
History
Andrew Nester 5f42791609
Added support for complex variables (#1467) 
## Changes
Added support for complex variables

Now it's possible to add and use complex variables as shown below

```
bundle:
  name: complex-variables

resources:
  jobs:
    my_job:
      job_clusters:
        - job_cluster_key: key
          new_cluster: ${var.cluster}
      tasks:
      - task_key: test
        job_cluster_key: key

variables:
  cluster:
    description: "A cluster definition"
    type: complex
    default:
      spark_version: "13.2.x-scala2.11"
      node_type_id: "Standard_DS3_v2"
      num_workers: 2
      spark_conf:
        spark.speculation: true
        spark.databricks.delta.retentionDurationCheck.enabled: false
```

Fixes #1298

- [x] Support for complex variables
- [x] Allow variable overrides (with shortcut) in targets
- [x] Don't allow to provide complex variables via flag or env variable
- [x] Fail validation if complex value is used but not `type: complex`
provided
- [x] Support using variables inside complex variables 

## Tests
Added unit tests

---------

Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com>
 		2024-06-26 10:25:32 +00:00
..
python PythonMutator: replace stdin/stdout with files (#1512) 2024-06-24 07:47:41 +00:00
default_queueing.go Enable job queueing by default (#1385) 2024-04-22 10:36:39 +00:00
default_queueing_test.go Fixed panic when loading incorrectly defined jobs (#1402) 2024-05-17 10:10:17 +00:00
default_target.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_target_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_workspace_paths.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_workspace_paths_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_workspace_root.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_workspace_root_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
environments_compat.go Use `dyn.InvalidValue` to indicate absence (#1507) 2024-06-19 15:24:57 +00:00
environments_compat_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
expand_pipeline_glob_paths.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
expand_pipeline_glob_paths_test.go Move path field to bundle type (#1316) 2024-03-27 09:03:24 +00:00
expand_workspace_root.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
expand_workspace_root_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
if.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
initialize_variables.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
initialize_variables_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
initialize_workspace_client.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
load_git_details.go Abstract over filesystem interaction with libs/vfs (#1452) 2024-05-30 07:41:50 +00:00
merge_job_clusters.go Use `dyn.InvalidValue` to indicate absence (#1507) 2024-06-19 15:24:57 +00:00
merge_job_clusters_test.go Bump github.com/databricks/databricks-sdk-go from 0.36.0 to 0.37.0 (#1326) 2024-04-03 10:39:53 +00:00
merge_job_tasks.go Use `dyn.InvalidValue` to indicate absence (#1507) 2024-06-19 15:24:57 +00:00
merge_job_tasks_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
merge_pipeline_clusters.go Use `dyn.InvalidValue` to indicate absence (#1507) 2024-06-19 15:24:57 +00:00
merge_pipeline_clusters_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
mutator.go PythonMutator: replace stdin/stdout with files (#1512) 2024-06-24 07:47:41 +00:00
noop.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
override_compute.go Bump github.com/databricks/databricks-sdk-go from 0.37.0 to 0.38.0 (#1361) 2024-04-16 12:03:21 +00:00
override_compute_test.go Bump github.com/databricks/databricks-sdk-go from 0.37.0 to 0.38.0 (#1361) 2024-04-16 12:03:21 +00:00
populate_current_user.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
populate_current_user_test.go Add short_name helper function to bundle init templates (#1167) 2024-02-01 16:46:07 +00:00
process_target_mode.go Set bool pointer to disable lock (#1516) 2024-06-21 11:14:33 +00:00
process_target_mode_test.go Set bool pointer to disable lock (#1516) 2024-06-21 11:14:33 +00:00
resolve_resource_references.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
resolve_resource_references_test.go Added support for complex variables (#1467) 2024-06-26 10:25:32 +00:00
resolve_variable_references.go Added support for complex variables (#1467) 2024-06-26 10:25:32 +00:00
resolve_variable_references_test.go Added support for complex variables (#1467) 2024-06-26 10:25:32 +00:00
rewrite_sync_paths.go Return `dyn.InvalidValue` instead of `dyn.NilValue` when errors happen (#1514) 2024-06-21 14:22:42 +00:00
rewrite_sync_paths_test.go Move path field to bundle type (#1316) 2024-03-27 09:03:24 +00:00
run_as.go Add support for Lakehouse monitoring in bundles (#1307) 2024-05-31 09:42:25 +00:00
run_as_test.go Properly deal with nil values in `convert.FromTyped` (#1511) 2024-06-21 13:43:21 +00:00
select_default_target.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
select_default_target_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
select_target.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
select_target_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
set_variables.go Added support for complex variables (#1467) 2024-06-26 10:25:32 +00:00
set_variables_test.go Added support for complex variables (#1467) 2024-06-26 10:25:32 +00:00
trampoline.go Move path field to bundle type (#1316) 2024-03-27 09:03:24 +00:00
trampoline_test.go Move path field to bundle type (#1316) 2024-03-27 09:03:24 +00:00
translate_paths.go Add context type and value to path rewriting (#1525) 2024-06-25 10:04:22 +00:00
translate_paths_artifacts.go Add context type and value to path rewriting (#1525) 2024-06-25 10:04:22 +00:00
translate_paths_jobs.go Add context type and value to path rewriting (#1525) 2024-06-25 10:04:22 +00:00
translate_paths_pipelines.go Add context type and value to path rewriting (#1525) 2024-06-25 10:04:22 +00:00
translate_paths_test.go Added support for job environments (#1379) 2024-04-22 11:44:34 +00:00
validate_git_details.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
validate_git_details_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
verify_cli_version.go Allow specifying CLI version constraints required to run the bundle (#1320) 2024-04-02 12:55:21 +00:00
verify_cli_version_test.go Allow specifying CLI version constraints required to run the bundle (#1320) 2024-04-02 12:55:21 +00:00