databricks-cli/libs
Andrew Nester 5f42791609
Added support for complex variables (#1467)
## Changes
Added support for complex variables

Now it's possible to add and use complex variables as shown below

```
bundle:
  name: complex-variables

resources:
  jobs:
    my_job:
      job_clusters:
        - job_cluster_key: key
          new_cluster: ${var.cluster}
      tasks:
      - task_key: test
        job_cluster_key: key

variables:
  cluster:
    description: "A cluster definition"
    type: complex
    default:
      spark_version: "13.2.x-scala2.11"
      node_type_id: "Standard_DS3_v2"
      num_workers: 2
      spark_conf:
        spark.speculation: true
        spark.databricks.delta.retentionDurationCheck.enabled: false
```

Fixes #1298

- [x] Support for complex variables
- [x] Allow variable overrides (with shortcut) in targets
- [x] Don't allow to provide complex variables via flag or env variable
- [x] Fail validation if complex value is used but not `type: complex`
provided
- [x] Support using variables inside complex variables 

## Tests
Added unit tests

---------

Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com>
2024-06-26 10:25:32 +00:00
..
auth Improve token refresh flow (#1434) 2024-05-16 10:22:09 +00:00
cmdgroup Add trailing newline in usage string (#1382) 2024-04-19 14:12:52 +00:00
cmdio Added `auth describe` command (#1244) 2024-04-03 08:14:04 +00:00
databrickscfg Update error checks that use the `os` package to use `errors.Is` (#1461) 2024-06-03 12:39:36 +00:00
diag Make bundle validation print text output by default (#1335) 2024-04-03 15:33:43 +00:00
dyn Added support for complex variables (#1467) 2024-06-26 10:25:32 +00:00
env Fix `panic: $HOME is not set` (#1027) 2023-11-29 19:08:27 +00:00
errs Added ability for deferred mutator execution (#380) 2023-05-16 18:01:50 +02:00
exec Fixed building Python artifacts on Windows with WSL (#1249) 2024-03-01 15:59:47 +00:00
filer Return `fs.ModeDir` for Git folders in the workspace (#1521) 2024-06-24 10:15:13 +00:00
fileset Abstract over filesystem interaction with libs/vfs (#1452) 2024-05-30 07:41:50 +00:00
flags Add override to support YAML inputs for apps (#921) 2023-10-27 18:57:26 +00:00
folders Move folders package into libs (#1184) 2024-02-07 16:33:18 +00:00
git Remove user credentials specified in the Git origin URL (#1494) 2024-06-17 09:49:00 +00:00
jsonschema Fix bundle schema for variables (#1396) 2024-04-25 11:23:50 +00:00
locker Correct name for force acquire deploy flag (#656) 2023-08-15 19:03:43 +00:00
log Rewrite the friendly log handler (#1038) 2023-12-01 12:17:04 +00:00
notebook Update error checks that use the `os` package to use `errors.Is` (#1461) 2024-06-03 12:39:36 +00:00
process Add ApplyPythonMutator (#1430) 2024-06-20 08:43:08 +00:00
python Added `python.DetectInterpreters` and other utils (#805) 2023-10-03 10:47:09 +00:00
set Added support for sync.include and sync.exclude sections (#671) 2023-08-18 08:07:25 +00:00
sync Clean up unused code (#1502) 2024-06-18 14:14:27 +00:00
tags Library to validate and normalize cloud specific tags (#819) 2023-09-29 08:49:08 +00:00
template Fix typo in DBT template (#1498) 2024-06-17 15:56:49 +00:00
terraform Add bundle destroy command (#300) 2023-04-06 12:54:58 +02:00
testfile Refactor and cover edge cases in sync integration tests (#160) 2023-01-10 13:16:30 +01:00
textutil Add support for Lakehouse monitoring in bundles (#1307) 2024-05-31 09:42:25 +00:00
vfs Abstract over filesystem interaction with libs/vfs (#1452) 2024-05-30 07:41:50 +00:00