steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/bundle/schema
History
Andrew Nester 5f42791609
Added support for complex variables (#1467) 
## Changes
Added support for complex variables

Now it's possible to add and use complex variables as shown below

```
bundle:
  name: complex-variables

resources:
  jobs:
    my_job:
      job_clusters:
        - job_cluster_key: key
          new_cluster: ${var.cluster}
      tasks:
      - task_key: test
        job_cluster_key: key

variables:
  cluster:
    description: "A cluster definition"
    type: complex
    default:
      spark_version: "13.2.x-scala2.11"
      node_type_id: "Standard_DS3_v2"
      num_workers: 2
      spark_conf:
        spark.speculation: true
        spark.databricks.delta.retentionDurationCheck.enabled: false
```

Fixes #1298

- [x] Support for complex variables
- [x] Allow variable overrides (with shortcut) in targets
- [x] Don't allow to provide complex variables via flag or env variable
- [x] Fail validation if complex value is used but not `type: complex`
provided
- [x] Support using variables inside complex variables 

## Tests
Added unit tests

---------

Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com>
 		2024-06-26 10:25:32 +00:00
..
docs Bump github.com/databricks/databricks-sdk-go from 0.42.0 to 0.43.0 (#1522) 2024-06-25 12:51:17 +00:00
README.md Move bundle schema update to an internal module (#1012) 2023-12-06 10:45:18 +00:00
docs.go Fix description memoization in bundle schema (#1409) 2024-05-01 11:04:37 +00:00
docs_test.go Add JSON schema validation for input template parameters (#598) 2023-08-01 14:09:27 +00:00
openapi.go Fix description memoization in bundle schema (#1409) 2024-05-01 11:04:37 +00:00
openapi_test.go Fix description memoization in bundle schema (#1409) 2024-05-01 11:04:37 +00:00
schema.go Allow variable references in non-string fields in the JSON schema (#1398) 2024-04-25 11:20:45 +00:00
schema_test.go Added support for complex variables (#1467) 2024-06-26 10:25:32 +00:00
tracker.go Use tracker for reference loop tracking (#252) 2023-03-16 12:57:57 +01:00

README.md

Overview

docs/bundle_descriptions.json contains both autogenerated as well as manually written descriptions for the json schema. Specifically

  1. resources : almost all descriptions are autogenerated from the OpenAPI spec
  2. targets : almost all descriptions are copied over from root level entities (eg: bundle, artifacts)
  3. bundle : manually editted
  4. include : manually editted
  5. workspace : manually editted
  6. artifacts : manually editted

These descriptions are rendered in the inline documentation in an IDE

SOP: Add schema descriptions for new fields in bundle config

Manually edit bundle_descriptions.json to add your descriptions. Note that the descriptions in resources block is generated from the OpenAPI spec, and thus any changes there will be overwritten.