steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/bundle/schema
History
shreyas-goenka cddc5f97f8
Fix the generated DABs JSON schema (#1322) 
## Changes
This PR fixes bundle schema being broken because `for_each_task: null`
was set in the generated schema. This is not valid according to the JSON
schema specification and thus the Red Hat YAML VSCode extension was
failing to parse the YAML configuration.

This PR fixes: https://github.com/databricks/cli/issues/1312

## Tests
The fix itself was tested manually. I asserted that the autocompletion
works now. This was mistakenly overlooked the first time around when the
regression was introduced in https://github.com/databricks/cli/pull/1204
because the YAML extension provides best-effort autocomplete suggestions
even if the JSON schema fails to load.

To prevent future regressions we also add a test to assert that the JSON
schema generated itself is a valid JSON schema object. This is done via
using the `ajv-cli` to validate the schema. This package is also used by
the Red Hat YAML extension and thus provides a high fidelity check for
ensuring the JSON schema is valid.

Before, with the old schema:
```
shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-216.json -d ../bundle-playground-3/databricks.yml
schema proj/schema-216.json is invalid
error: schema is invalid: data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items/properties/for_each_task must be object,boolean, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must be array, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must match a schema in anyOf
```

After, with the new schema:
```
shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-dev.json -d ../bundle-playground-3/databricks.yml
../bundle-playground-3/databricks.yml valid
```

After, autocomplete suggestions:

<img width="600" alt="Screenshot 2024-03-27 at 6 35 57 PM"
src="https://github.com/databricks/cli/assets/88374338/d0a62402-e323-4f36-854d-332b33cbeab8">
 		2024-03-28 11:25:36 +00:00
..
docs Update Go SDK to v0.36.0 (#1304) 2024-03-22 13:15:54 +00:00
README.md Move bundle schema update to an internal module (#1012) 2023-12-06 10:45:18 +00:00
docs.go Move bundle schema update to an internal module (#1012) 2023-12-06 10:45:18 +00:00
docs_test.go Add JSON schema validation for input template parameters (#598) 2023-08-01 14:09:27 +00:00
openapi.go Skip `for_each_task` when generating the bundle schema (#1204) 2024-02-13 14:13:47 +00:00
openapi_test.go Add JSON schema validation for input template parameters (#598) 2023-08-01 14:09:27 +00:00
schema.go Fix the generated DABs JSON schema (#1322) 2024-03-28 11:25:36 +00:00
schema_test.go Add internal tag for bundle fields to be skipped from schema (#636) 2023-08-10 10:03:52 +00:00
tracker.go Use tracker for reference loop tracking (#252) 2023-03-16 12:57:57 +01:00

README.md

Overview

docs/bundle_descriptions.json contains both autogenerated as well as manually written descriptions for the json schema. Specifically

  1. resources : almost all descriptions are autogenerated from the OpenAPI spec
  2. targets : almost all descriptions are copied over from root level entities (eg: bundle, artifacts)
  3. bundle : manually editted
  4. include : manually editted
  5. workspace : manually editted
  6. artifacts : manually editted

These descriptions are rendered in the inline documentation in an IDE

SOP: Add schema descriptions for new fields in bundle config

Manually edit bundle_descriptions.json to add your descriptions. Note that the descriptions in resources block is generated from the OpenAPI spec, and thus any changes there will be overwritten.