steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/bundle/tests
History
Andrew Nester 27f51c760f
Added validate mutator to surface additional bundle warnings (#1352) 
## Changes
All these validators will return warnings as part of `bundle validate`
run

Added 2 mutators: 
1. To check that if tasks use job_cluster_key it is actually defined
2. To check if there are any files to sync as part of deployment

Also added `bundle.Parallel` to run them in parallel

To make sure mutators under bundle.Parallel do not mutate config,
introduced new `ReadOnlyMutator`, `ReadOnlyBundle` and `ReadOnlyConfig`.

Example 

```
databricks bundle validate -p deco-staging
Warning: unknown field: new_cluster
  at resources.jobs.my_job
  in bundle.yml:24:7

Warning: job_cluster_key high_cpu_workload_job_cluster is not defined
  at resources.jobs.my_job.tasks[0].job_cluster_key
  in bundle.yml:35:28

Warning: There are no files to sync, please check your your .gitignore and sync.exclude configuration
  at sync.exclude
  in bundle.yml:18:5

Name: test
Target: default
Workspace:
  Host: https://acme.databricks.com
  User: andrew.nester@databricks.com
  Path: /Users/andrew.nester@databricks.com/.bundle/test/default

Found 3 warnings
```

## Tests
Added unit tests
 		2024-04-18 15:13:16 +00:00
..
autoload_git Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
basic First look for databricks.yml before falling back to bundle.yml (#580) 2023-07-18 12:16:34 +02:00
bundle_permissions Added support for top-level permissions (#928) 2023-11-13 11:29:40 +00:00
conflicting_resource_ids Require include glob patterns to be explicitly defined (#602) 2023-07-25 10:00:46 +02:00
environment_overrides Use dynamic configuration model in bundles (#1098) 2024-02-16 19:41:58 +00:00
environments_autoload_git Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
environments_job_and_pipeline Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
environments_override_job_cluster Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
git_branch_validation Fix bundle git branch validation (#645) 2023-08-07 17:29:02 +00:00
include_default Require include glob patterns to be explicitly defined (#602) 2023-07-25 10:00:46 +02:00
include_invalid First look for databricks.yml before falling back to bundle.yml (#580) 2023-07-18 12:16:34 +02:00
include_multiple Require include glob patterns to be explicitly defined (#602) 2023-07-25 10:00:46 +02:00
include_with_glob First look for databricks.yml before falling back to bundle.yml (#580) 2023-07-18 12:16:34 +02:00
interpolation First look for databricks.yml before falling back to bundle.yml (#580) 2023-07-18 12:16:34 +02:00
interpolation_target Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
job_and_pipeline Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
job_cluster_key Added validate mutator to surface additional bundle warnings (#1352) 2024-04-18 15:13:16 +00:00
job_with_spark_conf Test existing behavior when loading non-string spark conf values (#1071) 2023-12-18 11:22:22 +00:00
model_serving_endpoint Support Model Serving Endpoints in bundles (#682) 2023-09-07 21:54:31 +00:00
override_job_cluster Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
override_job_tasks Enable environment overrides for job tasks (#779) 2023-09-18 14:13:50 +00:00
override_pipeline_cluster Enable target overrides for pipeline clusters (#792) 2023-09-21 19:21:20 +00:00
override_sync Allow target overrides for sync section (#856) 2023-10-10 15:18:18 +00:00
override_sync_no_root Allow target overrides for sync section (#856) 2023-10-10 15:18:18 +00:00
path_translation Rewrite relative paths using `dyn.Location` of the underlying value (#1273) 2024-03-18 16:23:39 +00:00
pipeline_glob_paths Move bundle tests into bundle/tests (#1299) 2024-03-21 10:37:05 +00:00
python_wheel Move bundle tests into bundle/tests (#1299) 2024-03-21 10:37:05 +00:00
registered_model Support Unity Catalog Registered Models in bundles (#846) 2023-10-16 15:32:49 +00:00
relative_path_translation Retain location information of variable reference (#1333) 2024-04-03 10:40:29 +00:00
relative_path_with_includes Make resource and artifact paths in bundle config relative to config folder (#708) 2023-09-04 09:55:01 +00:00
run_as Add allow list for resources when bundle `run_as` is set (#1233) 2024-03-27 16:13:53 +00:00
target_empty Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
target_overrides Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
variables Fixed instance pool resolving by name (#1102) 2024-01-05 10:50:53 +00:00
yaml_anchors First look for databricks.yml before falling back to bundle.yml (#580) 2023-07-18 12:16:34 +02:00
yaml_anchors_separate_block Do not emit warning on YAML anchor blocks (#1354) 2024-04-10 09:55:02 +00:00
README.md Move mutator interface to top level bundle package (#105) 2022-11-28 10:59:43 +01:00
basic_test.go Move mutator interface to top level bundle package (#105) 2022-11-28 10:59:43 +01:00
bundle_permissions_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
conflicting_resource_ids_test.go Load bundle configuration from mutator (#1318) 2024-03-27 10:49:05 +00:00
environment_git_test.go Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
environment_overrides_test.go Use dynamic configuration model in bundles (#1098) 2024-02-16 19:41:58 +00:00
environments_job_and_pipeline_test.go Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
environments_override_job_cluster_test.go Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
git_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
include_test.go Load bundle configuration from mutator (#1318) 2024-03-27 10:49:05 +00:00
interpolation_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
job_and_pipeline_test.go Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
job_cluster_key_test.go Added validate mutator to surface additional bundle warnings (#1352) 2024-04-18 15:13:16 +00:00
job_with_spark_conf_test.go Use dynamic configuration model in bundles (#1098) 2024-02-16 19:41:58 +00:00
loader.go Do not emit warning on YAML anchor blocks (#1354) 2024-04-10 09:55:02 +00:00
model_serving_endpoint_test.go Support Model Serving Endpoints in bundles (#682) 2023-09-07 21:54:31 +00:00
override_job_cluster_test.go Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
override_job_tasks_test.go Enable environment overrides for job tasks (#779) 2023-09-18 14:13:50 +00:00
override_pipeline_cluster_test.go Enable target overrides for pipeline clusters (#792) 2023-09-21 19:21:20 +00:00
override_sync_test.go Use dynamic configuration model in bundles (#1098) 2024-02-16 19:41:58 +00:00
path_translation_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
pipeline_glob_paths_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
python_wheel_test.go Load bundle configuration from mutator (#1318) 2024-03-27 10:49:05 +00:00
registered_model_test.go Support Unity Catalog Registered Models in bundles (#846) 2023-10-16 15:32:49 +00:00
relative_path_translation_test.go Retain location information of variable reference (#1333) 2024-04-03 10:40:29 +00:00
relative_path_with_includes_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
run_as_test.go Add allow list for resources when bundle `run_as` is set (#1233) 2024-03-27 16:13:53 +00:00
suggest_target_test.go List available targets when incorrect target passed (#756) 2023-09-08 15:37:55 +00:00
sync_include_exclude_no_matches_test.go Added validate mutator to surface additional bundle warnings (#1352) 2024-04-18 15:13:16 +00:00
target_empty_test.go Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
target_overrides_test.go Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
variables_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
yaml_anchors_test.go Do not emit warning on YAML anchor blocks (#1354) 2024-04-10 09:55:02 +00:00

README.md

Bundle configuration tests

Every test here uses an example bundle configuration. Each bundle configuration is located in a dedicated subdirectory.