steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/bundle/config/mutator
History
Andrew Nester 70fe0e36ef
Added `databricks bundle generate job` command (#1043) 
## Changes
Now it's possible to generate bundle configuration for existing job.
For now it only supports jobs with notebook tasks.

It will download notebooks referenced in the job tasks and generate
bundle YAML config for this job which can be included in larger bundle.

## Tests
Running command manually

Example of generated config
```
resources:
  jobs:
    job_128737545467921:
      name: Notebook job
      format: MULTI_TASK
      tasks:
        - task_key: as_notebook
          existing_cluster_id: 0704-xxxxxx-yyyyyyy
          notebook_task:
            base_parameters:
              bundle_root: /Users/andrew.nester@databricks.com/.bundle/job_with_module_imports/development/files
            notebook_path: ./entry_notebook.py
            source: WORKSPACE
          run_if: ALL_SUCCESS
      max_concurrent_runs: 1
 ```

## Tests
Manual (on our last 100 jobs) + added end-to-end test

```
--- PASS: TestAccGenerateFromExistingJobAndDeploy (50.91s)
PASS
coverage: 61.5% of statements in ./...
ok github.com/databricks/cli/internal/bundle 51.209s coverage: 61.5% of
statements in ./...
```
 		2024-01-17 14:26:33 +00:00
..
default_target.go Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
default_target_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
default_workspace_paths.go Make `file_path` and `artifact_path` fields consistent with json tag (#987) 2023-11-15 13:37:26 +00:00
default_workspace_paths_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
default_workspace_root.go Renamed `environments` to `targets` in bundle configuration (#670) 2023-08-17 15:22:32 +00:00
default_workspace_root_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
expand_pipeline_glob_paths.go Do not replace pipeline libraries if there are no matches for pattern (#1021) 2023-11-29 13:20:13 +00:00
expand_pipeline_glob_paths_test.go Do not replace pipeline libraries if there are no matches for pattern (#1021) 2023-11-29 13:20:13 +00:00
expand_workspace_root.go Added support for bundle.Seq, simplified Mutator.Apply interface (#403) 2023-05-24 14:45:19 +02:00
expand_workspace_root_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
if.go Make a notebook wrapper for Python wheel tasks optional (#797) 2023-09-26 14:32:20 +00:00
initialize_variables.go Initialize variable definitions that are defined without properties (#966) 2023-11-08 11:01:14 +00:00
initialize_variables_test.go Initialize variable definitions that are defined without properties (#966) 2023-11-08 11:01:14 +00:00
initialize_workspace_client.go Fix panic when bundle auth resolution fails (#1002) 2023-11-30 14:28:01 +00:00
load_git_details.go Persist deployment metadata in WSFS (#845) 2023-10-27 12:55:43 +00:00
mutator.go Initialize variable definitions that are defined without properties (#966) 2023-11-08 11:01:14 +00:00
noop.go Make a notebook wrapper for Python wheel tasks optional (#797) 2023-09-26 14:32:20 +00:00
override_compute.go Consolidate environment variable interaction (#747) 2023-09-11 08:18:43 +00:00
override_compute_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
populate_current_user.go Added `databricks bundle generate job` command (#1043) 2024-01-17 14:26:33 +00:00
populate_current_user_test.go Allow digits in the generated short name (#820) 2023-09-29 06:58:40 +00:00
process_include.go Added support for bundle.Seq, simplified Mutator.Apply interface (#403) 2023-05-24 14:45:19 +02:00
process_include_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
process_root_includes.go Consolidate environment variable interaction (#747) 2023-09-11 08:18:43 +00:00
process_root_includes_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
process_target_mode.go Change recommended production deployment path from /Shared to /Users (#1091) 2024-01-02 19:58:24 +00:00
process_target_mode_test.go Change recommended production deployment path from /Shared to /Users (#1091) 2024-01-02 19:58:24 +00:00
resolve_resource_references.go Allow referencing bundle resources by name (#872) 2024-01-04 21:04:42 +00:00
resolve_resource_references_test.go Allow referencing bundle resources by name (#872) 2024-01-04 21:04:42 +00:00
run_as.go Added run_as section for bundle configuration (#692) 2023-08-23 16:47:07 +00:00
select_default_target.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
select_default_target_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
select_target.go List available targets when incorrect target passed (#756) 2023-09-08 15:37:55 +00:00
select_target_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
set_variables.go Allow referencing bundle resources by name (#872) 2024-01-04 21:04:42 +00:00
set_variables_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00
trampoline.go Make `file_path` and `artifact_path` fields consistent with json tag (#987) 2023-11-15 13:37:26 +00:00
trampoline_test.go Consolidate environment variable interaction (#747) 2023-09-11 08:18:43 +00:00
translate_paths.go Make `file_path` and `artifact_path` fields consistent with json tag (#987) 2023-11-15 13:37:26 +00:00
translate_paths_artifacts.go Make resource and artifact paths in bundle config relative to config folder (#708) 2023-09-04 09:55:01 +00:00
translate_paths_jobs.go Enable `spark_jar_task` with local JAR libraries (#993) 2023-11-21 10:15:09 +00:00
translate_paths_pipelines.go Make resource and artifact paths in bundle config relative to config folder (#708) 2023-09-04 09:55:01 +00:00
translate_paths_test.go Enable `spark_jar_task` with local JAR libraries (#993) 2023-11-21 10:15:09 +00:00
validate_git_details.go Add validation for Git settings in bundles (#578) 2023-07-30 12:44:33 +00:00
validate_git_details_test.go Replace direct calls with `bundle.Apply` (#990) 2023-11-15 14:19:18 +00:00