databricks-cli/bundle
Andrew Nester 70fe0e36ef
Added `databricks bundle generate job` command (#1043)
## Changes
Now it's possible to generate bundle configuration for existing job.
For now it only supports jobs with notebook tasks.

It will download notebooks referenced in the job tasks and generate
bundle YAML config for this job which can be included in larger bundle.

## Tests
Running command manually

Example of generated config
```
resources:
  jobs:
    job_128737545467921:
      name: Notebook job
      format: MULTI_TASK
      tasks:
        - task_key: as_notebook
          existing_cluster_id: 0704-xxxxxx-yyyyyyy
          notebook_task:
            base_parameters:
              bundle_root: /Users/andrew.nester@databricks.com/.bundle/job_with_module_imports/development/files
            notebook_path: ./entry_notebook.py
            source: WORKSPACE
          run_if: ALL_SUCCESS
      max_concurrent_runs: 1
 ```

## Tests
Manual (on our last 100 jobs) + added end-to-end test

```
--- PASS: TestAccGenerateFromExistingJobAndDeploy (50.91s)
PASS
coverage: 61.5% of statements in ./...
ok github.com/databricks/cli/internal/bundle 51.209s coverage: 61.5% of
statements in ./...
```
2024-01-17 14:26:33 +00:00
..
artifacts Added exec.NewCommandExecutor to execute commands with correct interpreter (#1075) 2023-12-21 15:45:23 +00:00
config Added `databricks bundle generate job` command (#1043) 2024-01-17 14:26:33 +00:00
deploy Improve error when bundle root is not writable (#1093) 2023-12-28 13:15:21 +00:00
deployer Fix locker unlock for destroy (#492) 2023-06-19 15:57:25 +02:00
env Consolidate environment variable interaction (#747) 2023-09-11 08:18:43 +00:00
internal Upgrade golang.org/x/crypto to v0.17.0 in internal module (#1110) 2024-01-10 13:53:01 +00:00
libraries Fixed using repo files as pipeline libraries (#847) 2023-10-09 10:10:28 +00:00
metadata Make `file_path` and `artifact_path` fields consistent with json tag (#987) 2023-11-15 13:37:26 +00:00
permissions Use resource key as name in permissions code (#1087) 2023-12-22 14:45:53 +00:00
phases Allow referencing bundle resources by name (#872) 2024-01-04 21:04:42 +00:00
python Pass parameters to task when run with `--python-params` and `python_wheel_wrapper` is true (#1037) 2023-12-01 10:35:20 +00:00
run Support passing job parameters to bundle run (#1115) 2024-01-15 07:42:36 +00:00
schema Do not require positional arguments if they should be provided in JSON (#1125) 2024-01-17 10:53:50 +00:00
scripts Added exec.NewCommandExecutor to execute commands with correct interpreter (#1075) 2023-12-21 15:45:23 +00:00
tests Fixed instance pool resolving by name (#1102) 2024-01-05 10:50:53 +00:00
bundle.go Fix panic when bundle auth resolution fails (#1002) 2023-11-30 14:28:01 +00:00
bundle_test.go Consolidate environment variable interaction (#747) 2023-09-11 08:18:43 +00:00
context.go Rename variable `bundle -> b` (#989) 2023-11-15 14:03:36 +00:00
context_test.go Add command that writes the materialized bundle configuration to stdout (#95) 2022-11-21 15:39:53 +01:00
deferred.go Added support for bundle.Seq, simplified Mutator.Apply interface (#403) 2023-05-24 14:45:19 +02:00
deferred_test.go Rename variable `bundle -> b` (#989) 2023-11-15 14:03:36 +00:00
log_string.go Tune output of bundle deploy command (#1047) 2023-12-21 08:00:37 +00:00
mutator.go Added support for bundle.Seq, simplified Mutator.Apply interface (#403) 2023-05-24 14:45:19 +02:00
mutator_test.go Rename variable `bundle -> b` (#989) 2023-11-15 14:03:36 +00:00
root.go Consolidate environment variable interaction (#747) 2023-09-11 08:18:43 +00:00
root_test.go Consolidate environment variable interaction (#747) 2023-09-11 08:18:43 +00:00
seq.go Added support for bundle.Seq, simplified Mutator.Apply interface (#403) 2023-05-24 14:45:19 +02:00
seq_test.go Rename variable `bundle -> b` (#989) 2023-11-15 14:03:36 +00:00