steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/bundle/config/mutator
History
Andrew Nester 48ff18e5fc
Upload local libraries even if they don't have artifact defined (#1664) 
## Changes
Previously for all the libraries referenced in configuration DABs made
sure that there is corresponding artifact section.
But this is not really necessary and flexible, because local libraries
might be built outside of dabs context.
It also created difficult to follow logic in code where we back
referenced libraries to artifacts which was difficult to fllow


This PR does 3 things:
1. Allows all local libraries referenced in DABs config to be uploaded
to remote
2. Simplifies upload and glob references expand logic by doing this in
single place
3. Speed things up by uploading library only once and doing this in
parallel

## Tests
Added unit + integration tests + made sure that change is backward
compatible (no changes in existing tests)

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
 		2024-08-14 09:03:44 +00:00
..
python Support multiple paths for diagnostics (#1616) 2024-07-25 15:16:27 +00:00
configure_wsfs.go Add read-only mode for extension aware workspace filer (#1609) 2024-07-18 14:17:42 +00:00
default_queueing.go Enable job queueing by default (#1385) 2024-04-22 10:36:39 +00:00
default_queueing_test.go Fixed panic when loading incorrectly defined jobs (#1402) 2024-05-17 10:10:17 +00:00
default_target.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_target_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_workspace_paths.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_workspace_paths_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_workspace_root.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
default_workspace_root_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
environments_compat.go Compare `.Kind()` instead of direct equality checks on a `dyn.Value` (#1520) 2024-06-27 13:28:19 +00:00
environments_compat_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
expand_pipeline_glob_paths.go Track multiple locations associated with a `dyn.Value` (#1510) 2024-07-16 11:27:27 +00:00
expand_pipeline_glob_paths_test.go Move path field to bundle type (#1316) 2024-03-27 09:03:24 +00:00
expand_workspace_root.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
expand_workspace_root_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
initialize_variables.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
initialize_variables_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
initialize_workspace_client.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
load_git_details.go Use `vfs.Path` for filesystem interaction (#1554) 2024-07-03 10:13:22 +00:00
merge_job_clusters.go Compare `.Kind()` instead of direct equality checks on a `dyn.Value` (#1520) 2024-06-27 13:28:19 +00:00
merge_job_clusters_test.go Bump github.com/databricks/databricks-sdk-go from 0.36.0 to 0.37.0 (#1326) 2024-04-03 10:39:53 +00:00
merge_job_parameters.go Merge job parameters based on their name (#1659) 2024-08-06 16:12:18 +00:00
merge_job_parameters_test.go Merge job parameters based on their name (#1659) 2024-08-06 16:12:18 +00:00
merge_job_tasks.go Compare `.Kind()` instead of direct equality checks on a `dyn.Value` (#1520) 2024-06-27 13:28:19 +00:00
merge_job_tasks_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
merge_pipeline_clusters.go Compare `.Kind()` instead of direct equality checks on a `dyn.Value` (#1520) 2024-06-27 13:28:19 +00:00
merge_pipeline_clusters_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
mutator.go Use dynamic walking to validate unique resource keys (#1614) 2024-07-29 13:04:02 +00:00
noop.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
override_compute.go Bump github.com/databricks/databricks-sdk-go from 0.37.0 to 0.38.0 (#1361) 2024-04-16 12:03:21 +00:00
override_compute_test.go Bump github.com/databricks/databricks-sdk-go from 0.37.0 to 0.38.0 (#1361) 2024-04-16 12:03:21 +00:00
populate_current_user.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
populate_current_user_test.go Add short_name helper function to bundle init templates (#1167) 2024-02-01 16:46:07 +00:00
process_target_mode.go Add resource for UC schemas to DABs (#1413) 2024-07-31 12:16:28 +00:00
process_target_mode_test.go Add resource for UC schemas to DABs (#1413) 2024-07-31 12:16:28 +00:00
resolve_resource_references.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
resolve_resource_references_test.go Added support for complex variables (#1467) 2024-06-26 10:25:32 +00:00
resolve_variable_references.go Fixed resolving variable references inside slice variable (#1550) 2024-07-02 11:45:16 +00:00
resolve_variable_references_test.go Fixed resolving variable references inside slice variable (#1550) 2024-07-02 11:45:16 +00:00
rewrite_sync_paths.go Track multiple locations associated with a `dyn.Value` (#1510) 2024-07-16 11:27:27 +00:00
rewrite_sync_paths_test.go Move path field to bundle type (#1316) 2024-03-27 09:03:24 +00:00
run_as.go Support multiple paths for diagnostics (#1616) 2024-07-25 15:16:27 +00:00
run_as_test.go Add resource for UC schemas to DABs (#1413) 2024-07-31 12:16:28 +00:00
select_default_target.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
select_default_target_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
select_target.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
select_target_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
set_variables.go Change SetVariables mutator to mutate dynamic configuration instead (#1573) 2024-07-09 11:12:42 +00:00
set_variables_test.go Change SetVariables mutator to mutate dynamic configuration instead (#1573) 2024-07-09 11:12:42 +00:00
trampoline.go Move path field to bundle type (#1316) 2024-03-27 09:03:24 +00:00
trampoline_test.go Stop tracking file path locations in bundle resources (#1673) 2024-08-13 12:50:15 +00:00
translate_paths.go Track multiple locations associated with a `dyn.Value` (#1510) 2024-07-16 11:27:27 +00:00
translate_paths_artifacts.go Add context type and value to path rewriting (#1525) 2024-06-25 10:04:22 +00:00
translate_paths_jobs.go Upload local libraries even if they don't have artifact defined (#1664) 2024-08-14 09:03:44 +00:00
translate_paths_pipelines.go Add context type and value to path rewriting (#1525) 2024-06-25 10:04:22 +00:00
translate_paths_test.go Change SetVariables mutator to mutate dynamic configuration instead (#1573) 2024-07-09 11:12:42 +00:00
validate_git_details.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
validate_git_details_test.go Return `diag.Diagnostics` from mutators (#1305) 2024-03-25 14:18:47 +00:00
verify_cli_version.go Allow specifying CLI version constraints required to run the bundle (#1320) 2024-04-02 12:55:21 +00:00
verify_cli_version_test.go Allow specifying CLI version constraints required to run the bundle (#1320) 2024-04-02 12:55:21 +00:00