databricks-cli/bundle/tests/run_as/legacy/databricks.yml

69 lines
1.2 KiB
YAML
Raw Normal View History

Add legacy option for `run_as` (#1384) ## Changes This PR partially reverts the changes in https://github.com/databricks/cli/pull/1233 and puts the old code under an "experimental.use_legacy_run_as" configuration. This gives customers who ran into the breaking change made in the PR a way out. ## Tests Both manually and via unit tests. Manually verified that run_as works for pipelines now. And if a user wants to use the feature they need to be both a Metastore and a workspace admin. --------- Error when the deploying user is a workspace admin but not a metastore admin: ``` Error: terraform apply: exit status 1 Error: cannot update permissions: User is not a metastore admin for Metastore 'deco-uc-prod-aws-us-east-1'. with databricks_permissions.pipeline_foo, on bundle.tf.json line 23, in resource.databricks_permissions.pipeline_foo: 23: } ``` -------- Output of bundle validate: ``` ➜ bundle-playground git:(master) ✗ cli bundle validate Warning: You are using the legacy mode of run_as. The support for this mode is experimental and might be removed in a future release of the CLI. In order to run the DLT pipelines in your DAB as the run_as user this mode changes the owners of the pipelines to the run_as identity, which requires the user deploying the bundle to be a workspace admin, and also a Metastore admin if the pipeline target is in UC. at experimental.use_legacy_run_as in databricks.yml:13:22 Name: bundle-playground Target: default Workspace: Host: https://dbc-a39a1eb1-ef95.cloud.databricks.com User: shreyas.goenka@databricks.com Path: /Users/shreyas.goenka@databricks.com/.bundle/bundle-playground/default Found 1 warning ```
2024-04-22 11:51:41 +00:00
bundle:
name: "run_as"
run_as:
service_principal_name: "my_service_principal"
experimental:
use_legacy_run_as: true
resources:
jobs:
job_one:
name: Job One
tasks:
- task_key: "task_one"
notebook_task:
notebook_path: "./test.py"
job_two:
name: Job Two
tasks:
- task_key: "task_two"
notebook_task:
notebook_path: "./test.py"
job_three:
name: Job Three
run_as:
service_principal_name: "my_service_principal_for_job"
tasks:
- task_key: "task_three"
notebook_task:
notebook_path: "./test.py"
pipelines:
nyc_taxi_pipeline:
name: "nyc taxi loader"
permissions:
- level: CAN_VIEW
service_principal_name: my_service_principal
- level: CAN_VIEW
user_name: my_user_name
libraries:
- notebook:
path: ./dlt/nyc_taxi_loader
models:
model_one:
name: "skynet"
registered_models:
model_two:
name: "skynet (in UC)"
experiments:
experiment_one:
name: "experiment_one"
model_serving_endpoints:
model_serving_one:
name: "skynet"