steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/bundle/internal/docs/output/reference.md

28 KiB
Raw Blame History

description: Configuration reference for databricks.yml

Configuration reference

This article provides reference for keys supported by configuration (YAML). See _.

For complete bundle examples, see _ and the bundle-examples GitHub repository.

artifacts

Type: Map

Defines the attributes to build artifacts, where each key is the name of the artifact, and the value is a Map that defines the artifact build settings. For information about the artifacts mapping, see _.

Artifact settings defined in the top level of the bundle configuration can be overridden in the targets mapping. See _.

artifacts:
  <artifact-name>:
    <artifact-field-name>: <artifact-field-value>

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • build
    • String
    • An optional set of non-default build commands to run locally before deployment.
    • executable
    • String
    • The executable type. Valid values are bash, sh, and cmd.
    • files
    • Sequence
    • The source files for the artifact. See _.
    • path
    • String
    • The location where the built artifact will be saved.
    • type
    • String
    • Required. The type of the artifact. Valid values are whl.

Example

artifacts:
  default:
    type: whl
    build: poetry build
    path: .

artifacts..files

Type: Sequence

The source files for the artifact.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • source
    • String
    • Required. The path of the files used to build the artifact.

bundle

Type: Map

The bundle attributes when deploying to this target,

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • cluster_id
    • String
    • The ID of a cluster to use to run the bundle. See _.
    • compute_id
    • String
    • databricks_cli_version
    • String
    • The Databricks CLI version to use for the bundle. See _.
    • deployment
    • Map
    • The definition of the bundle deployment. For supported attributes see _. See _.
    • git
    • Map
    • The Git version control details that are associated with your bundle. For supported attributes see _. See _.
    • name
    • String
    • The name of the bundle.
    • uuid
    • String

bundle.deployment

Type: Map

The definition of the bundle deployment

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • fail_on_active_runs
    • Boolean
    • Whether to fail on active runs. If this is set to true a deployment that is running can be interrupted.
    • lock
    • Map
    • The deployment lock attributes. See _.

bundle.deployment.lock

Type: Map

The deployment lock attributes.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • enabled
    • Boolean
    • Whether this lock is enabled.
    • force
    • Boolean
    • Whether to force this lock if it is enabled.

bundle.git

Type: Map

The Git version control details that are associated with your bundle.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • branch
    • String
    • The Git branch name. See _.
    • origin_url
    • String
    • The origin URL of the repository. See _.

experimental

Type: Map

Defines attributes for experimental features.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • pydabs
    • Map
    • The PyDABs configuration. See _.
    • python
    • Map
    • Configures loading of Python code defined with 'databricks-bundles' package. See _.
    • python_wheel_wrapper
    • Boolean
    • Whether to use a Python wheel wrapper.
    • scripts
    • Map
    • The commands to run.
    • use_legacy_run_as
    • Boolean
    • Whether to use the legacy run_as behavior.

experimental.pydabs

Type: Map

The PyDABs configuration.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • enabled
    • Boolean
    • Whether or not PyDABs (Private Preview) is enabled
    • import
    • Sequence
    • The PyDABs project to import to discover resources, resource generator and mutators
    • venv_path
    • String
    • The Python virtual environment path

experimental.python

Type: Map

Configures loading of Python code defined with 'databricks-bundles' package.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • mutators
    • Sequence
    • Mutators contains a list of fully qualified function paths to mutator functions. Example: ["my_project.mutators:add_default_cluster"]
    • resources
    • Sequence
    • Resources contains a list of fully qualified function paths to load resources defined in Python code. Example: ["my_project.resources:load_resources"]
    • venv_path
    • String
    • VEnvPath is path to the virtual environment. If enabled, Python code will execute within this environment. If disabled, it defaults to using the Python interpreter available in the current shell.

include

Type: Sequence

Specifies a list of path globs that contain configuration files to include within the bundle. See _

permissions

Type: Sequence

A Sequence that defines the permissions to apply to experiments, jobs, pipelines, and models defined in the bundle, where each item in the sequence is a permission for a specific entity.

See _ and _.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • group_name
    • String
    • The name of the group that has the permission set in level.
    • level
    • String
    • The allowed permission for user, group, service principal defined for this permission.
    • service_principal_name
    • String
    • The name of the service principal that has the permission set in level.
    • user_name
    • String
    • The name of the user that has the permission set in level.

Example

permissions:
  - level: CAN_VIEW
    group_name: test-group
  - level: CAN_MANAGE
    user_name: someone@example.com
  - level: CAN_RUN
    service_principal_name: 123456-abcdef

presets

Type: Map

Defines bundle deployment presets. See _.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • jobs_max_concurrent_runs
    • Integer
    • The maximum concurrent runs for a job.
    • name_prefix
    • String
    • The prefix for job runs of the bundle.
    • pipelines_development
    • Boolean
    • Whether pipeline deployments should be locked in development mode.
    • source_linked_deployment
    • Boolean
    • Whether to link the deployment to the bundle source.
    • tags
    • Map
    • The tags for the bundle deployment.
    • trigger_pause_status
    • String
    • A pause status to apply to all job triggers and schedules. Valid values are PAUSED or UNPAUSED.

resources

Type: Map

A Map that defines the resources for the bundle, where each key is the name of the resource, and the value is a Map that defines the resource. For more information about supported resources, and resource definition reference, see _.

resources:
  <resource-type>:
    <resource-name>:
      <resource-field-name>: <resource-field-value>

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • apps
    • Map
    • clusters
    • Map
    • The cluster definitions for the bundle, where each key is the name of a cluster. See _
    • dashboards
    • Map
    • The dashboard definitions for the bundle, where each key is the name of the dashboard. See _
    • experiments
    • Map
    • The experiment definitions for the bundle, where each key is the name of the experiment. See _
    • jobs
    • Map
    • The job definitions for the bundle, where each key is the name of the job. See _
    • model_serving_endpoints
    • Map
    • The model serving endpoint definitions for the bundle, where each key is the name of the model serving endpoint. See _
    • models
    • Map
    • The model definitions for the bundle, where each key is the name of the model. See _
    • pipelines
    • Map
    • The pipeline definitions for the bundle, where each key is the name of the pipeline. See _
    • quality_monitors
    • Map
    • The quality monitor definitions for the bundle, where each key is the name of the quality monitor. See _
    • registered_models
    • Map
    • The registered model definitions for the bundle, where each key is the name of the registered model. See _
    • schemas
    • Map
    • The schema definitions for the bundle, where each key is the name of the schema. See _
    • volumes
    • Map
    • The volume definitions for the bundle, where each key is the name of the volume. See _

run_as

Type: Map

The identity to use when running workflows. See _.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • service_principal_name
    • String
    • The application ID of an active service principal. Setting this field requires the servicePrincipal/user role.
    • user_name
    • String
    • The email of an active workspace user. Non-admin users can only set this field to their own email.

sync

Type: Map

The files and file paths to include or exclude in the bundle. See _.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • exclude
    • Sequence
    • A list of files or folders to exclude from the bundle.
    • include
    • Sequence
    • A list of files or folders to include in the bundle.
    • paths
    • Sequence
    • The local folder paths, which can be outside the bundle root, to synchronize to the workspace when the bundle is deployed.

targets

Type: Map

Defines deployment targets for the bundle. See _

targets:
  <target-name>:
    <target-field-name>: <target-field-value>

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • artifacts
    • Map
    • The artifacts to include in the target deployment. See _.
    • bundle
    • Map
    • The bundle attributes when deploying to this target. See _.
    • cluster_id
    • String
    • The ID of the cluster to use for this target.
    • compute_id
    • String
    • Deprecated. The ID of the compute to use for this target.
    • default
    • Boolean
    • Whether this target is the default target.
    • git
    • Map
    • The Git version control settings for the target. See _.
    • mode
    • String
    • The deployment mode for the target. Valid values are development or production. See _.
    • permissions
    • Sequence
    • The permissions for deploying and running the bundle in the target. See _.
    • presets
    • Map
    • The deployment presets for the target. See _.
    • resources
    • Map
    • The resource definitions for the target. See _.
    • run_as
    • Map
    • The identity to use to run the bundle, see _. See _.
    • sync
    • Map
    • The local paths to sync to the target workspace when a bundle is run or deployed. See _.
    • variables
    • Map
    • The custom variable definitions for the target. See _.
    • workspace
    • Map
    • The Databricks workspace for the target. See _.

targets..artifacts

Type: Map

The artifacts to include in the target deployment.

artifacts:
  <artifact-name>:
    <artifact-field-name>: <artifact-field-value>

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • build
    • String
    • An optional set of non-default build commands to run locally before deployment.
    • executable
    • String
    • The executable type. Valid values are bash, sh, and cmd.
    • files
    • Sequence
    • The source files for the artifact. See _.
    • path
    • String
    • The location where the built artifact will be saved.
    • type
    • String
    • Required. The type of the artifact. Valid values are whl.

targets..artifacts..files

Type: Sequence

The source files for the artifact.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • source
    • String
    • Required. The path of the files used to build the artifact.

targets..bundle

Type: Map

The bundle attributes when deploying to this target.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • cluster_id
    • String
    • The ID of a cluster to use to run the bundle. See _.
    • compute_id
    • String
    • databricks_cli_version
    • String
    • The Databricks CLI version to use for the bundle. See _.
    • deployment
    • Map
    • The definition of the bundle deployment. For supported attributes see _. See _.
    • git
    • Map
    • The Git version control details that are associated with your bundle. For supported attributes see _. See _.
    • name
    • String
    • The name of the bundle.
    • uuid
    • String

targets..bundle.deployment

Type: Map

The definition of the bundle deployment

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • fail_on_active_runs
    • Boolean
    • Whether to fail on active runs. If this is set to true a deployment that is running can be interrupted.
    • lock
    • Map
    • The deployment lock attributes. See _.

targets..bundle.deployment.lock

Type: Map

The deployment lock attributes.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • enabled
    • Boolean
    • Whether this lock is enabled.
    • force
    • Boolean
    • Whether to force this lock if it is enabled.

targets..bundle.git

Type: Map

The Git version control details that are associated with your bundle.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • branch
    • String
    • The Git branch name. See _.
    • origin_url
    • String
    • The origin URL of the repository. See _.

targets..git

Type: Map

The Git version control settings for the target.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • branch
    • String
    • The Git branch name. See _.
    • origin_url
    • String
    • The origin URL of the repository. See _.

targets..permissions

Type: Sequence

The permissions for deploying and running the bundle in the target.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • group_name
    • String
    • The name of the group that has the permission set in level.
    • level
    • String
    • The allowed permission for user, group, service principal defined for this permission.
    • service_principal_name
    • String
    • The name of the service principal that has the permission set in level.
    • user_name
    • String
    • The name of the user that has the permission set in level.

targets..presets

Type: Map

The deployment presets for the target.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • jobs_max_concurrent_runs
    • Integer
    • The maximum concurrent runs for a job.
    • name_prefix
    • String
    • The prefix for job runs of the bundle.
    • pipelines_development
    • Boolean
    • Whether pipeline deployments should be locked in development mode.
    • source_linked_deployment
    • Boolean
    • Whether to link the deployment to the bundle source.
    • tags
    • Map
    • The tags for the bundle deployment.
    • trigger_pause_status
    • String
    • A pause status to apply to all job triggers and schedules. Valid values are PAUSED or UNPAUSED.

targets..resources

Type: Map

The resource definitions for the target.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • apps
    • Map
    • clusters
    • Map
    • The cluster definitions for the bundle, where each key is the name of a cluster. See _
    • dashboards
    • Map
    • The dashboard definitions for the bundle, where each key is the name of the dashboard. See _
    • experiments
    • Map
    • The experiment definitions for the bundle, where each key is the name of the experiment. See _
    • jobs
    • Map
    • The job definitions for the bundle, where each key is the name of the job. See _
    • model_serving_endpoints
    • Map
    • The model serving endpoint definitions for the bundle, where each key is the name of the model serving endpoint. See _
    • models
    • Map
    • The model definitions for the bundle, where each key is the name of the model. See _
    • pipelines
    • Map
    • The pipeline definitions for the bundle, where each key is the name of the pipeline. See _
    • quality_monitors
    • Map
    • The quality monitor definitions for the bundle, where each key is the name of the quality monitor. See _
    • registered_models
    • Map
    • The registered model definitions for the bundle, where each key is the name of the registered model. See _
    • schemas
    • Map
    • The schema definitions for the bundle, where each key is the name of the schema. See _
    • volumes
    • Map
    • The volume definitions for the bundle, where each key is the name of the volume. See _

targets..run_as

Type: Map

The identity to use to run the bundle.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • service_principal_name
    • String
    • The application ID of an active service principal. Setting this field requires the servicePrincipal/user role.
    • user_name
    • String
    • The email of an active workspace user. Non-admin users can only set this field to their own email.

targets..sync

Type: Map

The local paths to sync to the target workspace when a bundle is run or deployed.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • exclude
    • Sequence
    • A list of files or folders to exclude from the bundle.
    • include
    • Sequence
    • A list of files or folders to include in the bundle.
    • paths
    • Sequence
    • The local folder paths, which can be outside the bundle root, to synchronize to the workspace when the bundle is deployed.

targets..variables

Type: Map

The custom variable definitions for the target.

variables:
  <variable-name>:
    <variable-field-name>: <variable-field-value>

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • default
    • Any
    • description
    • String
    • The description of the variable.
    • lookup
    • Map
    • The name of the alert, cluster_policy, cluster, dashboard, instance_pool, job, metastore, pipeline, query, service_principal, or warehouse object for which to retrieve an ID. See _.
    • type
    • String
    • The type of the variable.

targets..variables..lookup

Type: Map

The name of the alert, cluster_policy, cluster, dashboard, instance_pool, job, metastore, pipeline, query, service_principal, or warehouse object for which to retrieve an ID.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • alert
    • String
    • cluster
    • String
    • cluster_policy
    • String
    • dashboard
    • String
    • instance_pool
    • String
    • job
    • String
    • metastore
    • String
    • notification_destination
    • String
    • pipeline
    • String
    • query
    • String
    • service_principal
    • String
    • warehouse
    • String

targets..workspace

Type: Map

The Databricks workspace for the target.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • artifact_path
    • String
    • The artifact path to use within the workspace for both deployments and workflow runs
    • auth_type
    • String
    • The authentication type.
    • azure_client_id
    • String
    • The Azure client ID
    • azure_environment
    • String
    • The Azure environment
    • azure_login_app_id
    • String
    • The Azure login app ID
    • azure_tenant_id
    • String
    • The Azure tenant ID
    • azure_use_msi
    • Boolean
    • Whether to use MSI for Azure
    • azure_workspace_resource_id
    • String
    • The Azure workspace resource ID
    • client_id
    • String
    • The client ID for the workspace
    • file_path
    • String
    • The file path to use within the workspace for both deployments and workflow runs
    • google_service_account
    • String
    • The Google service account name
    • host
    • String
    • The Databricks workspace host URL
    • profile
    • String
    • The Databricks workspace profile name
    • resource_path
    • String
    • The workspace resource path
    • root_path
    • String
    • The Databricks workspace root path
    • state_path
    • String
    • The workspace state path

variables

Type: Map

Defines a custom variable for the bundle. See _.

variables:
  <variable-name>:
    <variable-field-name>: <variable-field-value>

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • default
    • Any
    • description
    • String
    • The description of the variable
    • lookup
    • Map
    • The name of the alert, cluster_policy, cluster, dashboard, instance_pool, job, metastore, pipeline, query, service_principal, or warehouse object for which to retrieve an ID. See _.
    • type
    • String
    • The type of the variable.

variables..lookup

Type: Map

The name of the alert, cluster_policy, cluster, dashboard, instance_pool, job, metastore, pipeline, query, service_principal, or warehouse object for which to retrieve an ID.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • alert
    • String
    • cluster
    • String
    • cluster_policy
    • String
    • dashboard
    • String
    • instance_pool
    • String
    • job
    • String
    • metastore
    • String
    • notification_destination
    • String
    • pipeline
    • String
    • query
    • String
    • service_principal
    • String
    • warehouse
    • String

workspace

Type: Map

Defines the Databricks workspace for the bundle. See _.

.. list-table:: :header-rows: 1

    • Key
    • Type
    • Description
    • artifact_path
    • String
    • The artifact path to use within the workspace for both deployments and workflow runs
    • auth_type
    • String
    • The authentication type.
    • azure_client_id
    • String
    • The Azure client ID
    • azure_environment
    • String
    • The Azure environment
    • azure_login_app_id
    • String
    • The Azure login app ID
    • azure_tenant_id
    • String
    • The Azure tenant ID
    • azure_use_msi
    • Boolean
    • Whether to use MSI for Azure
    • azure_workspace_resource_id
    • String
    • The Azure workspace resource ID
    • client_id
    • String
    • The client ID for the workspace
    • file_path
    • String
    • The file path to use within the workspace for both deployments and workflow runs
    • google_service_account
    • String
    • The Google service account name
    • host
    • String
    • The Databricks workspace host URL
    • profile
    • String
    • The Databricks workspace profile name
    • resource_path
    • String
    • The workspace resource path
    • root_path
    • String
    • The Databricks workspace root path
    • state_path
    • String
    • The workspace state path