## Changes
It covers both https://$DATABRICKS_HOST and http://$DATABRICKS_HOST so
the test output does not change between local and the cloud.
## Tests
Existing tests using golden files (acceptance and integration) catch
this and were updated.
## Changes
When regenerating CLI with a new Go SDK
https://github.com/databricks/cli/pull/2126 I've noticed that some
parameters such as `no_compute` for apps are not added as flags for the
CLI commands.
This happened because we ignored all other top level fields if there's a
request body object field.
This PR relies on new AllFields method from Genkit which returns fields
from both request object and request body object.
## Changes
If before running an app, the app was stopped with an active deployment,
then Apps backend start it and does the auto-deploy of the last active
deployment. In such cases StartApp API won't return any active or
pending deployments in its response but doing the deploy immediately
after the start might result in the error `Cannot deploy app *** as
there is an active deployment in progress`.
From DABs side, we have to do a new deployment on every `bundle run`
(command which start the app and does deployment) because local files in
bundle might have been changed and users expect to have the app running
with new code.
Thus this PR works around the error by catching “deployment in progress”
error, getting any active / pending deployments, waits for them to
finish and start the new deployment again. If 2nd attempts fails, the
whole command fails.
## Tests
Added unit test.
## Changes
Replacement was split between the type `ReplacementContext` and the
`ReplaceOutput` function. The latter also ran a couple of regular
expressions. This change consolidates them such that it is up to the
caller to compose the set of replacements to use.
This change is required to accommodate UUID replacement in #2146.
## Changes
When setting up PATH in tests, put desired entry first but keep the rest
as well. Otherwise it fails on Windows
```
D:/a/cli/cli/libs/exec/exec_test.go:108
Error: Received unexpected error:
exit status 0xc0000135
```
Explanation from Claude:
> The error code 0xc0000135 is a Windows error indicating "Unable to
locate DLL"
> When code coverage is enabled, Go instruments the binary with coverage
tracking code, which requires additional DLL dependencies on Windows.
## Tests
Separate draft PR with coverage enabled on CI:
https://github.com/databricks/cli/pull/2141
The current default of building a binary is not frequently used.
The new default is useful after switching branches, stashing/unstashing,
AI changes.
Note "fmt" and "lint" are separate steps because "fmt" can fix
formatting and imports in presence of compilation errors and "lint"
cannot. Without compilation errors, "lint" also does formatting.
This test checks load git details functionality + variable interpolation
there.
The variables are not working there because LoadGitDetails mutator is
running before variable interpolation.
Additionally, correctly replace tmp path that is used for DATABRICKS_TF_EXEC_PATH
## Changes
Use name format "TestAccept/bundle/variables/host" (previously slashes
were reversed on Windows).
## Tests
Manually run "go test ./acceptance -v -run
TestAccept/bundle/variables/host" in Windows VM.
## Changes
To accommodate:
* Add the server URL to the set of output replacements
* Include a call to the permissions API to the dummy server
* Run the main script in a subshell to isolate working directory changes
## Changes
Always include both verbatim and json version of replacement.
This helps when the string in question contains \\ or other chars that
need to be quoted.
Needed for https://github.com/databricks/cli/pull/2118
## Tests
Existing tests.
## Changes
Add two new make commands:
- make acc-cover: runs acceptance tests and outputs
coverage-acceptance.txt
- make acc-showcover: show coverage-acceptance.txt locally in browser
Using the GOCOVERDIR functionality:
https://go.dev/blog/integration-test-coverage
This works, but there are a couple of issues encountered:
- GOCOVERDIR does not play well with regular "go test -cover". Once this
fixed, we can simplify the code and have 'make cover' output coverage
for everything at once. We can also probably get rid of CLI_GOCOVERDIR.
https://github.com/golang/go/issues/66225
- When running tests in parallel to the same directory there is rare
conflict on writing covmeta file. For this reason each tests writes
coverage to their own directory which is then merged together by 'make
acc-cover'.
<!-- Summary of your changes that are easy to understand --
## Tests
Manually running the new make commands.
## Changes
- If CLOUD_ENV variable is set, acceptance will no longer set up server
& override DATABRICKS_HOST/DATABRICKS_TOKEN/HOME env vars.
- I've updated replacements logic in testdiff to use tester /
tester@databricks.com convention.
## Tests
Manually running current acceptance tests against dogfood on my laptop I
get all test pass except for 2 failures.
```
--- FAIL: TestAccept/bundle/variables/env_overrides (0.09s)
--- FAIL: TestAccept/bundle/variables/resolve-builtin (1.30s)
```
## Changes
We perform a check during path translation that the path being
referenced is contained in the bundle's sync root. If it isn't, it's not
a valid remote reference. However, this doesn't apply to paths that are
_always_ local, such as the artifact path. An artifact's build command
is executed in its path. Files created by the artifact build (e.g.
wheels or JARs) don't need to be in the sync root because they have a
dedicated and different upload path into `${workspace.artifact_path}`.
Therefore, this check that a path is contained in the bundle's sync root
doesn't apply to artifact paths. This change modifies the structure of
path translation to allow opting out of this check.
Fixes#1927.
## Tests
* Existing and new tests pass.
* Manually confirmed that building and using a wheel built outside the
sync root path works as expected.
* No acceptance tests because we don't run build as part of validate.
## Changes
As of the clusters API v2.1 the results include system clusters. On
large workspaces this can lead to long load times and include many
irrelevant results. The cluster picker should only show interactive
clusters.
Also see #1754.
## Tests
Manually confirmed the picker runs fast on a large workspace.
## Changes
Now it's possible to configure new `app` resource in bundle and point it
to the custom `source_code_path` location where Databricks App code is
defined.
On `databricks bundle deploy` DABs will create an app. All consecutive
`databricks bundle deploy` execution will update an existing app if
there are any updated
On `databricks bundle run <my_app>` DABs will execute app deployment. If
the app is not started yet, it will start the app first.
### Bundle configuration
```
bundle:
name: apps
variables:
my_job_id:
description: "ID of job to run app"
lookup:
job: "My Job"
databricks_name:
description: "Name for app user"
additional_flags:
description: "Additional flags to run command app"
default: ""
my_app_config:
type: complex
description: "Configuration for my Databricks App"
default:
command:
- flask
- --app
- hello
- run
- ${var.additional_flags}
env:
- name: DATABRICKS_NAME
value: ${var.databricks_name}
resources:
apps:
my_app:
name: "anester-app" # required and has to be unique
description: "My App"
source_code_path: ./app # required and points to location of app code
config: ${var.my_app_config}
resources:
- name: "my-job"
description: "A job for app to be able to run"
job:
id: ${var.my_job_id}
permission: "CAN_MANAGE_RUN"
permissions:
- user_name: "foo@bar.com"
level: "CAN_VIEW"
- service_principal_name: "my_sp"
level: "CAN_MANAGE"
targets:
dev:
variables:
databricks_name: "Andrew (from dev)"
additional_flags: --debug
prod:
variables:
databricks_name: "Andrew (from prod)"
```
### Execution
1. `databricks bundle deploy -t dev`
2. `databricks bundle run my_app -t dev`
**If app is started**
```
✓ Getting the status of the app my-app
✓ App is in RUNNING state
✓ Preparing source code for new app deployment.
✓ Deployment is pending
✓ Starting app with command: flask --app hello run --debug
✓ App started successfully
You can access the app at <app-url>
```
**If app is not started**
```
✓ Getting the status of the app my-app
✓ App is in UNAVAILABLE state
✓ Starting the app my-app
✓ App is starting...
....
✓ App is starting...
✓ App is started!
✓ Preparing source code for new app deployment.
✓ Downloading source code from /Workspace/Users/...
✓ Starting app with command: flask --app hello run --debug
✓ App started successfully
You can access the app at <app-url>
```
## Tests
Added unit and config tests + manual test.
```
--- PASS: TestAccDeployBundleWithApp (404.59s)
PASS
coverage: 36.8% of statements in ./...
ok github.com/databricks/cli/internal/bundle 405.035s coverage: 36.8% of statements in ./...
```
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.27.0 to
0.28.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="40b02d69cd"><code>40b02d6</code></a>
go.mod: update golang.org/x dependencies</li>
<li>See full diff in <a
href="https://github.com/golang/term/compare/v0.27.0...v0.28.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from
0.24.0 to 0.25.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="49a531d12a"><code>49a531d</code></a>
all: make method and struct comments match the names</li>
<li>See full diff in <a
href="https://github.com/golang/oauth2/compare/v0.24.0...v0.25.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Changes
Move mutator.Merge{JobClusters,JobParameters,JobTasks,PipelineClusters}
after variable resolution. This helps with the case when key contains a
variable.
@pietern mentioned here
https://github.com/databricks/cli/pull/2101#pullrequestreview-2539168762
it should be safe.
## Tests
Existing acceptance that was capturing the bug is updated with corrected
output.
## Changes
This updates `mode: production` to allow `root_path` to indicate
uniqueness. Historically, we required `run_as` for this, which isn't
actually very effective for that purpose. `run_as` also had the problem
that it doesn't work for pipelines.
This is a cherry-pick from https://github.com/databricks/cli/pull/1387
---------
Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com>
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 4
to 5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/astral-sh/setup-uv/releases">astral-sh/setup-uv's
releases</a>.</em></p>
<blockquote>
<h2>v5.0.0 🎄 Merry Christmas - Help fastly and users by default</h2>
<h2>Changes</h2>
<p>This christmans 🎄 release is a bit early bit still full of presents 🎁
Since we are changing some of the defaults this can lead to breaking
changes, thus the major version increase.</p>
<p>Here are the highlights:</p>
<h3><a
href="https://redirect.github.com/astral-sh/setup-uv/pull/193">Default
to enable-cache: true on GitHub hosted runners</a></h3>
<p>Did you know that that Fastly, the company hosting PyPI,
theoretically has to pay $12.5 million per month and so far have served
more than 2.41 <strong>exabytes</strong> of data?
<img
src="https://github.com/user-attachments/assets/f2f6cb3f-68f6-4e37-abb1-d3bf1f278533"
alt="image" /></p>
<p>This is why <a
href="https://redirect.github.com/astral-sh/setup-uv/issues/54">they
asked us</a> to turn on caching by default. After weighting the pros and
cons we decided to automatically upload the cache to the GitHub Actions
cache when running on GitHub hosted runners. You can still disable that
with <code>enable-cache: false</code>.</p>
<p>I remember when I first got into actions and didn't understand all
the magic. I was baffled that some actions did something behind the
scenes to make everything faster. I hope with this change we help a lot
of users who are don't want to or are afraid to understand what
<code>enable-cache</code> does.</p>
<h3><a
href="https://redirect.github.com/astral-sh/setup-uv/pull/185">Add
**/requirements*.txt to default cache-dependency-glob</a></h3>
<p>If caching is enabled we automatically searched for a
<code>uv.lock</code> file and when this changed we knew we had to
refresh the cache. A lot of projects don't use this but rather the good
old <code>requirements.txt</code>. We now automatically search for both
<code>uv.lock</code>and <code>requirements*.txt</code> (this means also
<code>requirements-test.txt</code>, <code>requirements-dev.txt</code>,
...) files.
You can change this with <code>cache-dependency-glob</code></p>
<h3><a
href="https://redirect.github.com/astral-sh/setup-uv/pull/194">Auto
activate venv when python-version is set</a></h3>
<p>Some workflows install packages on the fly. This automatically works
when using a python version that is already present on the runner. But
if uv installs the version, e.g. because it is a free-threaded version
or an old one, it is a <a
href="https://astral.sh/blog/python-build-standalone">standalone-build</a>
and installing packages "into the system" is not possible.</p>
<p>We now automatically create a new virtual environment with <code>uv
venv</code> and activate it for the rest of the workflow if
<code>python-version</code> is used. This means you can now do</p>
<pre lang="yaml"><code>- name: Install uv
uses: astral-sh/setup-uv@auto-environment
with:
python-version: 3.13t
- run: uv pip install -i
https://pypi.anaconda.org/scientific-python-nightly-wheels/simple cython
</code></pre>
<h2>🚨 Breaking changes</h2>
<ul>
<li>Default to enable-cache: true on GitHub hosted runners <a
href="https://github.com/eifinger"><code>@eifinger</code></a> (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/193">#193</a>)</li>
<li>Add **/requirements*.txt to default cache-dependency-glob <a
href="https://github.com/eifinger"><code>@eifinger</code></a> (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/185">#185</a>)</li>
</ul>
<h2>🐛 Bug fixes</h2>
<ul>
<li>Always use api.github.com <a
href="https://github.com/eifinger"><code>@eifinger</code></a> (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/191">#191</a>)</li>
</ul>
<h2>🚀 Enhancements</h2>
<ul>
<li>Auto activate venv when python-version is set <a
href="https://github.com/eifinger"><code>@eifinger</code></a> (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/194">#194</a>)</li>
<li>Add python version to cache key <a
href="https://github.com/eifinger"><code>@eifinger</code></a> (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/187">#187</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="887a942a15"><code>887a942</code></a>
Set VIRTUAL_ENV to .venv instead of .venv/bin (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/210">#210</a>)</li>
<li><a
href="d174a24c07"><code>d174a24</code></a>
Align use of <code>actions/setup-python</code> with uv docu (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/207">#207</a>)</li>
<li><a
href="12c852e6ba"><code>12c852e</code></a>
Remove uv version from cache key (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/206">#206</a>)</li>
<li><a
href="180f8b4439"><code>180f8b4</code></a>
Fix wrong cacheDependencyPathHash (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/201">#201</a>)</li>
<li><a
href="e3fb95a689"><code>e3fb95a</code></a>
Warn instead of fail for no-dependency-glob (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/200">#200</a>)</li>
<li><a
href="2af22b5b2d"><code>2af22b5</code></a>
chore: update known checksums for 0.5.11 (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/198">#198</a>)</li>
<li><a
href="dd578776bb"><code>dd57877</code></a>
Auto activate venv when python-version is set (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/194">#194</a>)</li>
<li><a
href="85aa0bf0c1"><code>85aa0bf</code></a>
chore: update known checksums for 0.5.10 (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/196">#196</a>)</li>
<li><a
href="1f2cbfa7bb"><code>1f2cbfa</code></a>
Bump <code>@types/node</code> from 22.10.1 to 22.10.2 (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/189">#189</a>)</li>
<li><a
href="25b3ce6330"><code>25b3ce6</code></a>
chore: update known checksums for 0.5.9 (<a
href="https://redirect.github.com/astral-sh/setup-uv/issues/195">#195</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/astral-sh/setup-uv/compare/v4...v5">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Changes
The main CLI entry point used to be a global variable, and the global
state had to be cleaned up after every test run. This hasn't been the
case for a while, and instead, the CLI is initialized in a function
call. State accumulated by a single CLI "instance" can no longer leak
into other instances, so we no longer have to perform cleanup.
## Tests
Existing tests pass.
See Makefile for explanation on difference between 'make fmt' and 'make
lint'.
I also removed lint.sh. Original motivation was to use it in aider, but
it's not a good fit there, because aider passes filenames and it does
not work well with most golang linters which requires whole packages to
work.
Follow up to #2062, #2056, #2051.
Giving 7 days to react before closing is too aggressive, IMO. Changed it
to 30.
Also changed 'stale' label from 30d to 60d.
Also removed dry-run setting, it does not appear to do anything.
## Changes
Previously diagnostics were not seen in JSON output mode. This change
prints them to stderr.
This also fixes acceptance tests to preprocess all output with
s/execPath/$CLI/ not just output.txt.
## Tests
Existing acceptance tests. In one case I've added non-json command to
check that they match in output.
## Changes
This is useful to track telemetry associated with the templates and can
later be useful for functional usecases as well. Mlops stacks does the
same here: https://github.com/databricks/mlops-stacks/pull/185
## Tests
Existing tests.
## Changes
This workflow only worked if it was triggered on the tag to publish
itself. This means it is not possible to release a version if the
workflow configuration at that tag is broken (as is the case for
v0.238.0 because of #2105).
This change adds a "tag" input that can be set when manually triggering
the workflow.
## Tests
* Succesful run with this change:
https://github.com/databricks/cli/actions/runs/12689281843
* Pull request that the run created:
https://github.com/microsoft/winget-pkgs/pull/209220
## Changes
This action uses a token to access the release artifacts and, as such,
needs to execute on the runner that's on the allowlist.
Related PRs:
* #2098
* #2077
Since merge happens first, before variable resolution, the two jobs are
seen as different.
I also updated override/job_cluster/script to include more of the
output.
It's easier to remember and type and also validated and part of help:
```
% go test ./acceptance -updat 2>&1 | grep updat
flag provided but not defined: -updat
-update
```
Denis Bilenko
Andrew Nester
Pieter Noordhuis
Gleb Kanterov
dependabot[bot]
Lennart Kats (databricks)
shreyas-goenka