## Changes
1. Removes default yaml-fields during schema generation, caused by [this
PR](https://github.com/databricks/cli/pull/2032) (current yaml package
can't read `json` annotations in struct fields)
2. Addresses missing annotations for fields from OpenAPI spec, which are
named differently in go SDK
3. Adds filtering for annotations.yaml to include only CLI package
fields
4. Implements alphabetical sort for yaml keys to avoid unnecessary diff
in PRs
## Tests
Manually tested
## Changes
Add new type of test helpers that run the command and compare full
output (golden files approach).
In case of JSON, there is also an option to ignore certain paths.
Add test for different versions of Python to go through bundle init
default-python / validate / deploy / summary.
## Tests
New integration tests.
## Changes
Simplify logic for selecting Python to run when calculating default whl
build command: "python" on Windows and "python3" everywhere.
Python installers from python.org do not install python3.exe. In
virtualenv there is no python3.exe.
## Tests
Added new unit tests to create real venv with uv and simulate activation
by prepending venv/bin to PATH.
## Changes
The existing workflow already had 2 trigger conditions, so instead of
adding a third (and seeing more "skipped" jobs), I split them up into
dedicated workflow files, each with their own trigger condition.
The integration test status is reported back via commit status.
## Tests
We can confirm that everything works as expected as this PR moves from
here to the merge group to main.
## Changes
I noticed that #1957 took a dep on this library even though we no longer
need it. This change removes the dep and cleans up other (unused) uses
of the library. We originally relied on this library to deserialize
bundle configuration and JSON payloads to non-bundle CLI commands.
Relevant commits:
* The YAML flag was added to support apps (very early), and is not
longer used: e408b701
* First use for bundle configuration loading: e47fa619
* Switch bundle configuration loading to use `libs/dyn`: 87dd46a3
## Tests
The build works without the dependency.
## Changes
* Added support for `IsSingleNode`, `Kind` and `UseMlRuntime` for
clusters
* Added support for `CleanRoomsNotebookTask`
* `DaysOfWeek` for pipeline restart window is now a list
## Changes
Adds annotations to json-schema for fields which are not covered by
OpenAPI spec.
Custom descriptions were copy-pasted from documentation PR which is
still WIP so descriptions for some fields are missing
Further improvements:
* documentation autogen based on json-schema
* fix missing descriptions
## Tests
This script is not part of CLI package so I didn't test all corner
cases. Few high-level tests were added to be sure that schema
annotations is in sync with actual config
---------
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
## Changes
If there's required query params, it is a top-level field of request
object and not a field of nested request body.
This is needed for upcoming changes from OpenAPI spec changes where such
query parameters is introduced.
No changes after regenerating CLI with current spec and the fix (appears
we haven't had such params before)
## Changes
There was only one helper for AWS and not the other clouds. Found this
when looking through double calls to `acc.WorkspaceTest()` (see
`TestPythonWheelTaskDeployAndRunOnInteractiveCluster`).
## Tests
n/a
## Changes
A small subset of tests trigger cluster creation to run jobs. These
tests comprise a substantial amount of the total integration test
runtime. We can skip them on PRs and only run them on the main branch.
## Tests
Confirmed the short runtime is ~20 mins.
Remove unnecessary make command and github workflow - it's a subset of
"lint" now. However, keep "mod tidy" separately, don't think the linter
does that.
## Changes
The `acc` package is exclusively used by integration tests, so it
belongs under `integration/internal`.
It's not the best name we can rename later.
## Tests
n/a
## Changes
Fixed downloading arm64 binaries
Go 1.23 changed the way built binaries are prefixed on amd64, more
details here: https://tip.golang.org/doc/go1.23#arm64
## Changes
These test helpers were updated to handle the error internally and not
return it. Since they have testing.T object, they can do so directly. On
the caller side, this functions were always followed by
require.NoError(t, err), that was cleaned up.
This approach helps reduce the setup/teardown boilerplate in the test
cases.
## Tests
Existing tests.
## Changes
The `Setenv` helper function configures an environment variable and
resets it to its original value when exiting the test scope. It is
incompatible with running tests in parallel because it modifies
process-wide state. The `libs/env` package defines functions to interact
with the environment but records `Setenv` calls on a `context.Context`.
This enables us to override/specialize the environment scoped to a
context.
Pre-requisites for removing the `t.Setenv` calls:
* Make `cmdio.NewIO` accept a context and use it with `libs/env`
* Make all `internal/testcli` functions use a context
The rest of this change:
* Modifies integration tests to initialize a context to use if there
wasn't already one
* Updates `t.Setenv` calls to use `env.Set`
## Tests
n/a
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from
0.30.0 to 0.31.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b4f1988a35"><code>b4f1988</code></a>
ssh: make the public key cache a 1-entry FIFO cache</li>
<li>See full diff in <a
href="https://github.com/golang/crypto/compare/v0.30.0...v0.31.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/databricks/cli/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from
0.24.0 to 0.31.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b4f1988a35"><code>b4f1988</code></a>
ssh: make the public key cache a 1-entry FIFO cache</li>
<li><a
href="7042ebcbe0"><code>7042ebc</code></a>
openpgp/clearsign: just use rand.Reader in tests</li>
<li><a
href="3e90321ac7"><code>3e90321</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="8c4e668694"><code>8c4e668</code></a>
x509roots/fallback: update bundle</li>
<li><a
href="6018723c74"><code>6018723</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="71ed71b4fa"><code>71ed71b</code></a>
README: don't recommend go get</li>
<li><a
href="750a45fe5e"><code>750a45f</code></a>
sha3: add MarshalBinary, AppendBinary, and UnmarshalBinary</li>
<li><a
href="36b172546b"><code>36b1725</code></a>
sha3: avoid trailing permutation</li>
<li><a
href="80ea76eb17"><code>80ea76e</code></a>
sha3: fix padding for long cSHAKE parameters</li>
<li><a
href="c17aa50fbd"><code>c17aa50</code></a>
sha3: avoid buffer copy</li>
<li>Additional commits viewable in <a
href="https://github.com/golang/crypto/compare/v0.24.0...v0.31.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/databricks/cli/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Changes
VSCode extension no longer uses `databricks.python.envFile ` setting.
And older extension versions will use the same default value anyway.
## Tests
None
## Changes
These calls are no longer necessary now that integration tests use a
main function that performs this check. This change updates integration
tests that call this function. Of those, the call sites that initialize
a workspace client are updated to use `acc.WorkspaceTest(t)` to get one.
## Tests
n/a
Andrew Nester
Ilya Kuznetsov
Denis Bilenko
Pieter Noordhuis
dependabot[bot]
Ilia Babanov