3d5c076a4d
jsonTags[0] -> fieldName
2024-09-04 10:40:04 +02:00
176ced190e
add tests for bfs assertion
2024-09-04 10:26:47 +02:00
35cdf4010d
Do not error if we cannot prompt for a profile in `auth login` ( #1745 )
...
## Changes
With https://github.com/databricks/cli/pull/1370 we started to error if
a profile name was not provided in a non-tty setting. The Databricks
VSCode extension, however, uses the `auth login` command to simply
refresh the tokens. Thus, this PR is a regression fix for that use case.
## Tests
Manually, `databricks auth login --host
https://e2-dogfood.staging.cloud.databricks.com ` no longer errors.
Instead it successfully refreshes the credentials.
2024-09-04 07:14:21 +00:00
578019ba22
-
2024-09-03 20:14:02 +02:00
072fa812e2
Include a permissions section in all templates ( #1713 )
...
## Changes
This updates the templates to include a `permissions` section. Having a
permissions section is a best practice, is helpful to understand the
notion of permissions, and helps diagnose permission errors
(https://github.com/databricks/cli/pull/1386 ).
This is a cherry-pick from https://github.com/databricks/cli/pull/1387 .
This change was verified to work both in dev and prod. Existing unit
tests validate the validity of the templates in these modes.
2024-09-03 07:51:54 +00:00
096123674a
Fix streaming of stdout, stdin, stderr in cobra test runner ( #1742 )
...
## Changes
We were not using the readers and writers set in the test fixtures in
the progress logger. This PR fixes that. It also modifies
`TestAccAbortBind`, which was implicitly relying on the bug.
I encountered this bug while working on
https://github.com/databricks/cli/pull/1672 .
## Tests
Manually.
From non-tty:
```
Error: failed to bind the resource, err: This bind operation requires user confirmation, but the current console does not support prompting. Please specify --auto-approve if you would like to skip prompts and proceed.
```
From tty, bind works as expected.
```
Confirm import changes? Changes will be remotely applied only after running 'bundle deploy'. [y/n]: y
Updating deployment state...
Successfully bound databricks_pipeline with an id '9d2dedbb-f522-4503-96ba-4bc4d5bfa77d'. Run 'bundle deploy' to deploy changes to your workspace
```
2024-09-02 13:43:17 +00:00
0ce7be8ff0
Bump github.com/Masterminds/semver/v3 from 3.2.1 to 3.3.0 ( #1741 )
...
Bumps
[github.com/Masterminds/semver/v3](https://github.com/Masterminds/semver )
from 3.2.1 to 3.3.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/Masterminds/semver/releases ">github.com/Masterminds/semver/v3's
releases</a>.</em></p>
<blockquote>
<h2>v3.3.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Fix: bad package in README by <a
href="https://github.com/sdelicata "><code>@sdelicata</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/226 ">Masterminds/semver#226</a></li>
<li>Updating the GitHub Actions and versions of Go used by <a
href="https://github.com/mattfarina "><code>@mattfarina</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/229 ">Masterminds/semver#229</a></li>
<li>Fix spelling in README by <a
href="https://github.com/robinschneider "><code>@robinschneider</code></a>
in <a
href="https://redirect.github.com/Masterminds/semver/pull/222 ">Masterminds/semver#222</a></li>
<li>Adding go build cache to fuzz output by <a
href="https://github.com/mattfarina "><code>@mattfarina</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/232 ">Masterminds/semver#232</a></li>
<li>Add caching to fuzz testing by <a
href="https://github.com/mattfarina "><code>@mattfarina</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/234 ">Masterminds/semver#234</a></li>
<li>updating github actions by <a
href="https://github.com/mattfarina "><code>@mattfarina</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/235 ">Masterminds/semver#235</a></li>
<li>feat: nil version equality by <a
href="https://github.com/KnutZuidema "><code>@KnutZuidema</code></a> in
<a
href="https://redirect.github.com/Masterminds/semver/pull/213 ">Masterminds/semver#213</a></li>
<li>add >= and <= by <a
href="https://github.com/grosser "><code>@grosser</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/238 ">Masterminds/semver#238</a></li>
<li>doc: hyphen range constraint without whitespace by <a
href="https://github.com/johnnychen94 "><code>@johnnychen94</code></a>
in <a
href="https://redirect.github.com/Masterminds/semver/pull/216 ">Masterminds/semver#216</a></li>
<li>Removing reference to vert by <a
href="https://github.com/mattfarina "><code>@mattfarina</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/245 ">Masterminds/semver#245</a></li>
<li>simplify StrictNewVersion by <a
href="https://github.com/grosser "><code>@grosser</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/241 ">Masterminds/semver#241</a></li>
<li>Updating the testing version of Go used by <a
href="https://github.com/mattfarina "><code>@mattfarina</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/246 ">Masterminds/semver#246</a></li>
<li>bumping min version in go.mod based on what's tested by <a
href="https://github.com/mattfarina "><code>@mattfarina</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/248 ">Masterminds/semver#248</a></li>
<li>Updating changelog for 3.3.0 by <a
href="https://github.com/mattfarina "><code>@mattfarina</code></a> in <a
href="https://redirect.github.com/Masterminds/semver/pull/249 ">Masterminds/semver#249</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/sdelicata "><code>@sdelicata</code></a>
made their first contribution in <a
href="https://redirect.github.com/Masterminds/semver/pull/226 ">Masterminds/semver#226</a></li>
<li><a
href="https://github.com/robinschneider "><code>@robinschneider</code></a>
made their first contribution in <a
href="https://redirect.github.com/Masterminds/semver/pull/222 ">Masterminds/semver#222</a></li>
<li><a
href="https://github.com/KnutZuidema "><code>@KnutZuidema</code></a>
made their first contribution in <a
href="https://redirect.github.com/Masterminds/semver/pull/213 ">Masterminds/semver#213</a></li>
<li><a href="https://github.com/grosser "><code>@grosser</code></a> made
their first contribution in <a
href="https://redirect.github.com/Masterminds/semver/pull/238 ">Masterminds/semver#238</a></li>
<li><a
href="https://github.com/johnnychen94 "><code>@johnnychen94</code></a>
made their first contribution in <a
href="https://redirect.github.com/Masterminds/semver/pull/216 ">Masterminds/semver#216</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/Masterminds/semver/compare/v3.2.1...v3.3.0 ">https://github.com/Masterminds/semver/compare/v3.2.1...v3.3.0 </a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/Masterminds/semver/blob/master/CHANGELOG.md ">github.com/Masterminds/semver/v3's
changelog</a>.</em></p>
<blockquote>
<h2>3.3.0 (2024-08-27)</h2>
<h3>Added</h3>
<ul>
<li><a
href="https://redirect.github.com/Masterminds/semver/issues/238 ">#238</a>:
Add LessThanEqual and GreaterThanEqual functions (thanks <a
href="https://github.com/grosser "><code>@grosser</code></a>)</li>
<li><a
href="https://redirect.github.com/Masterminds/semver/issues/213 ">#213</a>:
nil version equality checking (thanks <a
href="https://github.com/KnutZuidema "><code>@KnutZuidema</code></a>)</li>
</ul>
<h3>Changed</h3>
<ul>
<li><a
href="https://redirect.github.com/Masterminds/semver/issues/241 ">#241</a>:
Simplify StrictNewVersion parsing (thanks <a
href="https://github.com/grosser "><code>@grosser</code></a>)</li>
<li>Testing support up through Go 1.23</li>
<li>Minimum version set to 1.21 as this is what's tested now</li>
<li>Fuzz testing now supports caching</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="e6e3d4d3cbhttps://redirect.github.com/Masterminds/semver/issues/249 ">#249</a>
from mattfarina/update-changelog-3.3.0</li>
<li><a
href="e80c4ea72380427ad56ehttps://redirect.github.com/Masterminds/semver/issues/248 ">#248</a>
from mattfarina/bump-min-version</li>
<li><a
href="b610837227a4cccd8ea5https://redirect.github.com/Masterminds/semver/issues/246 ">#246</a>
from mattfarina/bump-go-1.23</li>
<li><a
href="7c178cf0c629f94c1119https://redirect.github.com/Masterminds/semver/issues/241 ">#241</a>
from grosser/grosser/validate</li>
<li><a
href="2cf1b16b95https://redirect.github.com/Masterminds/semver/issues/245 ">#245</a>
from mattfarina/remove-vert</li>
<li><a
href="b55476af0ed07450b7cfhttps://github.com/Masterminds/semver/compare/v3.2.1...v3.3.0 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-02 12:39:16 +00:00
ed448815b4
PythonMutator: explain missing package error ( #1736 )
...
## Changes
Explain the error when the `databricks-pydabs` package is not installed
or the Python environment isn't correctly activated.
Example output:
```
Error: python mutator process failed: ".venv/bin/python3 -m databricks.bundles.build --phase load --input .../input.json --output .../output.json --diagnostics .../diagnostics.json: exit status 1", use --debug to enable logging
.../.venv/bin/python3: Error while finding module specification for 'databricks.bundles.build' (ModuleNotFoundError: No module named 'databricks')
Explanation: 'databricks-pydabs' library is not installed in the Python environment.
If using Python wheels, ensure that 'databricks-pydabs' is included in the dependencies,
and that the wheel is installed in the Python environment:
$ .venv/bin/pip install -e .
If using a virtual environment, ensure it is specified as the venv_path property in databricks.yml,
or activate the environment before running CLI commands:
experimental:
pydabs:
venv_path: .venv
```
## Tests
Unit tests
2024-09-02 09:49:30 +00:00
582558cac2
Do not suppress normalisation diagnostics for resolving variables ( #1740 )
...
## Changes
Tested on the following bundle configuration
```
bundle:
name: clusters
mode: development
variables:
webhook_notifications:
description: Webhook URL for notifications
type: complex
default:
on_failure:
id: 6a6c04c1-389c-4534-95af-b68b62a9dbe6
resources:
jobs:
test_job:
name: "Andrew Nester Test Job"
tasks:
- task_key: test_task
notebook_task:
notebook_path: "./src/test.py"
new_cluster:
num_workers: 2
node_type_id: "i3.xlarge"
autoscale:
min_workers: 2
max_workers: 7
spark_version: "12.2.x-scala2.12"
spark_conf:
"spark.executor.memory": "2g"
webhook_notifications: ${var.webhook_notifications}
```
bundle validate output is below
```
andrew.nester@HFW9Y94129 wheel % databricks bundle validate
Warning: expected sequence, found map
at resources.jobs.test_job.webhook_notifications.on_failure
in bundle.yml:11:9
Name: clusters
Target: default
Workspace:
User: andrew.nester@databricks.com
Path: /Users/andrew.nester@databricks.com/.bundle/clusters/default
```
**Note** that error correctly points to the variable
2024-09-02 09:17:18 +00:00
5d9910c8e0
Make lock optional in the JSON schema ( #1738 )
...
Fixes https://github.com/databricks/cli/issues/1561
2024-09-02 08:39:08 +00:00
70ce802518
PythonMutator: preserve normalize diagnostics ( #1735 )
...
## Changes
Preserve diagnostics if there are any errors or warnings when
PythonMutator normalizes output. If anything goes wrong during
conversion, diagnostics contain the relevant location and path.
## Tests
Unit tests
2024-08-30 13:29:00 +00:00
ed4a4585c0
Update templates to latest LTS DBR ( #1715 )
...
## Changes
This updates the templates to use the latest DBR 15 LTS version.
- [x] DB Connect 15.4 must be released + validated before this can go
out
2024-08-30 07:32:10 +00:00
2e000f1ebd
Use materialized views in the default-sql template ( #1709 )
...
## Changes
Materialized views now support `CREATE OR REPLACE`
([docs](https://docs.databricks.com/en/sql/language-manual/sql-ref-syntax-ddl-create-materialized-view.html ))!
This makes it possible to use them with Workflows in DABs.This PR
updates the template to use a materialized view rather than a regular
view.
## Tests
Manually validated in production.
2024-08-29 19:07:21 +00:00
5fac7edcdf
Pass along $AZURE_CONFIG_FILE to Terraform process ( #1734 )
...
## Changes
This ensures that the CLI and Terraform can both use an Azure CLI
session configured under a non-standard path. This is the default
behavior on Azure DevOps when using the AzureCLI@2 task.
Fixes #1722 .
## Tests
Unit test.
2024-08-29 14:41:12 +00:00
0f4891f0fe
Add `dyn.Time` to box a timestamp with its original string value ( #1732 )
...
## Changes
If not explicitly quoted, the YAML loader interprets a value like
`2024-08-29` as a timestamp. Such a value is usually intended to be a
string instead. Our normalization logic was not able to turn a time
value back into the original string.
This change boxes the time value to include its original string
representation. Normalization of one of these values into a string can
now use the original input value.
## Tests
Unit tests in `libs/dyn/convert`.
2024-08-29 13:02:34 +00:00
43ace69bb9
Consider serverless clusters as compatible for Python wheel tasks ( #1733 )
...
## Changes
Consider serverless clusters as compatible for Python wheel tasks.
Fixes a `Python wheel tasks require compute with DBR 13.3+ to include
local libraries` warning shown for serverless clusters
2024-08-29 12:47:44 +00:00
7dcc791b05
[Release] Release v0.227.1 ( #1729 )
...
CLI:
* Disable prompt for storage-credentials get command
([#1723 ](https://github.com/databricks/cli/pull/1723 )).
Bundles:
* Do not treat empty path as a local path
([#1717 ](https://github.com/databricks/cli/pull/1717 )).
* Correctly mark PyPI package name specs with multiple specifiers as
remote libraries ([#1725 ](https://github.com/databricks/cli/pull/1725 )).
* Improve error handling for /Volumes paths in mode: development
([#1716 ](https://github.com/databricks/cli/pull/1716 )).
Internal:
* Ignore CLI version check on development builds of the CLI
([#1714 ](https://github.com/databricks/cli/pull/1714 )).
API Changes:
* Added `databricks resource-quotas` command group.
* Added `databricks policy-compliance-for-clusters` command group.
* Added `databricks policy-compliance-for-jobs` command group.
OpenAPI commit 3eae49b444cac5a0118a3503e5b7ecef7f96527a (2024-08-21)
Dependency updates:
* Bump github.com/databricks/databricks-sdk-go from 0.44.0 to 0.45.0
([#1719 ](https://github.com/databricks/cli/pull/1719 )).
* Revert hc-install version to 0.7.0
([#1711 ](https://github.com/databricks/cli/pull/1711 )).
2024-08-28 13:09:07 +00:00
85459c1963
Improve error handling for /Volumes paths in mode: development ( #1716 )
...
## Changes
* Provide a more helpful error when using an artifact_path based on
/Volumes
* Allow the use of short_names in /Volumes paths
## Example cases
Example of a valid /Volumes artifact_path:
* `artifact_path:
/Volumes/catalog/schema/${workspace.current_user.short_name}/libs`
Example of an invalid /Volumes path (when using `mode: development`):
* `artifact_path: /Volumes/catalog/schema/libs`
* Resulting error: `artifact_path should contain the current username or
${workspace.current_user.short_name} to ensure uniqueness when using
'mode: development'`
2024-08-28 12:14:19 +00:00
70363836d5
Correctly mark PyPI package name specs with multiple specifiers as remote libraries ( #1725 )
...
Correctly mark pypi package name specs with multiple specifiers as
remote libraries
Fixes this https://github.com/databricks/cli/issues/1728
2024-08-28 11:39:06 +00:00
cb8d6a9f60
self pass 1
2024-08-27 19:50:43 +02:00
4141f4ea34
-
2024-08-27 17:49:09 +02:00
aac6687900
remove more todos
2024-08-27 17:46:57 +02:00
40f4d35c4a
cleanup todos
2024-08-27 17:45:20 +02:00
ad7503a176
cleanup todos
2024-08-27 17:40:36 +02:00
f194a5b86a
skip adding patterns for string schemas
2024-08-27 17:36:12 +02:00
edc08149d3
Disable prompt for storage-credentials get command ( #1723 )
...
## Changes
Fixes #1079
2024-08-27 12:51:10 +00:00
acc43092ad
fix lint
2024-08-27 14:17:04 +02:00
727036bb40
bundle schema command is OK now
2024-08-27 13:46:52 +02:00
483480f4a9
-
2024-08-27 13:39:23 +02:00
2c30cfa5a2
-
2024-08-27 13:38:42 +02:00
ac60163c19
move gen file
2024-08-27 13:35:56 +02:00
5b7974757d
-
2024-08-27 13:28:03 +02:00
e7fd063e8a
-
2024-08-27 13:10:13 +02:00
fcdccb359a
cleanu unused code
2024-08-27 12:47:49 +02:00
d07192fe03
merge
2024-08-27 11:54:41 +02:00
2dea889621
delete the old schema code
2024-08-27 11:53:29 +02:00
870e41912a
from_type method is mostly complete
2024-08-27 11:42:33 +02:00
056d203236
Bump github.com/databricks/databricks-sdk-go from 0.44.0 to 0.45.0 ( #1719 )
...
Bumps
[github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go )
from 0.44.0 to 0.45.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/releases ">github.com/databricks/databricks-sdk-go's
releases</a>.</em></p>
<blockquote>
<h2>v0.45.0</h2>
<h2>0.45.0</h2>
<h3>Bug Fixes</h3>
<ul>
<li>Add INVALID_STATE to error code mapping (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1014 ">#1014</a>).</li>
<li>Do not specify <code>--tenant</code> flag when fetching managed
identity access token from the CLI (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1021 ">#1021</a>).</li>
</ul>
<h3>Internal Changes</h3>
<ul>
<li>Add terraform aliases to Entity (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1017 ">#1017</a>).</li>
<li>Added Service.NamedIdMap (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1016 ">#1016</a>).</li>
<li>Fix billing test for budget configuration update (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1019 ">#1019</a>).</li>
</ul>
<h3>API Changes:</h3>
<ul>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#PolicyComplianceForClustersAPI ">w.PolicyComplianceForClusters</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PolicyComplianceForJobsAPI ">w.PolicyComplianceForJobs</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ResourceQuotasAPI ">w.ResourceQuotas</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#GetQuotaRequest ">catalog.GetQuotaRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#GetQuotaResponse ">catalog.GetQuotaResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListQuotasRequest ">catalog.ListQuotasRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListQuotasResponse ">catalog.ListQuotasResponse</a>
and <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#QuotaInfo ">catalog.QuotaInfo</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#ClusterCompliance ">compute.ClusterCompliance</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#ClusterSettingsChange ">compute.ClusterSettingsChange</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#EnforceClusterComplianceRequest ">compute.EnforceClusterComplianceRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#EnforceClusterComplianceResponse ">compute.EnforceClusterComplianceResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#GetClusterComplianceRequest ">compute.GetClusterComplianceRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#GetClusterComplianceResponse ">compute.GetClusterComplianceResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#ListClusterCompliancesRequest ">compute.ListClusterCompliancesRequest</a>
and <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#ListClusterCompliancesResponse ">compute.ListClusterCompliancesResponse</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#EnforcePolicyComplianceForJobResponseJobClusterSettingsChange ">jobs.EnforcePolicyComplianceForJobResponseJobClusterSettingsChange</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#EnforcePolicyComplianceRequest ">jobs.EnforcePolicyComplianceRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#EnforcePolicyComplianceResponse ">jobs.EnforcePolicyComplianceResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#GetPolicyComplianceRequest ">jobs.GetPolicyComplianceRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#GetPolicyComplianceResponse ">jobs.GetPolicyComplianceResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobCompliance ">jobs.JobCompliance</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#ListJobComplianceForPolicyResponse ">jobs.ListJobComplianceForPolicyResponse</a>
and <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#ListJobComplianceRequest ">jobs.ListJobComplianceRequest</a>.</li>
<li>Added <code>Fallback</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CreateExternalLocation ">catalog.CreateExternalLocation</a>.</li>
<li>Added <code>Fallback</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ExternalLocationInfo ">catalog.ExternalLocationInfo</a>.</li>
<li>Added <code>Fallback</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateExternalLocation ">catalog.UpdateExternalLocation</a>.</li>
<li>Added <code>JobRunId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#BaseRun ">jobs.BaseRun</a>.</li>
<li>Added <code>JobRunId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Run ">jobs.Run</a>.</li>
<li>Added <code>IncludeMetrics</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#ListQueryHistoryRequest ">sql.ListQueryHistoryRequest</a>.</li>
<li>Added <code>StatementIds</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#QueryFilter ">sql.QueryFilter</a>.</li>
<li>Removed <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#ContextFilter ">sql.ContextFilter</a>.</li>
<li>Removed <code>ContextFilter</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#QueryFilter ">sql.QueryFilter</a>.</li>
<li>Removed <code>PipelineId</code> and <code>PipelineUpdateId</code>
fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#QuerySource ">sql.QuerySource</a>.</li>
</ul>
<p>OpenAPI SHA: 3eae49b444cac5a0118a3503e5b7ecef7f96527a, Date:
2024-08-21</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/blob/main/CHANGELOG.md ">github.com/databricks/databricks-sdk-go's
changelog</a>.</em></p>
<blockquote>
<h2>[Release] Release v0.45.0</h2>
<h3>Bug Fixes</h3>
<ul>
<li>Add INVALID_STATE to error code mapping (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1014 ">#1014</a>).</li>
<li>Do not specify <code>--tenant</code> flag when fetching managed
identity access token from the CLI (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1021 ">#1021</a>).</li>
</ul>
<h3>Internal Changes</h3>
<ul>
<li>Add terraform aliases to Entity (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1017 ">#1017</a>).</li>
<li>Added Service.NamedIdMap (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1016 ">#1016</a>).</li>
<li>Fix billing test for budget configuration update (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1019 ">#1019</a>).</li>
</ul>
<h3>API Changes:</h3>
<ul>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#PolicyComplianceForClustersAPI ">w.PolicyComplianceForClusters</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PolicyComplianceForJobsAPI ">w.PolicyComplianceForJobs</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ResourceQuotasAPI ">w.ResourceQuotas</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#GetQuotaRequest ">catalog.GetQuotaRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#GetQuotaResponse ">catalog.GetQuotaResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListQuotasRequest ">catalog.ListQuotasRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListQuotasResponse ">catalog.ListQuotasResponse</a>
and <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#QuotaInfo ">catalog.QuotaInfo</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#ClusterCompliance ">compute.ClusterCompliance</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#ClusterSettingsChange ">compute.ClusterSettingsChange</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#EnforceClusterComplianceRequest ">compute.EnforceClusterComplianceRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#EnforceClusterComplianceResponse ">compute.EnforceClusterComplianceResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#GetClusterComplianceRequest ">compute.GetClusterComplianceRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#GetClusterComplianceResponse ">compute.GetClusterComplianceResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#ListClusterCompliancesRequest ">compute.ListClusterCompliancesRequest</a>
and <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/compute#ListClusterCompliancesResponse ">compute.ListClusterCompliancesResponse</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#EnforcePolicyComplianceForJobResponseJobClusterSettingsChange ">jobs.EnforcePolicyComplianceForJobResponseJobClusterSettingsChange</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#EnforcePolicyComplianceRequest ">jobs.EnforcePolicyComplianceRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#EnforcePolicyComplianceResponse ">jobs.EnforcePolicyComplianceResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#GetPolicyComplianceRequest ">jobs.GetPolicyComplianceRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#GetPolicyComplianceResponse ">jobs.GetPolicyComplianceResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobCompliance ">jobs.JobCompliance</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#ListJobComplianceForPolicyResponse ">jobs.ListJobComplianceForPolicyResponse</a>
and <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#ListJobComplianceRequest ">jobs.ListJobComplianceRequest</a>.</li>
<li>Added <code>Fallback</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CreateExternalLocation ">catalog.CreateExternalLocation</a>.</li>
<li>Added <code>Fallback</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ExternalLocationInfo ">catalog.ExternalLocationInfo</a>.</li>
<li>Added <code>Fallback</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateExternalLocation ">catalog.UpdateExternalLocation</a>.</li>
<li>Added <code>JobRunId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#BaseRun ">jobs.BaseRun</a>.</li>
<li>Added <code>JobRunId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Run ">jobs.Run</a>.</li>
<li>Added <code>IncludeMetrics</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#ListQueryHistoryRequest ">sql.ListQueryHistoryRequest</a>.</li>
<li>Added <code>StatementIds</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#QueryFilter ">sql.QueryFilter</a>.</li>
<li>Removed <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#ContextFilter ">sql.ContextFilter</a>.</li>
<li>Removed <code>ContextFilter</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#QueryFilter ">sql.QueryFilter</a>.</li>
<li>Removed <code>PipelineId</code> and <code>PipelineUpdateId</code>
fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#QuerySource ">sql.QuerySource</a>.</li>
</ul>
<p>OpenAPI SHA: 3eae49b444cac5a0118a3503e5b7ecef7f96527a, Date:
2024-08-21</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="6d867882d0https://redirect.github.com/databricks/databricks-sdk-go/issues/1023 ">#1023</a>)</li>
<li><a
href="ba4489b946f6248097d1https://redirect.github.com/databricks/databricks-sdk-go/issues/1019 ">#1019</a>)</li>
<li><a
href="27a5055609https://redirect.github.com/databricks/databricks-sdk-go/issues/1017 ">#1017</a>)</li>
<li><a
href="382a38d380https://redirect.github.com/databricks/databricks-sdk-go/issues/1016 ">#1016</a>)</li>
<li><a
href="1ef9931dc9https://redirect.github.com/databricks/databricks-sdk-go/issues/1014 ">#1014</a>)</li>
<li>See full diff in <a
href="https://github.com/databricks/databricks-sdk-go/compare/v0.44.0...v0.45.0 ">compare
view</a></li>
</ul>
</details>
<br />
<details>
<summary>Most Recent Ignore Conditions Applied to This Pull
Request</summary>
| Dependency Name | Ignore Conditions |
| --- | --- |
| github.com/databricks/databricks-sdk-go | [>= 0.28.a, < 0.29] |
</details>
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Nester <andrew.nester@databricks.com>
2024-08-27 08:54:05 +00:00
11dfdc036d
more iteration
2024-08-26 20:16:45 +02:00
783e05c939
Do not treat empty path as a local path ( #1717 )
...
## Changes
Fixes issue introduced here https://github.com/databricks/cli/pull/1699
where PyPi packages were treated as local library.
The reason is that `libraryPath` returns an empty string as a path for
PyPi packages and then `IsLibraryLocal` treated empty string as local
path.
Both of these functions are fixed in this PR.
## Tests
Added regression test
2024-08-26 10:03:56 +00:00
84b47745e4
Ignore CLI version check on development builds of the CLI ( #1714 )
...
## Changes
This changes makes sure we ignore CLI version check on development
builds of the CLI.
Before:
```
$ cat databricks.yml | grep cli_version
databricks_cli_version: ">= 0.223.1"
$ cli bundle deploy
Error: Databricks CLI version constraint not satisfied. Required: >= 0.223.1, current: 0.0.0-dev+06b169284737
```
after
```
...
$ cli bundle deploy
...
Warning: Ignoring Databricks CLI version constraint for development build. Required: >= 0.223.1, current: 0.0.0-dev+d52d6f08fcd5
```
## Tests
<!-- How is this tested? -->
2024-08-23 10:13:21 +00:00
535f670868
remove most of the debug helpers
2024-08-22 17:43:22 +02:00
790731f981
remove generic schema test
2024-08-22 17:42:19 +02:00
39efb705de
-
2024-08-22 17:40:43 +02:00
7c70179091
add self referential tesT
2024-08-22 17:38:24 +02:00
08133126ae
Merge remote-tracking branch 'origin' into improve/json-schema
2024-08-22 17:27:49 +02:00
7fe08c2386
Revert hc-install version to 0.7.0 ( #1711 )
...
## Changes
With hc-install version `0.8.0` there was a regression where debug logs
would be leaked into stderr. Reported upstream in
https://github.com/hashicorp/hc-install/issues/239 .
Meanwhile we need to revert and pin to version`0.7.0`. This PR also
includes a regression test.
## Tests
Regression test.
2024-08-22 15:04:26 +00:00
35e48be81c
[Release] Release v0.227.0 ( #1705 )
...
CLI:
* Added filtering flags for cluster list commands
([#1703 ](https://github.com/databricks/cli/pull/1703 )).
Bundles:
* Remove reference to "dbt" in the default-sql template
([#1696 ](https://github.com/databricks/cli/pull/1696 )).
* Pause continuous pipelines when 'mode: development' is used
([#1590 ](https://github.com/databricks/cli/pull/1590 )).
* Add configurable presets for name prefixes, tags, etc.
([#1490 ](https://github.com/databricks/cli/pull/1490 )).
* Report all empty resources present in error diagnostic
([#1685 ](https://github.com/databricks/cli/pull/1685 )).
* Improves detection of PyPI package names in environment dependencies
([#1699 ](https://github.com/databricks/cli/pull/1699 )).
* [DAB] Add support for requirements libraries in Job Tasks
([#1543 ](https://github.com/databricks/cli/pull/1543 )).
* Add paths field to bundle sync configuration
([#1694 ](https://github.com/databricks/cli/pull/1694 )).
Internal:
* Add `import` option for PyDABs
([#1693 ](https://github.com/databricks/cli/pull/1693 )).
* Make fileset take optional list of paths to list
([#1684 ](https://github.com/databricks/cli/pull/1684 )).
* Pass through paths argument to libs/sync
([#1689 ](https://github.com/databricks/cli/pull/1689 )).
* Correctly mark package names with versions as remote libraries
([#1697 ](https://github.com/databricks/cli/pull/1697 )).
* Share test initializer in common helper function
([#1695 ](https://github.com/databricks/cli/pull/1695 )).
* Make `pydabs/venv_path` optional
([#1687 ](https://github.com/databricks/cli/pull/1687 )).
* Use API mocks for duplicate path errors in workspace files extensions
client ([#1690 ](https://github.com/databricks/cli/pull/1690 )).
* Fix prefix preset used for UC schemas
([#1704 ](https://github.com/databricks/cli/pull/1704 )).
2024-08-22 08:44:22 +00:00
6e8cd835a3
Add paths field to bundle sync configuration ( #1694 )
...
## Changes
This field allows a user to configure paths to synchronize to the
workspace.
Allowed values are relative paths to files and directories anchored at
the directory where the field is set. If one or more values traverse up
the directory tree (to an ancestor of the bundle root directory), the
CLI will dynamically determine the root path to use to ensure that the
file tree structure remains intact.
For example, given a `databricks.yml` in `my_bundle` that includes:
```yaml
sync:
paths:
- ../common
- .
```
Then upon synchronization, the workspace will look like:
```
.
├── common
│ └── lib.py
└── my_bundle
├── databricks.yml
└── notebook.py
```
If not set behavior remains identical.
## Tests
* Newly added unit tests for the mutators and under `bundle/tests`.
* Manually confirmed a bundle without this configuration works the same.
* Manually confirmed a bundle with this configuration works.
2024-08-21 15:33:25 +00:00
6f345293b1
Added filtering flags for cluster list commands ( #1703 )
...
## Changes
Fixes #1701
## Tests
```
Usage:
databricks clusters list [flags]
Flags:
--cluster-sources []string Filter clusters by source
--cluster-states []string Filter clusters by states
-h, --help help for list
--is-pinned Filter clusters by pinned status
--page-size int Use this field to specify the maximum number of results to be returned by the server.
--page-token string Use next_page_token or prev_page_token returned from the previous request to list the next or previous page of clusters respectively.
--policy-id string Filter clusters by policy id
```
2024-08-21 15:05:49 +00:00
Shreyas Goenka
shreyas-goenka
Lennart Kats (databricks)
dependabot[bot]
Gleb Kanterov
Andrew Nester
Pieter Noordhuis