mirror of https://github.com/databricks/cli.git
1355 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
hectorcast-db
|
5a555de503
|
[Internal] Automatically trigger integration tests on PR (#1857)
## Changes Automatically trigger integration tests when a PR is opened or updated ## Tests Workflow below. --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> |
|
Pieter Noordhuis
|
ed84a33b0a
|
Reuse resource resolution code for the run command (#1858)
## Changes As of #1846 we have a generalized package for doing resource lookups and completion. This change updates the run command to use this instead of more specific code under `bundle/run`. ## Tests * Unit tests pass * Manually confirmed that completion and prompting works |
|
Andrew Nester
|
eaea308254
|
Added validator for folder permissions (#1824)
## Changes This validator checks permissions defined in top-level bundle config and permissions set in workspace for the folders bundle is deployed to. It raises the warning if the permissions defined in the workspace are not defined in bundle. This validator is executed only during `bundle validate` command. ## Tests ``` Warning: untracked permissions apply to target workspace path The following permissions apply to the workspace folder at "/Workspace/Users/andrew.nester@databricks.com/.bundle/clusters/default" but are not configured in the bundle: - level: CAN_MANAGE, user_name: andrew.nester@databricks.com ``` --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> |
|
|
Pieter Noordhuis
|
89ee7d8a99
|
Add command to open a resource in the browser (#1846)
## Changes This builds on the functionality added in #1731 that produces a URL for every resource. Adds `bundle/resources` package to deal with resource lookups and command completion. The new functionality is similar to the lookup and command completion functionality located in `bundle/run`. It differs in that it doesn't gracefully deal with ambiguous references to resources, now that we explicitly validate this doesn't occur in the bundle configuration. It still allows resources to be looked up with their fully qualified key, `<plural type>.<key>`. ## Tests * Added unit tests for resource lookup and completion * Manually confirmed that `bundle open` prompts, accepts a key argument, and opens a browser |
|
|
Pieter Noordhuis
|
eddaddaf8b
|
Attempt to reduce test flakiness on Windows (#1845)
## Changes
Test failures indicate that both stdout and stderr are consumed, yet the
content of stdout doesn't end up in the intended output. This can happen
if the goroutines responsible for writing to the combined output buffer
attempt to write to the same underlying buffer concurrently.
Example failure:
```
=== RUN TestBackgroundCombinedOutput
background_test.go:65:
Error Trace: D:/a/cli/cli/libs/process/background_test.go:65
Error: elements differ
extra elements in list A:
([]interface {}) (len=1) {
(string) (len=1) "2"
}
listA:
([]string) (len=2) {
(string) (len=1) "1",
(string) (len=1) "2"
}
listB:
([]string) (len=1) {
(string) (len=1) "1"
}
Test: TestBackgroundCombinedOutput
```
With the test body:
|
|
|
Andrew Nester
|
ab622e65bb
|
[Release] Release v0.231.0 (#1856)
CLI: * Added JSON input validation for CLI commands ([#1771](https://github.com/databricks/cli/pull/1771)). * Support Git worktrees for `sync` ([#1831](https://github.com/databricks/cli/pull/1831)). Bundles: * Add `bundle summary` to display URLs for deployed resources ([#1731](https://github.com/databricks/cli/pull/1731)). * Added a warning when incorrect permissions used for `/Workspace/Shared` bundle root ([#1821](https://github.com/databricks/cli/pull/1821)). * Show actionable errors for collaborative deployment scenarios ([#1386](https://github.com/databricks/cli/pull/1386)). * Fix path to repository-wide exclude file ([#1837](https://github.com/databricks/cli/pull/1837)). * Fixed typo in converting cluster permissions ([#1826](https://github.com/databricks/cli/pull/1826)). * Ignore metastore permission error during template generation ([#1819](https://github.com/databricks/cli/pull/1819)). * Handle normalization of `dyn.KindTime` into an any type ([#1836](https://github.com/databricks/cli/pull/1836)). * Added support for pip options in environment dependencies ([#1842](https://github.com/databricks/cli/pull/1842)). * Fix race condition when restarting continuous jobs ([#1849](https://github.com/databricks/cli/pull/1849)). * Fix pipeline in default-python template not working for certain workspaces ([#1854](https://github.com/databricks/cli/pull/1854)). * Add "output" flag to the bundle sync command ([#1853](https://github.com/databricks/cli/pull/1853)). Internal: * Move utility functions dealing with IAM to libs/iamutil ([#1820](https://github.com/databricks/cli/pull/1820)). * Remove unused `IS_OWNER` constant ([#1823](https://github.com/databricks/cli/pull/1823)). * Assert SDK version is consistent in the CLI generation process ([#1814](https://github.com/databricks/cli/pull/1814)). * Fixed unmarshalling json input into `interface{}` type ([#1832](https://github.com/databricks/cli/pull/1832)). * Fix `TestAccFsMkdirWhenFileExistsAtPath` in isolated Azure environments ([#1833](https://github.com/databricks/cli/pull/1833)). * Add behavioral tests for examples from the YAML spec ([#1835](https://github.com/databricks/cli/pull/1835)). * Remove Terraform conversion function that's no longer used ([#1840](https://github.com/databricks/cli/pull/1840)). * Encode assumptions about the dashboards API in a test ([#1839](https://github.com/databricks/cli/pull/1839)). * Add script to make testing of code on branches easier ([#1844](https://github.com/databricks/cli/pull/1844)). API Changes: * Added `databricks disable-legacy-dbfs` command group. OpenAPI commit cf9c61453990df0f9453670f2fe68e1b128647a2 (2024-10-14) Dependency updates: * Upgrade TF provider to 1.54.0 ([#1852](https://github.com/databricks/cli/pull/1852)). * Bump github.com/databricks/databricks-sdk-go from 0.48.0 to 0.49.0 ([#1843](https://github.com/databricks/cli/pull/1843)). |
|
Ilia Babanov
|
55a055d0f5
|
Add "output" flag to the bundle sync command (#1853)
## Changes We want to use 'bundle sync' in the vscode extension before running a file as an ad-hoc job (or through the context api). Right now we use bundle deploy in these cases, but deploying bundle resources is not always expected when you just want to quickly run a file. Sync makes more sense in these cases, but we still want to have verbose output to see what's happening. In the 'deploy' command we have hidden 'verbose' flag. For the sync I've just added 'output' flag, handling both json and text cases, similar to how it's done in the non-bundle `sync` command. The flag is not hidden (although we still don't show any output by default, if the flag is not set). VSCode Extension PR: https://github.com/databricks/databricks-vscode/pull/1401 ## Tests Manually |
|
Lennart Kats (databricks)
|
60c153c0e7
|
Fix pipeline in default-python template not working for certain workspaces (#1854)
Change the default-python template to not set the `catalog` field for the pipeline for workspaces that set `hive_metastore` as the default catalog. The Pipelines service currently returns an error when that value is used for the `catalog` field. This is the most simple fix for this issue, which was reported by a customer. As a followup, we should look at whether we want to prompt for a catalog instead, possibly just for this specific scenario. |
|
shreyas-goenka
|
3bab21e72e
|
Fix race condition when restarting continuous jobs (#1849)
## Changes We don't need to cancel existing runs when the job is continuous and unpaused. The `/jobs/run-now` command will cancel the existing run and trigger a new one automatically. Cancelling the job manually can cause a race condition where both the manual trigger from the CLI and the continuous trigger from the job configuration happens at the same time. This PR prevents that from happening. ## Tests Unit tests and manually |
|
|
Andrew Nester
|
68d69d6e0b
|
Upgrade TF provider to 1.54.0 (#1852)
## Changes Upgrade TF provider to 1.54.0 |
|
dependabot[bot]
|
f8bb3a8d72
|
Bump github.com/databricks/databricks-sdk-go from 0.48.0 to 0.49.0 (#1843)
Bumps
[github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go)
from 0.48.0 to 0.49.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/releases">github.com/databricks/databricks-sdk-go's
releases</a>.</em></p>
<blockquote>
<h2>v0.49.0</h2>
<h3>API Changes:</h3>
<ul>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/settings#DisableLegacyDbfsAPI">w.DisableLegacyDbfs</a>
workspace-level service.</li>
<li>Added <code>UnityCatalogProvisioningState</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#OnlineTable">catalog.OnlineTable</a>.</li>
<li>Added <code>IsTruncated</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Result">dashboards.Result</a>.</li>
<li>Added <code>EffectiveBudgetPolicyId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#BaseJob">jobs.BaseJob</a>.</li>
<li>Added <code>BudgetPolicyId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#CreateJob">jobs.CreateJob</a>.</li>
<li>Added <code>EffectiveBudgetPolicyId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Job">jobs.Job</a>.</li>
<li>Added <code>BudgetPolicyId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobSettings">jobs.JobSettings</a>.</li>
<li>Added <code>BudgetPolicyId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun">jobs.SubmitRun</a>.</li>
<li>Added <code>Report</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#IngestionConfig">pipelines.IngestionConfig</a>.</li>
<li>Added <code>SequenceBy</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#TableSpecificConfig">pipelines.TableSpecificConfig</a>.</li>
<li>Added <code>NotifyOnOk</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#Alert">sql.Alert</a>.</li>
<li>Added <code>NotifyOnOk</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#CreateAlertRequestAlert">sql.CreateAlertRequestAlert</a>.</li>
<li>Added <code>NotifyOnOk</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#ListAlertsResponseAlert">sql.ListAlertsResponseAlert</a>.</li>
<li>Added <code>NotifyOnOk</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#UpdateAlertRequestAlert">sql.UpdateAlertRequestAlert</a>.</li>
</ul>
<p>OpenAPI SHA: cf9c61453990df0f9453670f2fe68e1b128647a2, Date:
2024-10-14</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/blob/main/CHANGELOG.md">github.com/databricks/databricks-sdk-go's
changelog</a>.</em></p>
<blockquote>
<h2>[Release] Release v0.49.0</h2>
<h3>API Changes:</h3>
<ul>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/settings#DisableLegacyDbfsAPI">w.DisableLegacyDbfs</a>
workspace-level service.</li>
<li>Added <code>UnityCatalogProvisioningState</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#OnlineTable">catalog.OnlineTable</a>.</li>
<li>Added <code>IsTruncated</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Result">dashboards.Result</a>.</li>
<li>Added <code>EffectiveBudgetPolicyId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#BaseJob">jobs.BaseJob</a>.</li>
<li>Added <code>BudgetPolicyId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#CreateJob">jobs.CreateJob</a>.</li>
<li>Added <code>EffectiveBudgetPolicyId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Job">jobs.Job</a>.</li>
<li>Added <code>BudgetPolicyId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobSettings">jobs.JobSettings</a>.</li>
<li>Added <code>BudgetPolicyId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun">jobs.SubmitRun</a>.</li>
<li>Added <code>Report</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#IngestionConfig">pipelines.IngestionConfig</a>.</li>
<li>Added <code>SequenceBy</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#TableSpecificConfig">pipelines.TableSpecificConfig</a>.</li>
<li>Added <code>NotifyOnOk</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#Alert">sql.Alert</a>.</li>
<li>Added <code>NotifyOnOk</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#CreateAlertRequestAlert">sql.CreateAlertRequestAlert</a>.</li>
<li>Added <code>NotifyOnOk</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#ListAlertsResponseAlert">sql.ListAlertsResponseAlert</a>.</li>
<li>Added <code>NotifyOnOk</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/sql#UpdateAlertRequestAlert">sql.UpdateAlertRequestAlert</a>.</li>
</ul>
<p>OpenAPI SHA: cf9c61453990df0f9453670f2fe68e1b128647a2, Date:
2024-10-14</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="
|
|
Stephen Macke
|
571076d5e1
|
Support Git worktrees for `sync` (#1831)
## Changes This change allows the `sync` command to work from [git worktrees](https://git-scm.com/docs/git-worktree). ## Tests * Added unit tests for traversal of worktree related files. * Manually confirmed that synchronization of files from a main checkout, as well as a worktree, observed the same ignore rules (both locally defined as well as from `$GIT_DIR/info/exclude`). --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> |
|
|
Pieter Noordhuis
|
ca45e53f42
|
Add script to make testing of code on branches easier (#1844)
## Changes Convenience script to exec into a shell where a CLI build for a specific branch is made available. ## Tests Manually from `/tmp` with `bash <([path]) demo-dashboards`. |
|
|
Andrew Nester
|
ffdbec87cc
|
Added support for pip options in environment dependencies (#1842)
## Changes
Added support for specifying pip options such as `--extra-index-url` and
etc. in environments dependencies
```
environments:
- environment_key: Default
spec:
client: "1"
dependencies:
- --extra-index-url https://foo@bar.com/packages/smth somepackage
- json==1.0.0
```
## Tests
Added regression test
---------
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
|
|
|
Pieter Noordhuis
|
eefda8c198
|
Fix path to repository-wide exclude file (#1837)
## Changes This file is at `info/exclude`, and not `info/excludes`. Also see https://git-scm.com/docs/gitignore. ## Tests Manually confirmed that these ignore patterns are now picked up. I created a repository with a pattern in this file and ran `sync` to confirm it ignores files matching the pattern. |
|
|
Pieter Noordhuis
|
c9770503f8
|
Encode assumptions about the dashboards API in a test (#1839)
## Changes Dashboards can be imported either via its own CRUD API, or via the workspace import API. This test encodes the assumptions we can make about the API behavior. More specifically, the identity of the dashboard is retained on workspace import, as are the properties that aren't surfaced in the workspace import API. ## Tests The integration test passes. |
|
|
Pieter Noordhuis
|
3b1fb6d2df
|
Remove Terraform conversion function that's no longer used (#1840)
## Changes In #1218, the `BundleToTerraform` function was replaced by a version based on the dynamic configuration tree. Since then, it has only been used in tests to confirm that the output of the old function was equal to the output of the new function. We no longer need this and can exclusively rely on the version based on the dynamic configuration tree. ## Tests Tests pass. |
|
|
Andrew Nester
|
0c9c90208f
|
Added a warning when incorrect permissions used for `/Workspace/Shared` bundle root (#1821)
## Changes Added a warning when incorrect permissions used for `/Workspace/Shared` bundle root ## Tests Added unit test --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> |
|
|
Lennart Kats (databricks)
|
c5043c3d9d
|
Add `bundle summary` to display URLs for deployed resources (#1731)
## Changes Adds a textual output to the `databricks bundle summary` command, which includes URLs of deployed resources. Example usage: ``` $ databricks bundle summary Name: my_pipeline Target: dev Workspace: Host: https://domain.databricks.com User: user@databricks.com Path: /Users/user@databricks.com/.bundle/my_pipeline/dev Resources: Jobs: my_project_job: Name: [dev lennart] my_project_job URL: https://domain.databricks.com/jobs/206899209187287?o=6051921418418893 Pipelines: my_project_pipeline: Name: [dev lennart] my_project_pipeline URL: https://domain.databricks.com/pipelines/3f849fd5-ba7d-47fa-a34c-c6bf034b4f58?o=6051921418418893 ``` Notes: * The top headers of the output are the same as those from the existing `bundle validate` command * URLs are colored light blue in the output * For resources that haven't been deployed yet, we show `(not deployed)` in place of the URL --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com> |
|
|
Pieter Noordhuis
|
dedec58e41
|
Add behavioral tests for examples from the YAML spec (#1835)
## Changes I took the examples from https://yaml.org/spec/1.2.2. The required modifications to the loader are: * Correctly parse floating point infinities and NaN * Correctly parse octal numbers per the YAML 1.2 spec * Treat "null" keys in a map as valid ## Tests Existing and new unit tests pass. |
|
|
Pieter Noordhuis
|
e4d039a1aa
|
Handle normalization of `dyn.KindTime` into an any type (#1836)
## Changes The issue reported in #1828 illustrates how using a YAML timestamp-like value (a date in this case) causes an issue during conversion to and from the typed configuration tree. We use the `AsAny()` function on the `dyn.Value` when normalizing for the `any` type. We only use the `any` type for variable values, because they can assume every type. The `AsAny()` function returns a `time.Time` for the time value during conversion **to** the typed configuration tree. Upon conversion **from** the typed configuration tree back into the dynamic configuration tree, we cannot distinguish a `time.Time` struct from any other struct. To address this, we use the underlying string value of the time value when we normalize for the `any` type. Fixes #1828. ## Tests Existing unit tests pass |
|
|
shreyas-goenka
|
cc112961ce
|
Fix `TestAccFsMkdirWhenFileExistsAtPath` in isolated Azure environments (#1833)
## Changes This test passes on normal `azure-prod` but started to fail on `azure-prod-is`, which is the isolated version of azure-prod. This PR patches the test to include the error returned from the cloud setup in `azure-prod-is`. ## Tests The test passes now on `azure-prod-is`. |
|
|
Andrew Nester
|
0753dfe2f4
|
Fixed unmarshalling json input into `interface{}` type (#1832)
## Changes
Fixed unmarshalling json input into `interface{}` type
Commands like `api post` support free form request input, so it should
be unmarshaled correctly
## Tests
Added regression test + E2E test pass
|
|
|
shreyas-goenka
|
ab20624206
|
Assert SDK version is consistent in the CLI generation process (#1814)
## Changes Followup from https://github.com/databricks/cli/pull/1809#discussion_r1790045086. User will see the following error if the SDK version does not match during CLI code generation. ``` --- FAIL: TestConsistentDatabricksSdkVersion (1.34s) info_test.go:53: Error Trace: /Users/shreyas.goenka/cli/internal/build/info_test.go:53 Error: Not equal: expected: "0c86ea6dbd9a730c24ff0d4e509603e476955ac5" actual : "6f6b1371e640f2dfeba72d365ac566368656f6b6" Diff: --- Expected +++ Actual @@ -1 +1 @@ -0c86ea6dbd9a730c24ff0d4e509603e476955ac5 +6f6b1371e640f2dfeba72d365ac566368656f6b6 Test: TestConsistentDatabricksSdkVersion Messages: please update the SDK version before generating the CLI ``` ## Tests Manually asserted that: 1. Generating the CLI without the SDK bump causes an error. 2. Generating the CLI after the SDK bump works as expected. 3. The test works even when the SDK is pinned to a specific commit like `v0.47.1-0.20241002195128-6cecc224cbf7` |
|
|
Andrew Nester
|
f0e2981596
|
Added JSON input validation for CLI commands (#1771)
## Changes Added JSON input validation for CLI commands. Now when invalid JSON passed as a payload to CLI commands, CLI performs input normalisation and detects if there are any mismatches such as incorrect types, unknown fields and etc. This diagnostic information is printed in standard error output and does not block command execution, so the change is backward compatible. Fixes #1769 #1764 #1625 #1560 ## Tests Added unit tests ``` andrew.nester@HFW9Y94129 ~ % databricks jobs create --json '{"seeti}' Error: error decoding JSON at (inline):1:2: unexpected EOF andrew.nester@HFW9Y94129 ~ % databricks jobs create --json '{"seeti": true}' Warning: unknown field: seeti in (inline):1:9 Error: Job settings must be specified. ``` --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> |
|
|
Lennart Kats (databricks)
|
08a0d083c3
|
Ignore metastore permission error during template generation (#1819)
## Changes
This extends the `{{default_catalog}}` helper in templates to ignore any
`PERMISSION_DENIED` error. We're still reviewing when exactly this error
occurs, but if it does, it should not break templates. We should fall
back to assuming there's no default catalog (and no UC) instead.
## Testing
I have not been able to reproduce this issue, but there is a customer
report about "access denied to clusters that don't have unity catalog
enabled" being returned on a non-UC workspace. The error code in this PR
corresponds to that message.
## Next steps
We'll work together with the UC team to review if this error even makes
sense for this API. If that discussion leads to a behavior change in the
API we can update the CLI code again.
|
|
|
Andrew Nester
|
845d23ac21
|
Fixed typo in converting cluster permissions (#1826)
## Changes Fixed typo in converting cluster permissions |
|
|
Pieter Noordhuis
|
b9085de533
|
Remove unused `IS_OWNER` constant (#1823)
## Changes Leftover from #1386. ## Tests All tests pass (indicating it really wasn't used). |
|
|
Pieter Noordhuis
|
3270afaff4
|
Move utility functions dealing with IAM to libs/iamutil (#1820)
## Changes The two functions `GetShortUserName` and `IsServicePrincipal` are unrelated to auth or the purpose of the auth package. This change moves them into their own package and updates `IsServicePrincipal` to take an `*iam.User` argument instead of a string username. ## Tests Tests pass. |
|
|
Lennart Kats (databricks)
|
e885794722
|
Show actionable errors for collaborative deployment scenarios (#1386)
## Changes This adds diagnostics for collaborative (production) deployment scenarios, including: - Bob deploys a bundle that is normally deployed by Alice, but this fails because Bob can't write to `/Users/Alice/.bundle`. - Charlie deploys a bundle that is normally deployed by Alice, but this fails because he can't create a new pipeline where Alice would be the owner. - Alice deploys a bundle where she didn't list herself as one of the CAN_MANAGE users in permissions. That can work, but is probably a mistake. ## Tests Unit tests, manual testing. |
|
|
Andrew Nester
|
c92c67afc9
|
[Release] Release v0.230.0 (#1817)
Notable changes for Databricks Asset Bundles:
Workspace paths are automatically prefixed with `/Workspace`. In
addition, all usage of path strings such as
`/Workspace/${workspace.root_path}/...` in bundle configuration is
automatically replaced with `${workspace.root_path}/...` and generates a
warning as part of bundle validate.
More details can be find here:
https://docs.databricks.com/en/release-notes/dev-tools/bundles.html#workspace-paths-will-be-automatically-prefixed
Bundles:
* Add an error if state files grow bigger than the export limit
([#1795](https://github.com/databricks/cli/pull/1795)).
* Always prepend bundle remote paths with /Workspace
([#1724](https://github.com/databricks/cli/pull/1724)).
* Add resource path field to bundle workspace configuration
([#1800](https://github.com/databricks/cli/pull/1800)).
* Add validation for files with a `.(resource-name).yml` extension
([#1780](https://github.com/databricks/cli/pull/1780)).
Internal:
* Remove deprecated or readonly fields from the bundle schema
([#1809](https://github.com/databricks/cli/pull/1809)).
API Changes:
* Changed `databricks git-credentials create`, `databricks
git-credentials delete`, `databricks git-credentials get`, `databricks
git-credentials list`, `databricks git-credentials update` commands .
* Changed `databricks repos create`, `databricks repos delete`,
`databricks repos get`, `databricks repos update` command .
OpenAPI commit 0c86ea6dbd9a730c24ff0d4e509603e476955ac5 (2024-10-02)
Dependency updates:
* Upgrade TF provider to 1.53.0
([#1815](https://github.com/databricks/cli/pull/1815)).
* Bump golang.org/x/term from 0.24.0 to 0.25.0
([#1811](https://github.com/databricks/cli/pull/1811)).
* Bump golang.org/x/text from 0.18.0 to 0.19.0
([#1812](https://github.com/databricks/cli/pull/1812)).
* Bump github.com/databricks/databricks-sdk-go from 0.47.0 to 0.48.0
([#1810](https://github.com/databricks/cli/pull/1810)).
---------
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
|
|
|
Pieter Noordhuis
|
ae568743c5
|
Upgrade TF provider to 1.53.0 (#1815)
## Changes See https://github.com/databricks/terraform-provider-databricks/releases/tag/v1.53.0 ## Tests Integration tests pass. |
|
|
shreyas-goenka
|
88318d384a
|
Remove deprecated or readonly fields from the bundle schema (#1809)
## Changes We do not need the user to specify these fields in their bundle configuration, so we remove them from the JSON schema. ## Tests Manually and end to end tests. The JSON schema has also been regenerated after these changes. |
|
|
dependabot[bot]
|
7a2141fc5b
|
Bump github.com/databricks/databricks-sdk-go from 0.47.0 to 0.48.0 (#1810)
Bumps [github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go) from 0.47.0 to 0.48.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/databricks/databricks-sdk-go/releases">github.com/databricks/databricks-sdk-go's releases</a>.</em></p> <blockquote> <h2>v0.48.0</h2> <h3>Internal Changes</h3> <ul> <li>Update SDK to latest OpenAPI spec (<a href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1057">#1057</a>).</li> </ul> <p>Note: This release contains breaking changes, please see the API changes below for more details.</p> <h3>API Changes:</h3> <ul> <li>Added <code>DefaultSourceCodePath</code> and <code>Resources</code> fields for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#App">apps.App</a>.</li> <li>Added <code>Resources</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#CreateAppRequest">apps.CreateAppRequest</a>.</li> <li>Added <code>Resources</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#UpdateAppRequest">apps.UpdateAppRequest</a>.</li> <li>Added <code>Schema</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#CreatePipeline">pipelines.CreatePipeline</a>.</li> <li>Added <code>Schema</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#EditPipeline">pipelines.EditPipeline</a>.</li> <li>Added <code>Schema</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#PipelineSpec">pipelines.PipelineSpec</a>.</li> <li>[Breaking] Changed <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateCredentialsRequest">workspace.CreateCredentialsRequest</a>.</li> <li>[Breaking] Changed <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to type <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service.</li> <li>[Breaking] Changed <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#DeleteCredentialsRequest">workspace.DeleteCredentialsRequest</a>.</li> <li>[Breaking] Changed <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to return <code>any</code>.</li> <li>Changed <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to type <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service.</li> <li>[Breaking] Changed <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsRequest">workspace.GetCredentialsRequest</a>.</li> <li>Changed <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to type <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service.</li> <li>[Breaking] Changed <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to return <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsResponse">workspace.GetCredentialsResponse</a>.</li> <li>[Breaking] Changed <code>List</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to return <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ListCredentialsResponse">workspace.ListCredentialsResponse</a>.</li> <li>Changed <code>List</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to type <code>List</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service.</li> <li>Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to type <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service.</li> <li>[Breaking] Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to return <code>any</code>.</li> <li>[Breaking] Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateCredentialsRequest">workspace.UpdateCredentialsRequest</a>.</li> <li>Changed <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to type <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service.</li> <li>[Breaking] Changed <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateRepoRequest">workspace.CreateRepoRequest</a>.</li> <li>[Breaking] Changed <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to return <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateRepoResponse">workspace.CreateRepoResponse</a>.</li> <li>[Breaking] Changed <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to return <code>any</code>.</li> <li>Changed <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to type <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service.</li> <li>[Breaking] Changed <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to return <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetRepoResponse">workspace.GetRepoResponse</a>.</li> <li>Changed <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to type <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service.</li> <li>Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to return <code>any</code>.</li> <li>[Breaking] Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateRepoRequest">workspace.UpdateRepoRequest</a>.</li> <li>Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to type <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service.</li> <li>[Breaking] Changed <code>CredentialId</code> and <code>GitProvider</code> fields for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateCredentialsResponse">workspace.CreateCredentialsResponse</a> to be required.</li> <li>Changed <code>CredentialId</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CredentialInfo">workspace.CredentialInfo</a> to be required.</li> <li>Changed <code>CredentialId</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsResponse">workspace.GetCredentialsResponse</a> to be required.</li> <li>Changed <code>Patterns</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#SparseCheckout">workspace.SparseCheckout</a> to type <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#List">workspace.List</a>.</li> <li>Changed <code>Patterns</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#SparseCheckoutUpdate">workspace.SparseCheckoutUpdate</a> to type <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#List">workspace.List</a>.</li> <li>[Breaking] Changed <code>GitProvider</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateCredentialsRequest">workspace.UpdateCredentialsRequest</a> to be required.</li> </ul> <p>OpenAPI SHA: 0c86ea6dbd9a730c24ff0d4e509603e476955ac5, Date: 2024-10-02</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/databricks/databricks-sdk-go/blob/main/CHANGELOG.md">github.com/databricks/databricks-sdk-go's changelog</a>.</em></p> <blockquote> <h2>[Release] Release v0.48.0</h2> <h3>Internal Changes</h3> <ul> <li>Update SDK to latest OpenAPI spec (<a href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1057">#1057</a>).</li> </ul> <p>Note: This release contains breaking changes, please see the API changes below for more details.</p> <h3>API Changes:</h3> <ul> <li>Added <code>DefaultSourceCodePath</code> and <code>Resources</code> fields for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#App">apps.App</a>.</li> <li>Added <code>Resources</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#CreateAppRequest">apps.CreateAppRequest</a>.</li> <li>Added <code>Resources</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#UpdateAppRequest">apps.UpdateAppRequest</a>.</li> <li>Added <code>Schema</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#CreatePipeline">pipelines.CreatePipeline</a>.</li> <li>Added <code>Schema</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#EditPipeline">pipelines.EditPipeline</a>.</li> <li>Added <code>Schema</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#PipelineSpec">pipelines.PipelineSpec</a>.</li> <li>[Breaking] Changed <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateCredentialsRequest">workspace.CreateCredentialsRequest</a>.</li> <li>[Breaking] Changed <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to type <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service.</li> <li>[Breaking] Changed <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#DeleteCredentialsRequest">workspace.DeleteCredentialsRequest</a>.</li> <li>[Breaking] Changed <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to return <code>any</code>.</li> <li>Changed <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to type <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service.</li> <li>[Breaking] Changed <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsRequest">workspace.GetCredentialsRequest</a>.</li> <li>Changed <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to type <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service.</li> <li>[Breaking] Changed <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to return <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsResponse">workspace.GetCredentialsResponse</a>.</li> <li>[Breaking] Changed <code>List</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to return <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ListCredentialsResponse">workspace.ListCredentialsResponse</a>.</li> <li>Changed <code>List</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to type <code>List</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service.</li> <li>Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to type <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service.</li> <li>[Breaking] Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service to return <code>any</code>.</li> <li>[Breaking] Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateCredentialsRequest">workspace.UpdateCredentialsRequest</a>.</li> <li>Changed <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to type <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service.</li> <li>[Breaking] Changed <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateRepoRequest">workspace.CreateRepoRequest</a>.</li> <li>[Breaking] Changed <code>Create</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to return <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateRepoResponse">workspace.CreateRepoResponse</a>.</li> <li>[Breaking] Changed <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to return <code>any</code>.</li> <li>Changed <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to type <code>Delete</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service.</li> <li>[Breaking] Changed <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to return <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetRepoResponse">workspace.GetRepoResponse</a>.</li> <li>Changed <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to type <code>Get</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service.</li> <li>Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to return <code>any</code>.</li> <li>[Breaking] Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service . New request type is <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateRepoRequest">workspace.UpdateRepoRequest</a>.</li> <li>Changed <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service to type <code>Update</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a> workspace-level service.</li> <li>[Breaking] Changed <code>CredentialId</code> and <code>GitProvider</code> fields for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateCredentialsResponse">workspace.CreateCredentialsResponse</a> to be required.</li> <li>Changed <code>CredentialId</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CredentialInfo">workspace.CredentialInfo</a> to be required.</li> <li>Changed <code>CredentialId</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsResponse">workspace.GetCredentialsResponse</a> to be required.</li> <li>Changed <code>Patterns</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#SparseCheckout">workspace.SparseCheckout</a> to type <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#List">workspace.List</a>.</li> <li>Changed <code>Patterns</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#SparseCheckoutUpdate">workspace.SparseCheckoutUpdate</a> to type <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#List">workspace.List</a>.</li> <li>[Breaking] Changed <code>GitProvider</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateCredentialsRequest">workspace.UpdateCredentialsRequest</a> to be required.</li> </ul> <p>OpenAPI SHA: 0c86ea6dbd9a730c24ff0d4e509603e476955ac5, Date: 2024-10-02</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
|
|
dependabot[bot]
|
02bfd397ef
|
Bump golang.org/x/text from 0.18.0 to 0.19.0 (#1812)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.18.0 to 0.19.0. <details> <summary>Commits</summary> <ul> <li><a href=" |
|
|
dependabot[bot]
|
589e5ff1a4
|
Bump golang.org/x/term from 0.24.0 to 0.25.0 (#1811)
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.24.0 to
0.25.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="
|
|
|
shreyas-goenka
|
bca9c2eda4
|
Add validation for files with a `.(resource-name).yml` extension (#1780)
## Changes
We want to encourage a pattern of specifying only a single resource in a
YAML file when the `.(resource-type).yml` extension is used (for
example, `.job.yml`). This convention could allow us to bijectively map
a resource YAML file to its corresponding resource in the Databricks
workspace.
This PR:
1. Emits a recommendation diagnostic when we detect this convention is
being violated. We can promote this to a warning when we want to
encourage this pattern more strongly.
2. Visualises the recommendation diagnostics in the `bundle validate`
command.
**NOTE:** While this PR also shows the recommendation for `.yaml` files,
we do not encourage users to use this extension. We only support it here
since it's part of the YAML standard and some existing users might
already be using `.yaml`.
## Tests
Unit tests and manually. Here's what an example output looks like:
```
Recommendation: define a single job in a file with the .job.yml extension.
at resources.jobs.bar
resources.jobs.foo
in foo.job.yml:13:7
foo.job.yml:5:7
The following resources are defined or configured in this file:
- bar (job)
- foo (job)
```
---------
Co-authored-by: Lennart Kats (databricks) <lennart.kats@databricks.com>
|
|
|
Andrew Nester
|
a8cff48c0b
|
Always prepend bundle remote paths with /Workspace (#1724)
## Changes
Due to platform changes, all libraries, notebooks and etc. paths used in
Databricks must be started with either /Workspace or /Volumes prefix.
This PR makes sure that all bundle paths are correctly prefixed.
Note: this change is a breaking change if user previously configured and
used `/Workspace/Workspace` folder in their workspace file system or
having `/Workspace/${workspace.root_path}...` pattern configured
anywhere in their bundle config
Fixes: #1751
AI:
- [x] Scan DABs config and error out on
`/Workspace/${workspace.root_path}...` pattern usage
## Tests
Added unit tests
---------
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
|
|
|
Pieter Noordhuis
|
80d55f4540
|
Add resource path field to bundle workspace configuration (#1800)
## Changes Default workspace path for resources with a presence in the workspace tree. Note: this path is **not** created automatically (yet). We need this only for dashboards (so far), so can take care of creation if one or more dashboards are part of a deployment. This saves an API call for deployments where this is not necessary. ## Tests Expanded existing tests. |
|
|
Andrew Nester
|
044a00c7f9
|
Add an error if state files grow bigger than the export limit (#1795)
## Changes Currently API limits on exporting files from workspaces are set at 10 MBs while uploading to is 500 MBs. We want to prevent users running into deadlock when they won't be able to pull state file anymore so we prevent from uploading large state files (over 10 MBs) to Databricks workspace. |
|
|
Andrew Nester
|
c28d64f2dc
|
[Release] Release v0.229.0 (#1801)
Bundles: * Added support for creating all-purpose clusters ([#1698](https://github.com/databricks/cli/pull/1698)). * Reduce time until the prompt is shown for bundle run ([#1727](https://github.com/databricks/cli/pull/1727)). * Use Unity Catalog for pipelines in the default-python template ([#1766](https://github.com/databricks/cli/pull/1766)). * Add verbose flag to the "bundle deploy" command ([#1774](https://github.com/databricks/cli/pull/1774)). * Fixed full variable override detection ([#1787](https://github.com/databricks/cli/pull/1787)). * Add sub-extension to resource files in built-in templates ([#1777](https://github.com/databricks/cli/pull/1777)). * Fix panic in `apply_presets.go` ([#1796](https://github.com/databricks/cli/pull/1796)). Internal: * Assert tokens are redacted in origin URL when username is not specified ([#1785](https://github.com/databricks/cli/pull/1785)). * Refactor jobs path translation ([#1782](https://github.com/databricks/cli/pull/1782)). * Add JobTaskClusterSpec validate mutator ([#1784](https://github.com/databricks/cli/pull/1784)). * Pin Go toolchain to 1.22.7 ([#1790](https://github.com/databricks/cli/pull/1790)). * Modify SetLocation test utility to take full locations as argument ([#1788](https://github.com/databricks/cli/pull/1788)). * Simplified isFullVariableOverrideDef implementation ([#1791](https://github.com/databricks/cli/pull/1791)). * Sort tasks by `task_key` before generating the Terraform configuration ([#1776](https://github.com/databricks/cli/pull/1776)). * Trim trailing whitespace ([#1794](https://github.com/databricks/cli/pull/1794)). * Move trampoline code into trampoline package ([#1793](https://github.com/databricks/cli/pull/1793)). * Rename `RootPath` -> `BundleRootPath` ([#1792](https://github.com/databricks/cli/pull/1792)). API Changes: * Changed `databricks apps delete` command to return . * Changed `databricks apps deploy` command with new required argument order. * Changed `databricks apps start` command to return . * Changed `databricks apps stop` command to return . * Added `databricks temporary-table-credentials` command group. * Added `databricks serving-endpoints put-ai-gateway` command. * Added `databricks disable-legacy-access` command group. * Added `databricks account disable-legacy-features` command group. OpenAPI commit 6f6b1371e640f2dfeba72d365ac566368656f6b6 (2024-09-19) Dependency updates: * Upgrade to Go SDK 0.47.0 ([#1799](https://github.com/databricks/cli/pull/1799)). * Upgrade to TF provider 1.52 ([#1781](https://github.com/databricks/cli/pull/1781)). * Bump golang.org/x/mod from 0.20.0 to 0.21.0 ([#1758](https://github.com/databricks/cli/pull/1758)). * Bump github.com/hashicorp/hc-install from 0.7.0 to 0.9.0 ([#1772](https://github.com/databricks/cli/pull/1772)). |
|
|
Andrew Nester
|
84fc1ed131
|
Upgrade to Go SDK 0.47.0 (#1799)
## Changes Upgrade to Go SDK 0.47.0 ## Tests <!-- How is this tested? --> |
|
|
Lennart Kats (databricks)
|
da3b4f7c72
|
Fix panic in `apply_presets.go` (#1796)
## Changes This fixes the user-reported panic in `apply_presets.go`. I'm still unsure how to reproduce this, since the CLI just reports `ob broken_job is not defined` when I try to use `bundle deploy` with an empty job. That said — we may as well be defensive here and I see we have lots of checks for empty job/cluster/etc. settings scattered throughout our code base so at least we're somewhat consistent. |
|
|
Pieter Noordhuis
|
1d1aa0a416
|
Rename `RootPath` -> `BundleRootPath` (#1792)
## Changes After introducing the `SyncRootPath` field on the bundle (#1694), the previous `RootPath` became ambiguous. Does it mean the bundle root path or the sync root path? This PR renames to field to `BundleRootPath` to remove the ambiguity. ## Tests n/a --------- Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com> |
|
|
Pieter Noordhuis
|
56cd96cb93
|
Move trampoline code into trampoline package (#1793)
## Changes Doing this to make room for PyDABs under `bundle/python`. ## Tests n/a |
|
|
Pieter Noordhuis
|
a1dca56abf
|
Trim trailing whitespace (#1794)
## Changes Trailing whitespace is trimmed per the VS Code settings for this repository. ## Tests n/a |
|
|
shreyas-goenka
|
4e8e027380
|
Sort tasks by `task_key` before generating the Terraform configuration (#1776)
## Changes Sort the tasks in the resultant `bundle.tf.json`. This is important because configuration from one task can leak into another if the tasks are not sorted. For more details see: 1. https://github.com/databricks/terraform-provider-databricks/issues/3951 2. https://github.com/databricks/terraform-provider-databricks/issues/4011 ## Tests Unit test and manually. For manual testing I used the following configuration: ``` resources: jobs: foo: tasks: - task_key: task-Z notebook_task: notebook_path: nb.py source: GIT existing_cluster_id: 0715-133738-ju0ma84z - task_key: task-1 notebook_task: notebook_path: ${workspace.file_path}/local.py source: WORKSPACE existing_cluster_id: 0715-133738-ju0ma84z depends_on: - task_key: task-Z git_source: git_provider: gitHub git_url: https://github.com/shreyas-goenka/job-source-tmp.git git_branch: main ``` Steps (1): 1. Deploy this bundle. 2. Comment out "source: GIT" 3. Deploy again Before: Deploying this bundle twice would fail. This is because the "source: GIT" would carry over to the next deployment. After: There was no error on the subsequent deployment. Steps (2): 1. Deploy once 2. Deploy again Before: Works correctly but leads to a update API call every time. After: No diff is detected by terraform. |
|
|
Andrew Nester
|
66f2ba64a8
|
Simplified isFullVariableOverrideDef implementation (#1791)
## Changes Simplified isFullVariableOverrideDef implementation Follow up on https://github.com/databricks/cli/pull/1787 |
|
|
dependabot[bot]
|
94d8c3ba1e
|
Bump github.com/hashicorp/hc-install from 0.7.0 to 0.9.0 (#1772)
Bumps [github.com/hashicorp/hc-install](https://github.com/hashicorp/hc-install) from 0.7.0 to 0.9.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/hashicorp/hc-install/releases">github.com/hashicorp/hc-install's releases</a>.</em></p> <blockquote> <h2>v0.9.0</h2> <h2>What's Changed</h2> <ul> <li>Finish Release of 0.8.1 by updating VERSION by <a href="https://github.com/mutahhir"><code>@mutahhir</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/248">hashicorp/hc-install#248</a></li> <li>build(deps): bump golang.org/x/mod from 0.20.0 to 0.21.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/242">hashicorp/hc-install#242</a></li> <li>docs: Update release instructions by <a href="https://github.com/radeksimko"><code>@radeksimko</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/249">hashicorp/hc-install#249</a></li> <li>Prepare for v0.9.0 release by <a href="https://github.com/mutahhir"><code>@mutahhir</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/250">hashicorp/hc-install#250</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/hashicorp/hc-install/compare/v0.8.1...v0.9.0">https://github.com/hashicorp/hc-install/compare/v0.8.1...v0.9.0</a></p> <h2>v0.8.1</h2> <h2>What's Changed</h2> <ul> <li>Add artifacts manifest (automatically generated) by <a href="https://github.com/jeanneryan"><code>@jeanneryan</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/235">hashicorp/hc-install#235</a></li> <li>build(deps): bump golang.org/x/mod from 0.19.0 to 0.20.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/236">hashicorp/hc-install#236</a></li> <li>build(deps): Bump workflows to latest trusted versions by <a href="https://github.com/hashicorp-tsccr"><code>@hashicorp-tsccr</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/237">hashicorp/hc-install#237</a></li> <li>build(deps): Bump workflows to latest trusted versions by <a href="https://github.com/hashicorp-tsccr"><code>@hashicorp-tsccr</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/238">hashicorp/hc-install#238</a></li> <li>build(deps): bump hashicorp/action-setup-bob from 2.1.0 to 2.1.1 in the github-actions-backward-compatible group by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/241">hashicorp/hc-install#241</a></li> <li>httpclient: Reuse existing configured logger by <a href="https://github.com/radeksimko"><code>@radeksimko</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/240">hashicorp/hc-install#240</a></li> <li>build(deps): Bump workflows to latest trusted versions by <a href="https://github.com/hashicorp-tsccr"><code>@hashicorp-tsccr</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/243">hashicorp/hc-install#243</a></li> <li>Nightly and PR builds: fix "no space left on device" on macOS runner by <a href="https://github.com/kmoe"><code>@kmoe</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/245">hashicorp/hc-install#245</a></li> <li>Update CONTRIBUTING.md to add clean up step by <a href="https://github.com/mutahhir"><code>@mutahhir</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/246">hashicorp/hc-install#246</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/jeanneryan"><code>@jeanneryan</code></a> made their first contribution in <a href="https://redirect.github.com/hashicorp/hc-install/pull/235">hashicorp/hc-install#235</a></li> <li><a href="https://github.com/mutahhir"><code>@mutahhir</code></a> made their first contribution in <a href="https://redirect.github.com/hashicorp/hc-install/pull/246">hashicorp/hc-install#246</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/hashicorp/hc-install/compare/v0.8.0...v0.8.1">https://github.com/hashicorp/hc-install/compare/v0.8.0...v0.8.1</a></p> <h2>v0.8.0</h2> <p>ENHANCEMENTS:</p> <ul> <li>Add retries for HTTP operations by <a href="https://github.com/james0209"><code>@james0209</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/218">hashicorp/hc-install#218</a></li> <li>Allow <code>LicenseDir</code> field for non-enterprise usage by <a href="https://github.com/james0209"><code>@james0209</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/214">hashicorp/hc-install#214</a></li> </ul> <p>BUG FIXES:</p> <ul> <li>[fix] include custom url's "path" when creating Archive URL by <a href="https://github.com/james0209"><code>@james0209</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/234">hashicorp/hc-install#234</a></li> </ul> <p>INTERNAL:</p> <ul> <li>[chore] Remove unused variable by <a href="https://github.com/james0209"><code>@james0209</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/215">hashicorp/hc-install#215</a></li> <li>build(deps): bump github.com/hashicorp/go-retryablehttp from 0.7.6 to 0.7.7 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/221">hashicorp/hc-install#221</a></li> <li>build(deps): bump github.com/hashicorp/go-version from 1.6.0 to 1.7.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/216">hashicorp/hc-install#216</a></li> <li>build(deps): bump golang.org/x/mod from 0.17.0 to 0.18.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/223">hashicorp/hc-install#223</a></li> <li>build(deps): bump golang.org/x/mod from 0.18.0 to 0.19.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/229">hashicorp/hc-install#229</a></li> <li>build(deps): bump hashicorp/action-setup-bob from 2.0.0 to 2.0.3 in the github-actions-backward-compatible group by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/220">hashicorp/hc-install#220</a></li> <li>build(deps): bump hashicorp/action-setup-bob from 2.0.3 to 2.1.0 in the github-actions-backward-compatible group by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/222">hashicorp/hc-install#222</a></li> <li>build(deps): bump hashicorp/actions-packaging-linux from 1.7 to 1.8 in the github-actions-backward-compatible group by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/224">hashicorp/hc-install#224</a></li> <li>build(deps): Bump workflows to latest trusted versions by <a href="https://github.com/hashicorp-tsccr"><code>@hashicorp-tsccr</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/219">hashicorp/hc-install#219</a></li> <li>build(deps): Bump workflows to latest trusted versions by <a href="https://github.com/hashicorp-tsccr"><code>@hashicorp-tsccr</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/226">hashicorp/hc-install#226</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
|
|
dependabot[bot]
|
875b112f80
|
Bump golang.org/x/mod from 0.20.0 to 0.21.0 (#1758)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.20.0 to 0.21.0. <details> <summary>Commits</summary> <ul> <li><a href=" |