Commit Graph

1307 Commits

Author SHA1 Message Date
Lennart Kats (databricks) 530a819f2f
Merge branch 'main' into cp-summary-with-urls 2024-10-17 18:46:12 +02:00
Pieter Noordhuis 7658dbdff2
Use net/url to build resource URLs such that proper escaping is done 2024-10-17 17:30:52 +02:00
Pieter Noordhuis 7bf2ec378c
Always use ID to synthesize URLs 2024-10-17 17:17:17 +02:00
Pieter Noordhuis b1dd60c410
Include test case for clusters 2024-10-17 16:45:18 +02:00
Pieter Noordhuis dccd70bdb7
require -> assert 2024-10-17 16:40:46 +02:00
Pieter Noordhuis 85bc79f7a7
Use plural title resource header 2024-10-17 16:39:20 +02:00
Pieter Noordhuis 2c8bb75bc4
Add singular/plural titles to config.SupportedResources() 2024-10-17 15:58:04 +02:00
Pieter Noordhuis 13049a555b
Remove unused name field 2024-10-17 15:41:24 +02:00
Pieter Noordhuis dedec58e41
Add behavioral tests for examples from the YAML spec (#1835)
## Changes

I took the examples from https://yaml.org/spec/1.2.2.

The required modifications to the loader are:
* Correctly parse floating point infinities and NaN
* Correctly parse octal numbers per the YAML 1.2 spec
* Treat "null" keys in a map as valid

## Tests

Existing and new unit tests pass.
2024-10-17 13:13:30 +00:00
Pieter Noordhuis e4d039a1aa
Handle normalization of `dyn.KindTime` into an any type (#1836)
## Changes

The issue reported in #1828 illustrates how using a YAML timestamp-like
value (a date in this case) causes an issue during conversion to and
from the typed configuration tree.

We use the `AsAny()` function on the `dyn.Value` when normalizing for
the `any` type. We only use the `any` type for variable values, because
they can assume every type. The `AsAny()` function returns a `time.Time`
for the time value during conversion **to** the typed configuration
tree. Upon conversion **from** the typed configuration tree back into
the dynamic configuration tree, we cannot distinguish a `time.Time`
struct from any other struct.

To address this, we use the underlying string value of the time value
when we normalize for the `any` type.

Fixes #1828.

## Tests

Existing unit tests pass
2024-10-17 10:00:40 +00:00
Lennart Kats 2765a41cf2
Merge remote-tracking branch 'databricks/main' into cp-summary-with-urls 2024-10-17 10:25:25 +02:00
Lennart Kats aea4a6e634
Styling fix 2024-10-17 10:22:16 +02:00
shreyas-goenka cc112961ce
Fix `TestAccFsMkdirWhenFileExistsAtPath` in isolated Azure environments (#1833)
## Changes
This test passes on normal `azure-prod` but started to fail on
`azure-prod-is`, which is the isolated version of azure-prod. This PR
patches the test to include the error returned from the cloud setup in
`azure-prod-is`.

## Tests
The test passes now on `azure-prod-is`.
2024-10-16 12:50:17 +00:00
Andrew Nester 0753dfe2f4
Fixed unmarshalling json input into `interface{}` type (#1832)
## Changes
Fixed unmarshalling json input into `interface{}` type

Commands like `api post` support free form request input, so it should
be unmarshaled correctly

## Tests
Added regression test + E2E test pass
2024-10-15 12:10:02 +00:00
shreyas-goenka ab20624206
Assert SDK version is consistent in the CLI generation process (#1814)
## Changes
Followup from
https://github.com/databricks/cli/pull/1809#discussion_r1790045086. User
will see the following error if the SDK version does not match during
CLI code generation.

```
--- FAIL: TestConsistentDatabricksSdkVersion (1.34s)
    info_test.go:53: 
                Error Trace:    /Users/shreyas.goenka/cli/internal/build/info_test.go:53
                Error:          Not equal: 
                                expected: "0c86ea6dbd9a730c24ff0d4e509603e476955ac5"
                                actual  : "6f6b1371e640f2dfeba72d365ac566368656f6b6"
                            
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1 +1 @@
                                -0c86ea6dbd9a730c24ff0d4e509603e476955ac5
                                +6f6b1371e640f2dfeba72d365ac566368656f6b6
                Test:           TestConsistentDatabricksSdkVersion
                Messages:       please update the SDK version before generating the CLI
```

## Tests
Manually asserted that:
1. Generating the CLI without the SDK bump causes an error.
2. Generating the CLI after the SDK bump works as expected.
3. The test works even when the SDK is pinned to a specific commit like
`v0.47.1-0.20241002195128-6cecc224cbf7`
2024-10-14 16:19:48 +00:00
Lennart Kats 6218539a5e
Merge remote-tracking branch 'databricks/main' into cp-summary-with-urls 2024-10-11 19:29:03 +02:00
Andrew Nester f0e2981596
Added JSON input validation for CLI commands (#1771)
## Changes
Added JSON input validation for CLI commands. Now when invalid JSON
passed as a payload to CLI commands, CLI performs input normalisation
and detects if there are any mismatches such as incorrect types, unknown
fields and etc.

This diagnostic information is printed in standard error output and does
not block command execution, so the change is backward compatible.

Fixes #1769 #1764 #1625 #1560


## Tests
Added unit tests

```
andrew.nester@HFW9Y94129 ~ % databricks jobs create --json '{"seeti}'
Error: error decoding JSON at (inline):1:2: unexpected EOF


andrew.nester@HFW9Y94129 ~ % databricks jobs create --json '{"seeti": true}'
Warning: unknown field: seeti
  in (inline):1:9

Error: Job settings must be specified.
```

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-10-11 14:39:53 +00:00
Lennart Kats d54f641640
Fix test name 2024-10-11 14:43:36 +02:00
Lennart Kats ef2400fb76
Merge remote-tracking branch 'databricks/main' into cp-summary-with-urls 2024-10-11 14:43:33 +02:00
Lennart Kats (databricks) 08a0d083c3
Ignore metastore permission error during template generation (#1819)
## Changes

This extends the `{{default_catalog}}` helper in templates to ignore any
`PERMISSION_DENIED` error. We're still reviewing when exactly this error
occurs, but if it does, it should not break templates. We should fall
back to assuming there's no default catalog (and no UC) instead.

## Testing

I have not been able to reproduce this issue, but there is a customer
report about "access denied to clusters that don't have unity catalog
enabled" being returned on a non-UC workspace. The error code in this PR
corresponds to that message.

## Next steps
We'll work together with the UC team to review if this error even makes
sense for this API. If that discussion leads to a behavior change in the
API we can update the CLI code again.
2024-10-11 12:28:56 +00:00
Andrew Nester 845d23ac21
Fixed typo in converting cluster permissions (#1826)
## Changes
Fixed typo in converting cluster permissions
2024-10-10 14:10:16 +00:00
Pieter Noordhuis b9085de533
Remove unused `IS_OWNER` constant (#1823)
## Changes

Leftover from #1386.

## Tests

All tests pass (indicating it really wasn't used).
2024-10-10 13:43:21 +00:00
Pieter Noordhuis 3270afaff4
Move utility functions dealing with IAM to libs/iamutil (#1820)
## Changes

The two functions `GetShortUserName` and `IsServicePrincipal` are
unrelated to auth or the purpose of the auth package. This change moves
them into their own package and updates `IsServicePrincipal` to take an
`*iam.User` argument instead of a string username.

## Tests

Tests pass.
2024-10-10 13:02:25 +00:00
Lennart Kats (databricks) e885794722
Show actionable errors for collaborative deployment scenarios (#1386)
## Changes

This adds diagnostics for collaborative (production) deployment
scenarios, including:

- Bob deploys a bundle that is normally deployed by Alice, but this
fails because Bob can't write to `/Users/Alice/.bundle`.
- Charlie deploys a bundle that is normally deployed by Alice, but this
fails because he can't create a new pipeline where Alice would be the
owner.
- Alice deploys a bundle where she didn't list herself as one of the
CAN_MANAGE users in permissions. That can work, but is probably a
mistake.

## Tests

Unit tests, manual testing.
2024-10-10 11:18:23 +00:00
Andrew Nester c92c67afc9
[Release] Release v0.230.0 (#1817)
Notable changes for Databricks Asset Bundles:

Workspace paths are automatically prefixed with `/Workspace`. In
addition, all usage of path strings such as
`/Workspace/${workspace.root_path}/...` in bundle configuration is
automatically replaced with `${workspace.root_path}/...` and generates a
warning as part of bundle validate.

More details can be find here:
https://docs.databricks.com/en/release-notes/dev-tools/bundles.html#workspace-paths-will-be-automatically-prefixed

Bundles:
* Add an error if state files grow bigger than the export limit
([#1795](https://github.com/databricks/cli/pull/1795)).
* Always prepend bundle remote paths with /Workspace
([#1724](https://github.com/databricks/cli/pull/1724)).
* Add resource path field to bundle workspace configuration
([#1800](https://github.com/databricks/cli/pull/1800)).
* Add validation for files with a `.(resource-name).yml` extension
([#1780](https://github.com/databricks/cli/pull/1780)).

Internal:
* Remove deprecated or readonly fields from the bundle schema
([#1809](https://github.com/databricks/cli/pull/1809)).

API Changes:
* Changed `databricks git-credentials create`, `databricks
git-credentials delete`, `databricks git-credentials get`, `databricks
git-credentials list`, `databricks git-credentials update` commands .
* Changed `databricks repos create`, `databricks repos delete`,
`databricks repos get`, `databricks repos update` command .

OpenAPI commit 0c86ea6dbd9a730c24ff0d4e509603e476955ac5 (2024-10-02)
Dependency updates:
* Upgrade TF provider to 1.53.0
([#1815](https://github.com/databricks/cli/pull/1815)).
* Bump golang.org/x/term from 0.24.0 to 0.25.0
([#1811](https://github.com/databricks/cli/pull/1811)).
* Bump golang.org/x/text from 0.18.0 to 0.19.0
([#1812](https://github.com/databricks/cli/pull/1812)).
* Bump github.com/databricks/databricks-sdk-go from 0.47.0 to 0.48.0
([#1810](https://github.com/databricks/cli/pull/1810)).

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-10-09 13:06:57 +00:00
Pieter Noordhuis ae568743c5
Upgrade TF provider to 1.53.0 (#1815)
## Changes

See
https://github.com/databricks/terraform-provider-databricks/releases/tag/v1.53.0

## Tests

Integration tests pass.
2024-10-08 11:41:32 +00:00
shreyas-goenka 88318d384a
Remove deprecated or readonly fields from the bundle schema (#1809)
## Changes
We do not need the user to specify these fields in their bundle
configuration, so we remove them from the JSON schema.

## Tests
Manually and end to end tests. The JSON schema has also been regenerated
after these changes.
2024-10-07 15:13:42 +00:00
dependabot[bot] 7a2141fc5b
Bump github.com/databricks/databricks-sdk-go from 0.47.0 to 0.48.0 (#1810)
Bumps
[github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go)
from 0.47.0 to 0.48.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/releases">github.com/databricks/databricks-sdk-go's
releases</a>.</em></p>
<blockquote>
<h2>v0.48.0</h2>
<h3>Internal Changes</h3>
<ul>
<li>Update SDK to latest OpenAPI spec (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1057">#1057</a>).</li>
</ul>
<p>Note: This release contains breaking changes, please see the API
changes below for more details.</p>
<h3>API Changes:</h3>
<ul>
<li>Added <code>DefaultSourceCodePath</code> and <code>Resources</code>
fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#App">apps.App</a>.</li>
<li>Added <code>Resources</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#CreateAppRequest">apps.CreateAppRequest</a>.</li>
<li>Added <code>Resources</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#UpdateAppRequest">apps.UpdateAppRequest</a>.</li>
<li>Added <code>Schema</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#CreatePipeline">pipelines.CreatePipeline</a>.</li>
<li>Added <code>Schema</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#EditPipeline">pipelines.EditPipeline</a>.</li>
<li>Added <code>Schema</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#PipelineSpec">pipelines.PipelineSpec</a>.</li>
<li>[Breaking] Changed <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateCredentialsRequest">workspace.CreateCredentialsRequest</a>.</li>
<li>[Breaking] Changed <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to type <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#DeleteCredentialsRequest">workspace.DeleteCredentialsRequest</a>.</li>
<li>[Breaking] Changed <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to return <code>any</code>.</li>
<li>Changed <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to type <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsRequest">workspace.GetCredentialsRequest</a>.</li>
<li>Changed <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to type <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to return <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsResponse">workspace.GetCredentialsResponse</a>.</li>
<li>[Breaking] Changed <code>List</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to return <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ListCredentialsResponse">workspace.ListCredentialsResponse</a>.</li>
<li>Changed <code>List</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to type <code>List</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service.</li>
<li>Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to type <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to return <code>any</code>.</li>
<li>[Breaking] Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateCredentialsRequest">workspace.UpdateCredentialsRequest</a>.</li>
<li>Changed <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to type <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateRepoRequest">workspace.CreateRepoRequest</a>.</li>
<li>[Breaking] Changed <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to return <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateRepoResponse">workspace.CreateRepoResponse</a>.</li>
<li>[Breaking] Changed <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to return <code>any</code>.</li>
<li>Changed <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to type <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to return <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetRepoResponse">workspace.GetRepoResponse</a>.</li>
<li>Changed <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to type <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service.</li>
<li>Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to return <code>any</code>.</li>
<li>[Breaking] Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateRepoRequest">workspace.UpdateRepoRequest</a>.</li>
<li>Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to type <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>CredentialId</code> and
<code>GitProvider</code> fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateCredentialsResponse">workspace.CreateCredentialsResponse</a>
to be required.</li>
<li>Changed <code>CredentialId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CredentialInfo">workspace.CredentialInfo</a>
to be required.</li>
<li>Changed <code>CredentialId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsResponse">workspace.GetCredentialsResponse</a>
to be required.</li>
<li>Changed <code>Patterns</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#SparseCheckout">workspace.SparseCheckout</a>
to type <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#List">workspace.List</a>.</li>
<li>Changed <code>Patterns</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#SparseCheckoutUpdate">workspace.SparseCheckoutUpdate</a>
to type <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#List">workspace.List</a>.</li>
<li>[Breaking] Changed <code>GitProvider</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateCredentialsRequest">workspace.UpdateCredentialsRequest</a>
to be required.</li>
</ul>
<p>OpenAPI SHA: 0c86ea6dbd9a730c24ff0d4e509603e476955ac5, Date:
2024-10-02</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/blob/main/CHANGELOG.md">github.com/databricks/databricks-sdk-go's
changelog</a>.</em></p>
<blockquote>
<h2>[Release] Release v0.48.0</h2>
<h3>Internal Changes</h3>
<ul>
<li>Update SDK to latest OpenAPI spec (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1057">#1057</a>).</li>
</ul>
<p>Note: This release contains breaking changes, please see the API
changes below for more details.</p>
<h3>API Changes:</h3>
<ul>
<li>Added <code>DefaultSourceCodePath</code> and <code>Resources</code>
fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#App">apps.App</a>.</li>
<li>Added <code>Resources</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#CreateAppRequest">apps.CreateAppRequest</a>.</li>
<li>Added <code>Resources</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#UpdateAppRequest">apps.UpdateAppRequest</a>.</li>
<li>Added <code>Schema</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#CreatePipeline">pipelines.CreatePipeline</a>.</li>
<li>Added <code>Schema</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#EditPipeline">pipelines.EditPipeline</a>.</li>
<li>Added <code>Schema</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#PipelineSpec">pipelines.PipelineSpec</a>.</li>
<li>[Breaking] Changed <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateCredentialsRequest">workspace.CreateCredentialsRequest</a>.</li>
<li>[Breaking] Changed <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to type <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#DeleteCredentialsRequest">workspace.DeleteCredentialsRequest</a>.</li>
<li>[Breaking] Changed <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to return <code>any</code>.</li>
<li>Changed <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to type <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsRequest">workspace.GetCredentialsRequest</a>.</li>
<li>Changed <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to type <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to return <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsResponse">workspace.GetCredentialsResponse</a>.</li>
<li>[Breaking] Changed <code>List</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to return <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ListCredentialsResponse">workspace.ListCredentialsResponse</a>.</li>
<li>Changed <code>List</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to type <code>List</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service.</li>
<li>Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to type <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service to return <code>any</code>.</li>
<li>[Breaking] Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GitCredentialsAPI">w.GitCredentials</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateCredentialsRequest">workspace.UpdateCredentialsRequest</a>.</li>
<li>Changed <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to type <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateRepoRequest">workspace.CreateRepoRequest</a>.</li>
<li>[Breaking] Changed <code>Create</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to return <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateRepoResponse">workspace.CreateRepoResponse</a>.</li>
<li>[Breaking] Changed <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to return <code>any</code>.</li>
<li>Changed <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to type <code>Delete</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to return <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetRepoResponse">workspace.GetRepoResponse</a>.</li>
<li>Changed <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to type <code>Get</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service.</li>
<li>Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to return <code>any</code>.</li>
<li>[Breaking] Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service . New request type is <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateRepoRequest">workspace.UpdateRepoRequest</a>.</li>
<li>Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service to type <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#ReposAPI">w.Repos</a>
workspace-level service.</li>
<li>[Breaking] Changed <code>CredentialId</code> and
<code>GitProvider</code> fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CreateCredentialsResponse">workspace.CreateCredentialsResponse</a>
to be required.</li>
<li>Changed <code>CredentialId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#CredentialInfo">workspace.CredentialInfo</a>
to be required.</li>
<li>Changed <code>CredentialId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#GetCredentialsResponse">workspace.GetCredentialsResponse</a>
to be required.</li>
<li>Changed <code>Patterns</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#SparseCheckout">workspace.SparseCheckout</a>
to type <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#List">workspace.List</a>.</li>
<li>Changed <code>Patterns</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#SparseCheckoutUpdate">workspace.SparseCheckoutUpdate</a>
to type <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#List">workspace.List</a>.</li>
<li>[Breaking] Changed <code>GitProvider</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/workspace#UpdateCredentialsRequest">workspace.UpdateCredentialsRequest</a>
to be required.</li>
</ul>
<p>OpenAPI SHA: 0c86ea6dbd9a730c24ff0d4e509603e476955ac5, Date:
2024-10-02</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="80c963696b"><code>80c9636</code></a>
[Release] Release v0.48.0 (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/issues/1058">#1058</a>)</li>
<li><a
href="6cecc224cb"><code>6cecc22</code></a>
[Internal] Update SDK to latest OpenAPI spec (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/issues/1057">#1057</a>)</li>
<li>See full diff in <a
href="https://github.com/databricks/databricks-sdk-go/compare/v0.47.0...v0.48.0">compare
view</a></li>
</ul>
</details>
<br />

<details>
<summary>Most Recent Ignore Conditions Applied to This Pull
Request</summary>

| Dependency Name | Ignore Conditions |
| --- | --- |
| github.com/databricks/databricks-sdk-go | [>= 0.28.a, < 0.29] |
</details>


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/databricks/databricks-sdk-go&package-manager=go_modules&previous-version=0.47.0&new-version=0.48.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Nester <andrew.nester@databricks.com>
2024-10-07 13:21:05 +00:00
dependabot[bot] 02bfd397ef
Bump golang.org/x/text from 0.18.0 to 0.19.0 (#1812)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.18.0 to
0.19.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="3043346206"><code>3043346</code></a>
x/text: Correct examples in number/doc</li>
<li><a
href="38a95c2d4a"><code>38a95c2</code></a>
all: fix some comments</li>
<li><a
href="20097e45e6"><code>20097e4</code></a>
all: fix printf(var) mistakes detected by latest printf checker</li>
<li>See full diff in <a
href="https://github.com/golang/text/compare/v0.18.0...v0.19.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/text&package-manager=go_modules&previous-version=0.18.0&new-version=0.19.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-07 13:14:02 +00:00
dependabot[bot] 589e5ff1a4
Bump golang.org/x/term from 0.24.0 to 0.25.0 (#1811)
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.24.0 to
0.25.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9d5441ab55"><code>9d5441a</code></a>
go.mod: update golang.org/x dependencies</li>
<li>See full diff in <a
href="https://github.com/golang/term/compare/v0.24.0...v0.25.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/term&package-manager=go_modules&previous-version=0.24.0&new-version=0.25.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-07 12:58:25 +00:00
shreyas-goenka bca9c2eda4
Add validation for files with a `.(resource-name).yml` extension (#1780)
## Changes
We want to encourage a pattern of specifying only a single resource in a
YAML file when the `.(resource-type).yml` extension is used (for
example, `.job.yml`). This convention could allow us to bijectively map
a resource YAML file to its corresponding resource in the Databricks
workspace.

This PR:
1. Emits a recommendation diagnostic when we detect this convention is
being violated. We can promote this to a warning when we want to
encourage this pattern more strongly.
2. Visualises the recommendation diagnostics in the `bundle validate`
command.

**NOTE:** While this PR also shows the recommendation for `.yaml` files,
we do not encourage users to use this extension. We only support it here
since it's part of the YAML standard and some existing users might
already be using `.yaml`.

## Tests
Unit tests and manually. Here's what an example output looks like:
```
Recommendation: define a single job in a file with the .job.yml extension.
  at resources.jobs.bar
     resources.jobs.foo
  in foo.job.yml:13:7
     foo.job.yml:5:7

The following resources are defined or configured in this file:
  - bar (job)
  - foo (job)
```

---------

Co-authored-by: Lennart Kats (databricks) <lennart.kats@databricks.com>
2024-10-07 09:16:20 +00:00
Lennart Kats 65bad56e10
Fix test failure based on code from main 2024-10-04 21:18:03 -07:00
Lennart Kats 7911c672ba
Merge remote-tracking branch 'databricks/main' into cp-summary-with-urls 2024-10-04 20:48:55 -07:00
Lennart Kats 8cd03d118d
Merge branch 'cp-summary-with-urls' of github.com:lennartkats-db/cli into cp-summary-with-urls 2024-10-04 20:48:50 -07:00
Lennart Kats 189d40b1fa
Address reviewer comments 2024-10-04 20:43:00 -07:00
Andrew Nester a8cff48c0b
Always prepend bundle remote paths with /Workspace (#1724)
## Changes
Due to platform changes, all libraries, notebooks and etc. paths used in
Databricks must be started with either /Workspace or /Volumes prefix.

This PR makes sure that all bundle paths are correctly prefixed.

Note: this change is a breaking change if user previously configured and
used `/Workspace/Workspace` folder in their workspace file system or
having `/Workspace/${workspace.root_path}...` pattern configured
anywhere in their bundle config

Fixes: #1751

AI:
- [x] Scan DABs config and error out on
`/Workspace/${workspace.root_path}...` pattern usage

## Tests
Added unit tests

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-10-02 15:34:00 +00:00
Pieter Noordhuis 80d55f4540
Add resource path field to bundle workspace configuration (#1800)
## Changes

Default workspace path for resources with a presence in the workspace
tree.

Note: this path is **not** created automatically (yet). We need this
only for dashboards (so far), so can take care of creation if one or
more dashboards are part of a deployment. This saves an API call for
deployments where this is not necessary.

## Tests

Expanded existing tests.
2024-10-02 13:55:40 +00:00
Andrew Nester 044a00c7f9
Add an error if state files grow bigger than the export limit (#1795)
## Changes
Currently API limits on exporting files from workspaces are set at 10
MBs while uploading to is 500 MBs. We want to prevent users running into
deadlock when they won't be able to pull state file anymore so we
prevent from uploading large state files (over 10 MBs) to Databricks
workspace.
2024-10-02 13:53:24 +00:00
Andrew Nester c28d64f2dc
[Release] Release v0.229.0 (#1801)
Bundles:
* Added support for creating all-purpose clusters
([#1698](https://github.com/databricks/cli/pull/1698)).
* Reduce time until the prompt is shown for bundle run
([#1727](https://github.com/databricks/cli/pull/1727)).
* Use Unity Catalog for pipelines in the default-python template
([#1766](https://github.com/databricks/cli/pull/1766)).
* Add verbose flag to the "bundle deploy" command
([#1774](https://github.com/databricks/cli/pull/1774)).
* Fixed full variable override detection
([#1787](https://github.com/databricks/cli/pull/1787)).
* Add sub-extension to resource files in built-in templates
([#1777](https://github.com/databricks/cli/pull/1777)).
* Fix panic in `apply_presets.go`
([#1796](https://github.com/databricks/cli/pull/1796)).

Internal:
* Assert tokens are redacted in origin URL when username is not
specified ([#1785](https://github.com/databricks/cli/pull/1785)).
* Refactor jobs path translation
([#1782](https://github.com/databricks/cli/pull/1782)).
* Add JobTaskClusterSpec validate mutator
([#1784](https://github.com/databricks/cli/pull/1784)).
* Pin Go toolchain to 1.22.7
([#1790](https://github.com/databricks/cli/pull/1790)).
* Modify SetLocation test utility to take full locations as argument
([#1788](https://github.com/databricks/cli/pull/1788)).
* Simplified isFullVariableOverrideDef implementation
([#1791](https://github.com/databricks/cli/pull/1791)).
* Sort tasks by `task_key` before generating the Terraform configuration
([#1776](https://github.com/databricks/cli/pull/1776)).
* Trim trailing whitespace
([#1794](https://github.com/databricks/cli/pull/1794)).
* Move trampoline code into trampoline package
([#1793](https://github.com/databricks/cli/pull/1793)).
* Rename `RootPath` -> `BundleRootPath`
([#1792](https://github.com/databricks/cli/pull/1792)).

API Changes:
 * Changed `databricks apps delete` command to return .
* Changed `databricks apps deploy` command with new required argument
order.
 * Changed `databricks apps start` command to return .
 * Changed `databricks apps stop` command to return .
 * Added `databricks temporary-table-credentials` command group.
 * Added `databricks serving-endpoints put-ai-gateway` command.
 * Added `databricks disable-legacy-access` command group.
 * Added `databricks account disable-legacy-features` command group.

OpenAPI commit 6f6b1371e640f2dfeba72d365ac566368656f6b6 (2024-09-19)
Dependency updates:
* Upgrade to Go SDK 0.47.0
([#1799](https://github.com/databricks/cli/pull/1799)).
* Upgrade to TF provider 1.52
([#1781](https://github.com/databricks/cli/pull/1781)).
* Bump golang.org/x/mod from 0.20.0 to 0.21.0
([#1758](https://github.com/databricks/cli/pull/1758)).
* Bump github.com/hashicorp/hc-install from 0.7.0 to 0.9.0
([#1772](https://github.com/databricks/cli/pull/1772)).
2024-10-01 13:20:17 +00:00
Andrew Nester 84fc1ed131
Upgrade to Go SDK 0.47.0 (#1799)
## Changes
Upgrade to Go SDK 0.47.0

## Tests
<!-- How is this tested? -->
2024-10-01 10:44:47 +00:00
Lennart Kats (databricks) da3b4f7c72
Fix panic in `apply_presets.go` (#1796)
## Changes

This fixes the user-reported panic in `apply_presets.go`. I'm still
unsure how to reproduce this, since the CLI just reports `ob broken_job
is not defined` when I try to use `bundle deploy` with an empty job.
That said — we may as well be defensive here and I see we have lots of
checks for empty job/cluster/etc. settings scattered throughout our code
base so at least we're somewhat consistent.
2024-09-29 14:08:10 +00:00
Pieter Noordhuis 1d1aa0a416
Rename `RootPath` -> `BundleRootPath` (#1792)
## Changes

After introducing the `SyncRootPath` field on the bundle (#1694), the
previous `RootPath` became ambiguous. Does it mean the bundle root path
or the sync root path? This PR renames to field to `BundleRootPath` to
remove the ambiguity.

## Tests

n/a

---------

Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com>
2024-09-27 10:03:05 +00:00
Pieter Noordhuis 56cd96cb93
Move trampoline code into trampoline package (#1793)
## Changes

Doing this to make room for PyDABs under `bundle/python`.

## Tests

n/a
2024-09-27 09:32:54 +00:00
Pieter Noordhuis a1dca56abf
Trim trailing whitespace (#1794)
## Changes

Trailing whitespace is trimmed per the VS Code settings for this
repository.

## Tests

n/a
2024-09-27 09:30:39 +00:00
shreyas-goenka 4e8e027380
Sort tasks by `task_key` before generating the Terraform configuration (#1776)
## Changes
Sort the tasks in the resultant `bundle.tf.json`. This is important
because configuration from one task can leak into another if the tasks
are not sorted.

For more details see:
1.
https://github.com/databricks/terraform-provider-databricks/issues/3951
2.
https://github.com/databricks/terraform-provider-databricks/issues/4011

## Tests
Unit test and manually.

For manual testing I used the following configuration:
```
resources:
  jobs:
    foo:
      tasks: 
        - task_key: task-Z
          notebook_task: 
            notebook_path: nb.py
            source: GIT
          existing_cluster_id: 0715-133738-ju0ma84z

        - task_key: task-1
          notebook_task: 
            notebook_path: ${workspace.file_path}/local.py
            source: WORKSPACE
          existing_cluster_id: 0715-133738-ju0ma84z
          depends_on: 
            - task_key: task-Z


      git_source: 
        git_provider: gitHub
        git_url: https://github.com/shreyas-goenka/job-source-tmp.git
        git_branch: main
```

Steps (1):
1. Deploy this bundle.
2. Comment out "source: GIT"
3. Deploy again

Before:
Deploying this bundle twice would fail. This is because the "source:
GIT" would carry over to the next deployment.

After:
There was no error on the subsequent deployment.

Steps (2):
1. Deploy once
2. Deploy again

Before:
Works correctly but leads to a update API call every time.

After:
No diff is detected by terraform.
2024-09-26 13:22:22 +00:00
Andrew Nester 66f2ba64a8
Simplified isFullVariableOverrideDef implementation (#1791)
## Changes
Simplified isFullVariableOverrideDef implementation

Follow up on https://github.com/databricks/cli/pull/1787
2024-09-26 12:55:07 +00:00
dependabot[bot] 94d8c3ba1e
Bump github.com/hashicorp/hc-install from 0.7.0 to 0.9.0 (#1772)
Bumps
[github.com/hashicorp/hc-install](https://github.com/hashicorp/hc-install)
from 0.7.0 to 0.9.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/hashicorp/hc-install/releases">github.com/hashicorp/hc-install's
releases</a>.</em></p>
<blockquote>
<h2>v0.9.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Finish Release of 0.8.1 by updating VERSION by <a
href="https://github.com/mutahhir"><code>@​mutahhir</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/248">hashicorp/hc-install#248</a></li>
<li>build(deps): bump golang.org/x/mod from 0.20.0 to 0.21.0 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/242">hashicorp/hc-install#242</a></li>
<li>docs: Update release instructions by <a
href="https://github.com/radeksimko"><code>@​radeksimko</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/249">hashicorp/hc-install#249</a></li>
<li>Prepare for v0.9.0 release by <a
href="https://github.com/mutahhir"><code>@​mutahhir</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/250">hashicorp/hc-install#250</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/hashicorp/hc-install/compare/v0.8.1...v0.9.0">https://github.com/hashicorp/hc-install/compare/v0.8.1...v0.9.0</a></p>
<h2>v0.8.1</h2>
<h2>What's Changed</h2>
<ul>
<li>Add artifacts manifest (automatically generated) by <a
href="https://github.com/jeanneryan"><code>@​jeanneryan</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/235">hashicorp/hc-install#235</a></li>
<li>build(deps): bump golang.org/x/mod from 0.19.0 to 0.20.0 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/236">hashicorp/hc-install#236</a></li>
<li>build(deps): Bump workflows to latest trusted versions by <a
href="https://github.com/hashicorp-tsccr"><code>@​hashicorp-tsccr</code></a>
in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/237">hashicorp/hc-install#237</a></li>
<li>build(deps): Bump workflows to latest trusted versions by <a
href="https://github.com/hashicorp-tsccr"><code>@​hashicorp-tsccr</code></a>
in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/238">hashicorp/hc-install#238</a></li>
<li>build(deps): bump hashicorp/action-setup-bob from 2.1.0 to 2.1.1 in
the github-actions-backward-compatible group by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/241">hashicorp/hc-install#241</a></li>
<li>httpclient: Reuse existing configured logger by <a
href="https://github.com/radeksimko"><code>@​radeksimko</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/240">hashicorp/hc-install#240</a></li>
<li>build(deps): Bump workflows to latest trusted versions by <a
href="https://github.com/hashicorp-tsccr"><code>@​hashicorp-tsccr</code></a>
in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/243">hashicorp/hc-install#243</a></li>
<li>Nightly and PR builds: fix &quot;no space left on device&quot; on
macOS runner by <a
href="https://github.com/kmoe"><code>@​kmoe</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/245">hashicorp/hc-install#245</a></li>
<li>Update CONTRIBUTING.md to add clean up step by <a
href="https://github.com/mutahhir"><code>@​mutahhir</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/246">hashicorp/hc-install#246</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/jeanneryan"><code>@​jeanneryan</code></a> made
their first contribution in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/235">hashicorp/hc-install#235</a></li>
<li><a href="https://github.com/mutahhir"><code>@​mutahhir</code></a>
made their first contribution in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/246">hashicorp/hc-install#246</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/hashicorp/hc-install/compare/v0.8.0...v0.8.1">https://github.com/hashicorp/hc-install/compare/v0.8.0...v0.8.1</a></p>
<h2>v0.8.0</h2>
<p>ENHANCEMENTS:</p>
<ul>
<li>Add retries for HTTP operations by <a
href="https://github.com/james0209"><code>@​james0209</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/218">hashicorp/hc-install#218</a></li>
<li>Allow <code>LicenseDir</code> field for non-enterprise usage by <a
href="https://github.com/james0209"><code>@​james0209</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/214">hashicorp/hc-install#214</a></li>
</ul>
<p>BUG FIXES:</p>
<ul>
<li>[fix] include custom url's &quot;path&quot; when creating Archive
URL by <a
href="https://github.com/james0209"><code>@​james0209</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/234">hashicorp/hc-install#234</a></li>
</ul>
<p>INTERNAL:</p>
<ul>
<li>[chore] Remove unused variable by <a
href="https://github.com/james0209"><code>@​james0209</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/215">hashicorp/hc-install#215</a></li>
<li>build(deps): bump github.com/hashicorp/go-retryablehttp from 0.7.6
to 0.7.7 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/221">hashicorp/hc-install#221</a></li>
<li>build(deps): bump github.com/hashicorp/go-version from 1.6.0 to
1.7.0 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/216">hashicorp/hc-install#216</a></li>
<li>build(deps): bump golang.org/x/mod from 0.17.0 to 0.18.0 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/223">hashicorp/hc-install#223</a></li>
<li>build(deps): bump golang.org/x/mod from 0.18.0 to 0.19.0 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/229">hashicorp/hc-install#229</a></li>
<li>build(deps): bump hashicorp/action-setup-bob from 2.0.0 to 2.0.3 in
the github-actions-backward-compatible group by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/220">hashicorp/hc-install#220</a></li>
<li>build(deps): bump hashicorp/action-setup-bob from 2.0.3 to 2.1.0 in
the github-actions-backward-compatible group by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/222">hashicorp/hc-install#222</a></li>
<li>build(deps): bump hashicorp/actions-packaging-linux from 1.7 to 1.8
in the github-actions-backward-compatible group by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/224">hashicorp/hc-install#224</a></li>
<li>build(deps): Bump workflows to latest trusted versions by <a
href="https://github.com/hashicorp-tsccr"><code>@​hashicorp-tsccr</code></a>
in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/219">hashicorp/hc-install#219</a></li>
<li>build(deps): Bump workflows to latest trusted versions by <a
href="https://github.com/hashicorp-tsccr"><code>@​hashicorp-tsccr</code></a>
in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/226">hashicorp/hc-install#226</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="157a802cb6"><code>157a802</code></a>
Merge pull request <a
href="https://redirect.github.com/hashicorp/hc-install/issues/250">#250</a>
from hashicorp/release-0.9.0</li>
<li><a
href="4c734fc034"><code>4c734fc</code></a>
Prepare for v0.9.0 release</li>
<li><a
href="d78b32850e"><code>d78b328</code></a>
Merge pull request <a
href="https://redirect.github.com/hashicorp/hc-install/issues/249">#249</a>
from hashicorp/d-contributing-md-update</li>
<li><a
href="34f38b0890"><code>34f38b0</code></a>
docs: Update release instructions</li>
<li><a
href="6a5aa830d9"><code>6a5aa83</code></a>
build(deps): bump golang.org/x/mod from 0.20.0 to 0.21.0 (<a
href="https://redirect.github.com/hashicorp/hc-install/issues/242">#242</a>)</li>
<li><a
href="1784fccf08"><code>1784fcc</code></a>
Merge pull request <a
href="https://redirect.github.com/hashicorp/hc-install/issues/248">#248</a>
from hashicorp/revert-version-contents</li>
<li><a
href="ea2c69b3af"><code>ea2c69b</code></a>
Finish Release of 0.8.1 by updating VERSION</li>
<li><a
href="4f3e00edd9"><code>4f3e00e</code></a>
Releasing 0.8.1</li>
<li><a
href="c6d1ced5b4"><code>c6d1ced</code></a>
Merge pull request <a
href="https://redirect.github.com/hashicorp/hc-install/issues/246">#246</a>
from hashicorp/update-contributing</li>
<li><a
href="eea12f14a6"><code>eea12f1</code></a>
Update CONTRIBUTING.md to add clean up step</li>
<li>Additional commits viewable in <a
href="https://github.com/hashicorp/hc-install/compare/v0.7.0...v0.9.0">compare
view</a></li>
</ul>
</details>
<br />

<details>
<summary>Most Recent Ignore Conditions Applied to This Pull
Request</summary>

| Dependency Name | Ignore Conditions |
| --- | --- |
| github.com/hashicorp/hc-install | [>= 0.8.a, < 0.9] |
</details>


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/hashicorp/hc-install&package-manager=go_modules&previous-version=0.7.0&new-version=0.9.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-26 06:29:34 +00:00
dependabot[bot] 875b112f80
Bump golang.org/x/mod from 0.20.0 to 0.21.0 (#1758)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.20.0 to
0.21.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="46a3137dae"><code>46a3137</code></a>
zip: set GIT_DIR in test when using bare repositories</li>
<li><a
href="3afcd4e90a"><code>3afcd4e</code></a>
go.mod: set go version to 1.22.0</li>
<li><a
href="b1d336cfca"><code>b1d336c</code></a>
go.mod: update required go version to go1.22</li>
<li>See full diff in <a
href="https://github.com/golang/mod/compare/v0.20.0...v0.21.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/mod&package-manager=go_modules&previous-version=0.20.0&new-version=0.21.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Nester <andrew.nester@databricks.com>
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-09-26 06:07:01 +00:00
shreyas-goenka 495040e4cd
Modify SetLocation test utility to take full locations as argument (#1788)
I plan to use this in https://github.com/databricks/cli/pull/1780, to
set the line and column numbers as well for the locations.

gopatch file used:
```
@@
var x expression
var y expression
var z expression
@@
-bundletest.SetLocation(x, y, z)
+bundletest.SetLocation(x, y, []dyn.Location{{File: z}})
```
2024-09-25 16:13:48 +00:00
Pieter Noordhuis 7f1121d8d8
Pin Go toolchain to 1.22.7 (#1790)
## Changes

Relates to https://github.com/databricks/cli/pull/1758.

More information about toolchains:
* https://go.dev/blog/toolchain
* https://go.dev/doc/toolchain

We need to specify the toolchain as we need to bump Go to 1.22.0 for the
`mod` upgrade and want to use the latest toolchain on the 1.22 series.

## Tests

The previous release was made with Go 1.22.7 so we should continue to
use it.
2024-09-25 15:45:28 +00:00