Commit Graph

689 Commits

Author SHA1 Message Date
shreyas-goenka b2cb691988
Add alias for mlops-stack template URL (#869)
## Changes
Allows users to initialize mlops-stack by running `bundle init
mlops-stack`

## Tests
Manually

```
shreyas.goenka@THW32HFW6T playground % cli bundle init            
Template to use [default-python]: mlops-stack
Project Name [my-mlops-project]: ^C
shreyas.goenka@THW32HFW6T playground % cli bundle init mlops-stack
Project Name [my-mlops-project]: ^C
```
2023-10-16 08:36:01 +00:00
hectorcast-db 36f30c8b47
Update Go SDK to 0.23.0 and use custom marshaller (#772)
## Changes
Update Go SDK to 0.23.0 and use custom marshaller.
## Tests
* Run unit tests

* Run nightly

* Manual test:
```
./cli jobs create --json @myjob.json
```
with 
```
{
    "name": "my-job-marshal-test-go",
    "tasks": [{
        "task_key": "testgomarshaltask",
        "new_cluster": {
            "num_workers": 0,
            "spark_version": "10.4.x-scala2.12",
            "node_type_id": "Standard_DS3_v2"
        },
        "libraries": [
            {
                "jar": "dbfs:/max/jars/exampleJarTask.jar"
            }
        ],
        "spark_jar_task": {
            "main_class_name":  "com.databricks.quickstart.exampleTask"
        }
    }]
}
```
Main branch:
```
Error: Cluster validation error: Missing required field: settings.cluster_spec.new_cluster.size
```
This branch:
```
{
  "job_id":<jobid>
}
```

---------

Co-authored-by: Miles Yucht <miles@databricks.com>
2023-10-16 06:56:06 +00:00
Andrew Nester ff01898b61
Use already instantiated WorkspaceClient in sync command (#867)
## Changes
Since we use `root.MustWorkspaceClient` now, we should use already
initialised version of WorkspaceClient instead of instantiating a new
one.

Fixes #836
2023-10-13 13:04:15 +00:00
dependabot[bot] c0903b8b7b
Bump golang.org/x/net from 0.16.0 to 0.17.0 (#863)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.16.0 to
0.17.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b225e7ca6d"><code>b225e7c</code></a>
http2: limit maximum handler goroutines to MaxConcurrentStreams</li>
<li>See full diff in <a
href="https://github.com/golang/net/compare/v0.16.0...v0.17.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/net&package-manager=go_modules&previous-version=0.16.0&new-version=0.17.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/databricks/cli/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-13 08:01:42 +00:00
Andrew Nester bb92e0c0f0
Define goreleaser job output (#861)
## Changes
Define goreleaser job output
2023-10-11 13:50:55 +00:00
Andrew Nester 420a01e67f
Release v0.207.1 (#860)
CLI:
* Improve `workspace import` command by allowing references to local
files for content ([#793](https://github.com/databricks/cli/pull/793)).
* Add `--file` flag to workspace export command
([#794](https://github.com/databricks/cli/pull/794)).
* Ensure profile flag is respected for sync command
([#837](https://github.com/databricks/cli/pull/837)).
* Add hint to delete sync snapshot if parsing fails
([#853](https://github.com/databricks/cli/pull/853)).
* Use profile information when getting a token using the CLI
([#855](https://github.com/databricks/cli/pull/855)).

Bundles:
* Minor template tweaks
([#832](https://github.com/databricks/cli/pull/832)).
* Fixed using repo files as pipeline libraries
([#847](https://github.com/databricks/cli/pull/847)).
* Support .gitignore syntax in sync section and make sure it works
recursively ([#854](https://github.com/databricks/cli/pull/854)).
* Allow target overrides for sync section
([#856](https://github.com/databricks/cli/pull/856)).

Internal:
* Fix import export integration tests on windows
([#842](https://github.com/databricks/cli/pull/842)).
* Fix workspace import test
([#844](https://github.com/databricks/cli/pull/844)).
* Automatically create a release PR in homebrew-tap repo
([#841](https://github.com/databricks/cli/pull/841)).


Dependency updates:
* Bump golang.org/x/term from 0.12.0 to 0.13.0
([#852](https://github.com/databricks/cli/pull/852)).
* Bump golang.org/x/mod from 0.12.0 to 0.13.0
([#851](https://github.com/databricks/cli/pull/851)).
* Bump golang.org/x/sync from 0.3.0 to 0.4.0
([#849](https://github.com/databricks/cli/pull/849)).
* Bump golang.org/x/oauth2 from 0.12.0 to 0.13.0
([#850](https://github.com/databricks/cli/pull/850)).
2023-10-11 12:29:41 +00:00
hectorcast-db 77101c9b85
Use profile information when getting a token using the CLI (#855)
## Changes
Use stored profile information when the user provides the profile flag
when using the `databricks auth token` command.

## Tests
Run the command with and without the profile flag

```
./cli auth token
Databricks Host: https://e2-dogfood.staging.cloud.databricks.com/
{
  "access_token": "****",
  "token_type": "Bearer",
  "expiry": "2023-10-10T14:24:11.85617+02:00"
}%

./cli auth token --profile DEFAULT
{
  "access_token": "*****",
  "token_type": "Bearer",
  "expiry": "2023-10-10T14:24:11.85617+02:00"
}%

./cli auth token https://e2-dogfood.staging.cloud.databricks.com/
{
  "access_token": "*****",
  "token_type": "Bearer",
  "expiry": "2023-10-11T09:24:55.046029+02:00"
}%   

./cli auth token --profile DEFAULT https://e2-dogfood.staging.cloud.databricks.com/
Error: providing both a profile and a host parameters is not supported
```
2023-10-11 11:12:18 +00:00
Andrew Nester 943ea89728
Allow target overrides for sync section (#856)
## Changes
Allow target overrides for sync section

## Tests
Added tests
2023-10-10 15:18:18 +00:00
Andrew Nester 803ecb5efd
Automatically create a release PR in homebrew-tap repo (#841)
## Changes
Automatically create a release PR in homebrew-tap repo

## Tests
Ran ` act -j create-homebrew-tap-release-pr -s DECO_GITHUB_TOKEN="$(gh
auth token)"`

Result: https://github.com/databricks/homebrew-tap/pull/29
2023-10-10 10:58:02 +00:00
Andrew Nester 8c1441ff71
Support .gitignore syntax in sync section and make sure it works recursively (#854)
Fixes #815
2023-10-10 08:45:15 +00:00
dependabot[bot] 8131a8b5fa
Bump golang.org/x/oauth2 from 0.12.0 to 0.13.0 (#850)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from
0.12.0 to 0.13.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="3c5dbf08cc"><code>3c5dbf0</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="11625ccb95"><code>11625cc</code></a>
google: add authorized_user conditional to
Credentials.UniverseDomain</li>
<li><a
href="8d6d45b6cd"><code>8d6d45b</code></a>
google: add Credentials.UniverseDomain to support TPC</li>
<li><a
href="43b6a7ba19"><code>43b6a7b</code></a>
google: adding support for external account authorized user</li>
<li><a
href="14b275c918"><code>14b275c</code></a>
oauth2: workaround misspelling of verification_uri</li>
<li><a
href="18352fc433"><code>18352fc</code></a>
google/internal/externalaccount: adding BYOID Metrics</li>
<li><a
href="9095a51613"><code>9095a51</code></a>
oauth2: clarify error if endpoint missing DeviceAuthURL</li>
<li><a
href="2d9e4a2adf"><code>2d9e4a2</code></a>
oauth2/google: remove meta validations for aws external credentials</li>
<li><a
href="55cd552a36"><code>55cd552</code></a>
oauth2: support PKCE</li>
<li><a
href="e3fb0fb3af"><code>e3fb0fb</code></a>
oauth2: support device flow</li>
<li>See full diff in <a
href="https://github.com/golang/oauth2/compare/v0.12.0...v0.13.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/oauth2&package-manager=go_modules&previous-version=0.12.0&new-version=0.13.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-09 15:27:15 +00:00
shreyas-goenka 043e54950d
Add hint to delete sync snapshot if parsing fails (#853)
## Changes
Example error:
```
Error: error parsing existing sync state. Please delete your existing sync snapshot file (.databricks/sync-snapshots/f3c00bc127903f9b.json) and retry: invalid sync state representation. Remote file footxt is missing the corresponding local file
```

## Tests
Manually
2023-10-09 15:26:46 +00:00
dependabot[bot] b4086e4f1d
Bump golang.org/x/sync from 0.3.0 to 0.4.0 (#849)
Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.3.0 to
0.4.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="22ba2078e1"><code>22ba207</code></a>
singleflight: add panicError.Unwrap method</li>
<li>See full diff in <a
href="https://github.com/golang/sync/compare/v0.3.0...v0.4.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/sync&package-manager=go_modules&previous-version=0.3.0&new-version=0.4.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-09 12:58:22 +00:00
dependabot[bot] 8dcba1810a
Bump golang.org/x/mod from 0.12.0 to 0.13.0 (#851)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.12.0 to
0.13.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="5b692803cf"><code>5b69280</code></a>
modfile: use new go version string format in error message</li>
<li><a
href="273ef6c30f"><code>273ef6c</code></a>
go.mod: update to go 1.18 and x/tools v0.13.0</li>
<li>See full diff in <a
href="https://github.com/golang/mod/compare/v0.12.0...v0.13.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/mod&package-manager=go_modules&previous-version=0.12.0&new-version=0.13.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-09 12:44:05 +00:00
dependabot[bot] fa8ce65d29
Bump golang.org/x/term from 0.12.0 to 0.13.0 (#852)
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.12.0 to
0.13.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ea6303ae75"><code>ea6303a</code></a>
go.mod: update golang.org/x dependencies</li>
<li>See full diff in <a
href="https://github.com/golang/term/compare/v0.12.0...v0.13.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/term&package-manager=go_modules&previous-version=0.12.0&new-version=0.13.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-09 12:43:45 +00:00
Andrew Nester ad4b476270
Ensure profile flag is respected for sync command (#837)
## Changes
Fixes #836 

## Tests
Manually running `sync` command with and without the flag

Integration tests pass as well

```
--- PASS: TestAccSyncFullFileSync (13.38s)
PASS
coverage: 39.1% of statements in ./...
ok      github.com/databricks/cli/internal      14.148s coverage: 39.1% of statements in ./...


--- PASS: TestAccSyncIncrementalFileSync (11.38s)
PASS
coverage: 39.1% of statements in ./...
ok      github.com/databricks/cli/internal      11.674s coverage: 39.1% of statements in ./...
```
2023-10-09 10:37:18 +00:00
Andrew Nester 8d8de3f509
Fixed using repo files as pipeline libraries (#847)
## Changes
Fixed using repo files as pipeline libraries

## Tests
Added regression test
2023-10-09 10:10:28 +00:00
shreyas-goenka 054df2b58b
Fix workspace import test (#844)
Windows and unix have different new line characters. Separating the
string assertions here to make the test pass.
2023-10-06 18:09:56 +00:00
shreyas-goenka 847b6f4bc3
Fix import export integration tests on windows (#842)
We should be using the path package here because they are paths in WSFS
2023-10-06 10:28:18 +00:00
shreyas-goenka 1e9dbcfa2a
Add `--file` flag to workspace export command (#794)
This PR:
1. Adds the `--file` flag to the workspace export command. This allows
you to specify a output file to write to.
2. Adds e2e integration tests for the workspace export command
2023-10-05 13:20:33 +00:00
shreyas-goenka caade735e3
Improve `workspace import` command by allowing references to local files for content (#793)
## Changes
This PR makes a few really important QOL improvements to the `workspace
import` command.

They are:
1. Adds the `--file` flag, which allows a user to specify a file to read
the content from.
2. Wraps the most common error first time users of this command will run
into with a helpful hint.
3. Minor changes to the command Use string changing `PATH` ->
`TARGET_PATH`


## Tests
Integration tests. The newly added integration tests that check the
--file flag works as expected for both `SOURCE` and `AUTO` format.

Skipped the other formats because the API behaviour for them is
straightforward.
2023-10-05 12:48:59 +00:00
Lennart Kats (databricks) 0894584132
Minor template tweaks (#832)
## Changes

Minor tweaks to the template.
2023-10-04 15:27:09 +00:00
Andrew Nester 2ce4f74f9c
Release v0.207.0 (#839)
CLI:
* Refactor change computation for sync
([#785](https://github.com/databricks/cli/pull/785)).

Bundles:
* Allow digits in the generated short name
([#820](https://github.com/databricks/cli/pull/820)).
* Emit an error when incompatible all purpose cluster used with Python
wheel tasks ([#823](https://github.com/databricks/cli/pull/823)).
* Use normalized short name for tag value in development mode
([#821](https://github.com/databricks/cli/pull/821)).
* Added `python.DetectInterpreters` and other utils
([#805](https://github.com/databricks/cli/pull/805)).
* Mark artifacts properties as optional
([#834](https://github.com/databricks/cli/pull/834)).
* Added support for glob patterns in pipeline libraries section
([#833](https://github.com/databricks/cli/pull/833)).

Internal:
* Run tests to verify backend tag validation behavior
([#814](https://github.com/databricks/cli/pull/814)).
* Library to validate and normalize cloud specific tags
([#819](https://github.com/databricks/cli/pull/819)).
* Added test to submit and run various Python tasks on multiple DBR
versions ([#806](https://github.com/databricks/cli/pull/806)).
* Create a release PR in setup-cli repo on tag push
([#827](https://github.com/databricks/cli/pull/827)).

API Changes:
* Changed `databricks account metastore-assignments list` command to
return .
* Changed `databricks jobs cancel-all-runs` command with new required
argument order.
 * Added `databricks account o-auth-published-apps` command group.
* Changed `databricks serving-endpoints query` command . New request
type is .
 * Added `databricks serving-endpoints patch` command.
 * Added `databricks credentials-manager` command group.
 * Added `databricks settings` command group.
 * Changed `databricks clean-rooms list` command to require request of .
* Changed `databricks statement-execution execute-statement` command
with new required argument order.

OpenAPI commit bcbf6e851e3d82fd910940910dd31c10c059746c (2023-10-02)
Dependency updates:
* Bump github.com/google/uuid from 1.3.0 to 1.3.1
([#825](https://github.com/databricks/cli/pull/825)).
* Updated Go SDK to 0.22.0
([#831](https://github.com/databricks/cli/pull/831)).
2023-10-04 14:34:59 +00:00
Andrew Nester 706393b64f
Create a release PR in setup-cli repo on tag push (#827)
## Changes
Create a release PR in setup-cli repo on tag push
2023-10-04 14:03:37 +00:00
Andrew Nester aa54a8665a
Added support for glob patterns in pipeline libraries section (#833)
## Changes
Now it's possible to specify glob pattern in pipeline libraries section
and DAB will add all matched files as libraries

```
  pipelines:
    dummy:
      name: " DLT with Python files"
      target: "dlt_python_files"
      libraries:
        - file:
            path: ./*.py
```

## Tests
Added unit test
2023-10-04 13:23:13 +00:00
Andrew Nester 9b6a847178
Mark artifacts properties as optional (#834)
## Changes
Mark artifacts properties as optional

Fixes #816
2023-10-03 13:59:28 +00:00
shreyas-goenka 40ae23bb33
Refactor change computation for sync (#785)
## Changes
This PR pays some tech debt by refactoring sync diff computation into
interfaces that are more robust.

Specifically:
1. Refactor the single diff computation function into a `SnapshotState`
class that computes the target state only based on the current local
files making it more robust and not carrying over state from previous
iterations.
2. Adds new validations for the sync state which make sure that the
invariants that downstream code expects are actually held true. This
prevents a class of issues where these invariants break and the
synchroniser behaves unexpectedly.

Note, this does not change the existing schema for the snapshot, only
the way the diff is computed, and thus is backwards compatible (ie does
not require a schema version bump).

## Tests
<!-- How is this tested? -->
2023-10-03 13:47:46 +00:00
Andrew Nester e1d1e95525
Updated Go SDK to 0.22.0 (#831)
## Changes
Updated Go SDK to 0.22.0
2023-10-03 11:46:16 +00:00
Andrew Nester 79e271f859
Added test to submit and run various Python tasks on multiple DBR versions (#806)
## Changes
These tests allow us to get information for execution context
(PYTHONPATH, CWD) for various Python tasks and different cluster setups.

Note: this test won't be executed automatically as part of nightly
builds since it requires RUN_PYTHON_TASKS_TEST env to be executed.

## Tests
Integration test run successfully.

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2023-10-03 11:18:55 +00:00
dependabot[bot] 452565cbd3
Bump github.com/google/uuid from 1.3.0 to 1.3.1 (#825)
Bumps [github.com/google/uuid](https://github.com/google/uuid) from
1.3.0 to 1.3.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/google/uuid/releases">github.com/google/uuid's
releases</a>.</em></p>
<blockquote>
<h2>v1.3.1</h2>
<h2><a
href="https://github.com/google/uuid/compare/v1.3.0...v1.3.1">1.3.1</a>
(2023-08-18)</h2>
<h3>Bug Fixes</h3>
<ul>
<li>Use .EqualFold() to parse urn prefixed UUIDs (<a
href="https://redirect.github.com/google/uuid/issues/118">#118</a>) (<a
href="574e687494">574e687</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/google/uuid/blob/master/CHANGELOG.md">github.com/google/uuid's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/google/uuid/compare/v1.3.0...v1.3.1">1.3.1</a>
(2023-08-18)</h2>
<h3>Bug Fixes</h3>
<ul>
<li>Use .EqualFold() to parse urn prefixed UUIDs (<a
href="https://redirect.github.com/google/uuid/issues/118">#118</a>) (<a
href="574e687494">574e687</a>)</li>
</ul>
<h2>Changelog</h2>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b3cae7c306"><code>b3cae7c</code></a>
chore(master): release 1.3.1 (<a
href="https://redirect.github.com/google/uuid/issues/127">#127</a>)</li>
<li><a
href="7b8f57cbcc"><code>7b8f57c</code></a>
fix(ci): switch to release-please app (<a
href="https://redirect.github.com/google/uuid/issues/126">#126</a>)</li>
<li><a
href="e69e468f7c"><code>e69e468</code></a>
chore(ci): configure release-please, update contrib (<a
href="https://redirect.github.com/google/uuid/issues/122">#122</a>)</li>
<li><a
href="97c970d2a5"><code>97c970d</code></a>
chore(ci): add apidiff check for API compatibility (<a
href="https://redirect.github.com/google/uuid/issues/123">#123</a>)</li>
<li><a
href="53f93a84e6"><code>53f93a8</code></a>
docs: change godoc URL in README (<a
href="https://redirect.github.com/google/uuid/issues/124">#124</a>)</li>
<li><a
href="0b416df5d0"><code>0b416df</code></a>
docs: update link to RFC 4122 (<a
href="https://redirect.github.com/google/uuid/issues/93">#93</a>)</li>
<li><a
href="75e1ac5ba6"><code>75e1ac5</code></a>
docs: shell format go tool command (<a
href="https://redirect.github.com/google/uuid/issues/111">#111</a>)</li>
<li><a
href="d71986992e"><code>d719869</code></a>
docs: fix typo node_js docs (<a
href="https://redirect.github.com/google/uuid/issues/117">#117</a>)</li>
<li><a
href="574e687494"><code>574e687</code></a>
fix: Use .EqualFold() to parse urn prefixed UUIDs (<a
href="https://redirect.github.com/google/uuid/issues/118">#118</a>)</li>
<li><a
href="c91929c8bf"><code>c91929c</code></a>
Merge pull request <a
href="https://redirect.github.com/google/uuid/issues/121">#121</a> from
noahdietz/setup-codeowners-and-actions</li>
<li>Additional commits viewable in <a
href="https://github.com/google/uuid/compare/v1.3.0...v1.3.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/google/uuid&package-manager=go_modules&previous-version=1.3.0&new-version=1.3.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-03 10:48:40 +00:00
Serge Smertin 7d0f170eee
Added `python.DetectInterpreters` and other utils (#805)
This PR adds a few utilities related to Python interpreter detection:

- `python.DetectInterpreters` to detect all Python versions available in
`$PATH` by executing every matched binary name with `--version` flag.
- `python.DetectVirtualEnvPath` to detect if there's any child virtual
environment in `src` directory
- `python.DetectExecutable` to detect if there's python3 installed
either by `which python3` command or by calling
`python.DetectInterpreters().AtLeast("v3.8")`

To be merged after https://github.com/databricks/cli/pull/804, as one of
the steps to get https://github.com/databricks/cli/pull/637 in, as
previously discussed.
2023-10-03 10:47:09 +00:00
Pieter Noordhuis f1b068cefe
Use normalized short name for tag value in development mode (#821)
## Changes

The jobs backend propagates job tags to the underlying cloud provider's
resources. As such, they need to match the constraints a cloud provider
places on tag values. The display name can contain anything. With this
change, we modify the tag value to equal the short name as used in the
name prefix.

Additionally, we leverage tag normalization as introduced in #819 to
make sure characters that aren't accepted are removed before using the
value as a tag value.

This is a new stab at #810 and should completely eliminate this class of
problems.

## Tests

Tests pass.
2023-10-02 06:58:51 +00:00
Andrew Nester 775251d0dc
Emit an error when incompatible all purpose cluster used with Python wheel tasks (#823)
## Changes
Follow up for https://github.com/databricks/cli/pull/807 to also
validate configuration if existing cluster id is used.

## Tests
Added unit tests
2023-09-29 12:19:05 +00:00
Pieter Noordhuis 4226c88e98
Library to validate and normalize cloud specific tags (#819)
## Changes

Prompted by the proposed fix for a tagging-related problem in #810, I
investigated how tag validation works. This turned out to be quite a bit
more complex than anticipated. Tags at the job level (or cluster level)
are passed through to the underlying compute infrastructure and as such
are tested against cloud-specific validation rules. GCP appears to be
the most restrictive. It would be disappointing to always restrict to
`\w+`, so this package implements validation and normalization rules for
each cloud. It can pick the right cloud to use using a Go SDK
configuration.

## Tests

Exhaustive unit tests. The regular expressions were pulled by #814.
2023-09-29 08:49:08 +00:00
Pieter Noordhuis 3685eb16f4
Run tests to verify backend tag validation behavior (#814)
## Changes

Validation rules on tags are different per cloud (they are passed
through to the underlying clusters and as such must comply with
cloud-specific validation rules). This change adds tests to confirm the
current behavior to ensure the normalization we can apply is in line
with how the backend behaves.

## Tests

The new integration tests pass (tested locally).
2023-09-29 08:38:06 +00:00
Pieter Noordhuis 30b4b8ce58
Allow digits in the generated short name (#820)
## Changes

Digits were previously replaced by `_`.

## Tests

Additional test cases with uncommon variations of email addresses.
2023-09-29 06:58:40 +00:00
Andrew Nester f31e8b446c
Release v0.206.0 (#811)
Bundles:
* Enable target overrides for pipeline clusters
([#792](https://github.com/databricks/cli/pull/792)).
* Add support for regex patterns in template schema
([#768](https://github.com/databricks/cli/pull/768)).
* Make the default `databricks bundle init` template more
self-explanatory ([#796](https://github.com/databricks/cli/pull/796)).
* Make a notebook wrapper for Python wheel tasks optional
([#797](https://github.com/databricks/cli/pull/797)).
* Added a warning when Python wheel wrapper needs to be used
([#807](https://github.com/databricks/cli/pull/807)).

Internal:
* Added `process.Background()` and `process.Forwarded()`
([#804](https://github.com/databricks/cli/pull/804)).

Dependency updates:
* Bump golang.org/x/term from 0.11.0 to 0.12.0
([#798](https://github.com/databricks/cli/pull/798)).
* Bump github.com/hashicorp/terraform-exec from 0.18.1 to 0.19.0
([#801](https://github.com/databricks/cli/pull/801)).
* Bump golang.org/x/oauth2 from 0.11.0 to 0.12.0
([#802](https://github.com/databricks/cli/pull/802)).
2023-09-27 10:57:34 +00:00
Serge Smertin 7171874db0
Added `process.Background()` and `process.Forwarded()` (#804)
## Changes
This PR adds higher-level wrappers for calling subprocesses. One of the
steps to get https://github.com/databricks/cli/pull/637 in, as
previously discussed.

The reason to add `process.Forwarded()` is to proxy Python's `input()`
calls from a child process seamlessly. Another use-case is plugging in
`less` as a pager for the list results.

## Tests
`make test`
2023-09-27 09:04:44 +00:00
Andrew Nester 3ee89c41da
Added a warning when Python wheel wrapper needs to be used (#807)
## Changes
Added a warning when Python wheel wrapper needs to be used

## Tests
Added unit tests + manual run with different bundle configurations
2023-09-27 08:26:59 +00:00
dependabot[bot] 99cc01c91c
Bump golang.org/x/oauth2 from 0.11.0 to 0.12.0 (#802)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from
0.11.0 to 0.12.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="07085280e4"><code>0708528</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="a835fc4358"><code>a835fc4</code></a>
oauth2: move global auth style cache to be per-Config</li>
<li>See full diff in <a
href="https://github.com/golang/oauth2/compare/v0.11.0...v0.12.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/oauth2&package-manager=go_modules&previous-version=0.11.0&new-version=0.12.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-26 15:40:46 +00:00
dependabot[bot] 7cc8b4c17f
Bump github.com/hashicorp/terraform-exec from 0.18.1 to 0.19.0 (#801)
Bumps
[github.com/hashicorp/terraform-exec](https://github.com/hashicorp/terraform-exec)
from 0.18.1 to 0.19.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/hashicorp/terraform-exec/releases">github.com/hashicorp/terraform-exec's
releases</a>.</em></p>
<blockquote>
<h2>v0.19.0</h2>
<p>ENHANCEMENTS:</p>
<ul>
<li>Add support for <code>terraform test</code> command (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/398">#398</a>)</li>
<li>Add support for <code>-refresh-only</code> flag for
<code>Plan</code> and <code>Apply</code> methods. (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/402">#402</a>)</li>
<li>Add support for <code>-destroy</code> flag for <code>Apply</code>
(<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/292">#292</a>)</li>
</ul>
<p>BUG FIXES:</p>
<ul>
<li>Fix bug in which the <code>TF_WORKSPACE</code> env var was set to an
empty string, instead of being unset as intended. (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/388">#388</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/hashicorp/terraform-exec/blob/main/CHANGELOG.md">github.com/hashicorp/terraform-exec's
changelog</a>.</em></p>
<blockquote>
<h1>0.19.0 (August 31, 2023)</h1>
<p>ENHANCEMENTS:</p>
<ul>
<li>Add support for <code>terraform test</code> command (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/398">#398</a>)</li>
<li>Add support for <code>-refresh-only</code> flag for
<code>Plan</code> and <code>Apply</code> methods. (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/402">#402</a>)</li>
<li>Add support for <code>-destroy</code> flag for <code>Apply</code>
(<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/292">#292</a>)</li>
</ul>
<p>BUG FIXES:</p>
<ul>
<li>Fix bug in which the <code>TF_WORKSPACE</code> env var was set to an
empty string, instead of being unset as intended. (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/388">#388</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="44a44d702b"><code>44a44d7</code></a>
v0.19.0 [skip ci]</li>
<li><a
href="b55ace7c3f"><code>b55ace7</code></a>
Bump github.com/hashicorp/hc-install from 0.5.2 to 0.6.0 (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/407">#407</a>)</li>
<li><a
href="1581bc7ae3"><code>1581bc7</code></a>
Fix test file extensions in e2e tests (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/408">#408</a>)</li>
<li><a
href="abef36d075"><code>abef36d</code></a>
Bump github.com/zclconf/go-cty from 1.13.3 to 1.14.0 (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/406">#406</a>)</li>
<li><a
href="08caa9a099"><code>08caa9a</code></a>
Update CHANGELOG.md</li>
<li><a
href="f1b3c610bb"><code>f1b3c61</code></a>
Update CHANGELOG.md</li>
<li><a
href="ec5a394c63"><code>ec5a394</code></a>
feat: add refresh-only flag for plan and apply methods (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/402">#402</a>)</li>
<li><a
href="2210f68c70"><code>2210f68</code></a>
apply: allow use of -destroy flag for compatible terraform versions (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/292">#292</a>)</li>
<li><a
href="6a6a61a676"><code>6a6a61a</code></a>
Bump actions/checkout from 3.5.3 to 3.6.0 (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/404">#404</a>)</li>
<li><a
href="cff1cb8939"><code>cff1cb8</code></a>
tfexec: Initial test command support (<a
href="https://redirect.github.com/hashicorp/terraform-exec/issues/400">#400</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/hashicorp/terraform-exec/compare/v0.18.1...v0.19.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/hashicorp/terraform-exec&package-manager=go_modules&previous-version=0.18.1&new-version=0.19.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-26 14:52:37 +00:00
Andrew Nester 0daa0022af
Make a notebook wrapper for Python wheel tasks optional (#797)
## Changes
Instead of always using notebook wrapper for Python wheel tasks, let's
make this an opt-in option.

Now by default Python wheel tasks will be deployed as is to Databricks
platform.
If notebook wrapper required (DBR < 13.1 or other configuration
differences), users can provide a following experimental setting

```
experimental:
  python_wheel_wrapper: true
```

Fixes #783,
https://github.com/databricks/databricks-asset-bundles-dais2023/issues/8

## Tests
Added unit tests.

Integration tests passed for both cases

```
    helpers.go:163: [databricks stdout]: Hello from my func
    helpers.go:163: [databricks stdout]: Got arguments:
    helpers.go:163: [databricks stdout]: ['my_test_code', 'one', 'two']
    ...
Bundle remote directory is ***/.bundle/ac05d5e8-ed4b-4e34-b3f2-afa73f62b021
Deleted snapshot file at /var/folders/nt/xjv68qzs45319w4k36dhpylc0000gp/T/TestAccPythonWheelTaskDeployAndRunWithWrapper3733431114/001/.databricks/bundle/default/sync-snapshots/cac1e02f3941a97b.json
Successfully deleted files!
--- PASS: TestAccPythonWheelTaskDeployAndRunWithWrapper (214.18s)
PASS
coverage: 93.5% of statements in ./...
ok      github.com/databricks/cli/internal/bundle       214.495s        coverage: 93.5% of statements in ./...

```

```
    helpers.go:163: [databricks stdout]: Hello from my func
    helpers.go:163: [databricks stdout]: Got arguments:
    helpers.go:163: [databricks stdout]: ['my_test_code', 'one', 'two']
    ...
Bundle remote directory is ***/.bundle/0ef67aaf-5960-4049-bf1d-dc9e29157421
Deleted snapshot file at /var/folders/nt/xjv68qzs45319w4k36dhpylc0000gp/T/TestAccPythonWheelTaskDeployAndRunWithoutWrapper2340216760/001/.databricks/bundle/default/sync-snapshots/edf0b322cee93b13.json
Successfully deleted files!
--- PASS: TestAccPythonWheelTaskDeployAndRunWithoutWrapper (192.36s)
PASS
coverage: 93.5% of statements in ./...
ok      github.com/databricks/cli/internal/bundle       195.130s        coverage: 93.5% of statements in ./...

```
2023-09-26 14:32:20 +00:00
dependabot[bot] e1b5912f59
Bump golang.org/x/term from 0.11.0 to 0.12.0 (#798)
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.11.0 to
0.12.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="f413282cd8"><code>f413282</code></a>
go.mod: update golang.org/x dependencies</li>
<li>See full diff in <a
href="https://github.com/golang/term/compare/v0.11.0...v0.12.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/term&package-manager=go_modules&previous-version=0.11.0&new-version=0.12.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-26 14:27:02 +00:00
Lennart Kats (databricks) 0c1516c4ba
Make the default `databricks bundle init` template more self-explanatory (#796)
This makes the default-python template more self-explanatory and adds a
few other tweaks for a better out-of-the-box experience.
2023-09-26 09:12:34 +00:00
shreyas-goenka 757d5efe8d
Add support for regex patterns in template schema (#768)
## Changes
This PR introduces support for regex pattern validation in our custom
jsonschema validator. This allows us to fail early if a user enters an
invalid value for a field.

For example, now this is what initializing the default template looks
like with an invalid project name:
```
shreyas.goenka@THW32HFW6T bricks % cli bundle init
Template to use [default-python]: 
Unique name for this project [my_project]: (_*_)
Error: invalid value for project_name: (_*_). Must consist of letter and underscores only.
```

## Tests
New unit tests and manually.
2023-09-25 09:53:38 +00:00
Pieter Noordhuis ee30277119
Enable target overrides for pipeline clusters (#792)
## Changes

This is a follow-up to #658 and #779 for jobs.

This change applies label normalization the same way the backend does.

## Tests

Unit and config loading tests.
2023-09-21 19:21:20 +00:00
Andrew Nester c65e59751b
Release v0.205.2 (#791)
CLI:
* Prompt for profile only in interactive mode
([#788](https://github.com/databricks/cli/pull/788)).

Internal:
* Added setup Python action
([#789](https://github.com/databricks/cli/pull/789)).
2023-09-21 14:46:35 +00:00
Andrew Nester aa9c2a1eab
Prompt for profile only in interactive mode (#788)
## Changes
Do not prompt for profiles if not in interactive mode

## Tests
Running sample Go code

```
cmd := exec.Command("databricks", "auth", "login", "--host", "***")
out, err := cmd.CombinedOutput()
```
Before the change
```
Error: ^D

exit status 1
```

After
```
No error (empty output)
```
2023-09-21 12:38:45 +00:00
Andrew Nester 4a9dcd3231
Added setup Python action (#789)
## Changes
Added setup Python action

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2023-09-21 12:21:39 +00:00
Pieter Noordhuis 46996b884d
Release v0.205.1 (#787)
Bundles:
* Use enums for default python template
([#765](https://github.com/databricks/cli/pull/765)).
* Make bundle deploy work if no resources are defined
([#767](https://github.com/databricks/cli/pull/767)).
* Added support for experimental scripts section
([#632](https://github.com/databricks/cli/pull/632)).
* Error when unknown keys are encounters during template execution
([#766](https://github.com/databricks/cli/pull/766)).
* Fall back to full Git clone if shallow clone is not supported
([#775](https://github.com/databricks/cli/pull/775)).
* Enable environment overrides for job tasks
([#779](https://github.com/databricks/cli/pull/779)).
* Increase timeout waiting for job run to 1 day
([#786](https://github.com/databricks/cli/pull/786)).

Internal:
* Update Go SDK to v0.19.3 (unreleased)
([#778](https://github.com/databricks/cli/pull/778)).
2023-09-20 11:46:47 +00:00