e40b53f4ce
fix: Update artifacts
2025-01-20 12:26:56 +01:00
f04a841e15
Merge branch 'main' of github.com:databricks/cli into feat/config-reference-doc-autogen
2025-01-20 12:26:20 +01:00
98244606b3
Upgrade TF provider to 1.63.0 ( #2162 )
...
## Changes
No significant changes to call out for DABs.
2025-01-16 12:04:00 +00:00
13181fc5a7
fix: More ducmentation + tests
2025-01-15 22:10:39 +01:00
d27282c3a4
fix: Rename fields to make it more clear
2025-01-15 20:00:23 +01:00
26466d23b4
fix: Remove type inconsistency in maps
2025-01-15 19:47:39 +01:00
14721deb3a
fix: More documentation for internal types
2025-01-15 19:15:57 +01:00
b91bb4d167
fix: Remove extra dependency
2025-01-15 18:59:07 +01:00
33bd3b81cb
fix: Typo
2025-01-15 16:50:24 +01:00
e5497f75b8
fix: Move templates to separate files
2025-01-15 16:48:07 +01:00
1b73105457
fix: Split annotations/main.go in 2 files
2025-01-15 16:26:19 +01:00
c74ab7566f
fix: Stage output changes
2025-01-15 16:23:45 +01:00
14b2c86d61
fix: Add `vendor` step to `make docs`
2025-01-15 16:21:45 +01:00
72e677d0ac
Bump github.com/databricks/databricks-sdk-go from 0.54.0 to 0.55.0 ( #2126 )
...
Bumps
[github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go )
from 0.54.0 to 0.55.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/releases ">github.com/databricks/databricks-sdk-go's
releases</a>.</em></p>
<blockquote>
<h2>v0.55.0</h2>
<h3>Internal Changes</h3>
<ul>
<li>Bump staticcheck to 0.5.1 and add go 1.23 test coverage (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1106 ">#1106</a>).</li>
<li>Bump x/net, x/crypto dependencies (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1107 ">#1107</a>).</li>
<li>Create custom codeql.yml (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1114 ">#1114</a>).</li>
<li>Decouple serving and oauth2 package (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1110 ">#1110</a>).</li>
<li>Migrate workflows that need write access to use hosted runners (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1112 ">#1112</a>).</li>
<li>Move package credentials in config (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1115 ">#1115</a>).</li>
<li>Update Queries test (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1104 ">#1104</a>).</li>
</ul>
<h3>API Changes:</h3>
<ul>
<li>Added <code>NoCompute</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#CreateAppRequest ">apps.CreateAppRequest</a>.</li>
<li>Added <code>HasMore</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#BaseJob ">jobs.BaseJob</a>.</li>
<li>Added <code>HasMore</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#BaseRun ">jobs.BaseRun</a>.</li>
<li>Added <code>PageToken</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#GetJobRequest ">jobs.GetJobRequest</a>.</li>
<li>Added <code>HasMore</code> and <code>NextPageToken</code> fields for
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Job ">jobs.Job</a>.</li>
<li>Added <code>HasMore</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Run ">jobs.Run</a>.</li>
<li>Added <code>RunAs</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#CreatePipeline ">pipelines.CreatePipeline</a>.</li>
<li>Added <code>RunAs</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#EditPipeline ">pipelines.EditPipeline</a>.</li>
<li>Added <code>AuthorizationDetails</code> and <code>EndpointUrl</code>
fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#DataPlaneInfo ">serving.DataPlaneInfo</a>.</li>
<li>[Breaking] Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/oauth2#AccountFederationPolicyAPI ">a.AccountFederationPolicy</a>
account-level service with new required argument order.</li>
<li>[Breaking] Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/oauth2#ServicePrincipalFederationPolicyAPI ">a.ServicePrincipalFederationPolicy</a>
account-level service with new required argument order.</li>
<li>Changed <code>UpdateMask</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/oauth2#UpdateAccountFederationPolicyRequest ">oauth2.UpdateAccountFederationPolicyRequest</a>
to no longer be required.</li>
<li>Changed <code>UpdateMask</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/oauth2#UpdateServicePrincipalFederationPolicyRequest ">oauth2.UpdateServicePrincipalFederationPolicyRequest</a>
to no longer be required.</li>
<li>[Breaking] Changed <code>DaysOfWeek</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#RestartWindow ">pipelines.RestartWindow</a>
to type <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#DayOfWeekList ">pipelines.DayOfWeekList</a>.</li>
</ul>
<p>OpenAPI SHA: 779817ed8d63031f5ea761fbd25ee84f38feec0d, Date:
2025-01-08</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/blob/main/CHANGELOG.md ">github.com/databricks/databricks-sdk-go's
changelog</a>.</em></p>
<blockquote>
<h2>[Release] Release v0.55.0</h2>
<h3>Internal Changes</h3>
<ul>
<li>Bump staticcheck to 0.5.1 and add go 1.23 test coverage (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1106 ">#1106</a>).</li>
<li>Bump x/net, x/crypto dependencies (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1107 ">#1107</a>).</li>
<li>Create custom codeql.yml (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1114 ">#1114</a>).</li>
<li>Decouple serving and oauth2 package (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1110 ">#1110</a>).</li>
<li>Migrate workflows that need write access to use hosted runners (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1112 ">#1112</a>).</li>
<li>Move package credentials in config (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1115 ">#1115</a>).</li>
<li>Update Queries test (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/1104 ">#1104</a>).</li>
</ul>
<h3>API Changes:</h3>
<ul>
<li>Added <code>NoCompute</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#CreateAppRequest ">apps.CreateAppRequest</a>.</li>
<li>Added <code>HasMore</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#BaseJob ">jobs.BaseJob</a>.</li>
<li>Added <code>HasMore</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#BaseRun ">jobs.BaseRun</a>.</li>
<li>Added <code>PageToken</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#GetJobRequest ">jobs.GetJobRequest</a>.</li>
<li>Added <code>HasMore</code> and <code>NextPageToken</code> fields for
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Job ">jobs.Job</a>.</li>
<li>Added <code>HasMore</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Run ">jobs.Run</a>.</li>
<li>Added <code>RunAs</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#CreatePipeline ">pipelines.CreatePipeline</a>.</li>
<li>Added <code>RunAs</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#EditPipeline ">pipelines.EditPipeline</a>.</li>
<li>Added <code>AuthorizationDetails</code> and <code>EndpointUrl</code>
fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#DataPlaneInfo ">serving.DataPlaneInfo</a>.</li>
<li>[Breaking] Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/oauth2#AccountFederationPolicyAPI ">a.AccountFederationPolicy</a>
account-level service with new required argument order.</li>
<li>[Breaking] Changed <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/oauth2#ServicePrincipalFederationPolicyAPI ">a.ServicePrincipalFederationPolicy</a>
account-level service with new required argument order.</li>
<li>Changed <code>UpdateMask</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/oauth2#UpdateAccountFederationPolicyRequest ">oauth2.UpdateAccountFederationPolicyRequest</a>
to no longer be required.</li>
<li>Changed <code>UpdateMask</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/oauth2#UpdateServicePrincipalFederationPolicyRequest ">oauth2.UpdateServicePrincipalFederationPolicyRequest</a>
to no longer be required.</li>
<li>[Breaking] Changed <code>DaysOfWeek</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#RestartWindow ">pipelines.RestartWindow</a>
to type <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#DayOfWeekList ">pipelines.DayOfWeekList</a>.</li>
</ul>
<p>OpenAPI SHA: 779817ed8d63031f5ea761fbd25ee84f38feec0d, Date:
2025-01-08</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b83a7262d5https://redirect.github.com/databricks/databricks-sdk-go/issues/1117 ">#1117</a>)</li>
<li><a
href="23d9c1ea2bhttps://redirect.github.com/databricks/databricks-sdk-go/issues/1115 ">#1115</a>)</li>
<li><a
href="adc94cabf7https://redirect.github.com/databricks/databricks-sdk-go/issues/1110 ">#1110</a>)</li>
<li><a
href="83db3cbdabhttps://redirect.github.com/databricks/databricks-sdk-go/issues/1114 ">#1114</a>)</li>
<li><a
href="2b55375727https://redirect.github.com/databricks/databricks-sdk-go/issues/1 ">#1</a>...</li>
<li><a
href="03fb2681fahttps://redirect.github.com/databricks/databricks-sdk-go/issues/1107 ">#1107</a>)</li>
<li><a
href="28e1a698abhttps://redirect.github.com/databricks/databricks-sdk-go/issues/1106 ">#1106</a>)</li>
<li><a
href="2399d721fehttps://redirect.github.com/databricks/databricks-sdk-go/issues/1104 ">#1104</a>)</li>
<li>See full diff in <a
href="https://github.com/databricks/databricks-sdk-go/compare/v0.54.0...v0.55.0 ">compare
view</a></li>
</ul>
</details>
<br />
<details>
<summary>Most Recent Ignore Conditions Applied to This Pull
Request</summary>
| Dependency Name | Ignore Conditions |
| --- | --- |
| github.com/databricks/databricks-sdk-go | [>= 0.28.a, < 0.29] |
</details>
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Nester <andrew.nester@databricks.com>
2025-01-14 16:02:34 +00:00
1e41e6140d
fix: Apps fixes after merge
2025-01-14 12:59:44 +01:00
f684afe452
Merge branch 'main' of github.com:databricks/cli into feat/config-reference-doc-autogen
2025-01-14 11:52:44 +01:00
913e10a037
Added support for Databricks Apps in DABs ( #1928 )
...
## Changes
Now it's possible to configure new `app` resource in bundle and point it
to the custom `source_code_path` location where Databricks App code is
defined.
On `databricks bundle deploy` DABs will create an app. All consecutive
`databricks bundle deploy` execution will update an existing app if
there are any updated
On `databricks bundle run <my_app>` DABs will execute app deployment. If
the app is not started yet, it will start the app first.
### Bundle configuration
```
bundle:
name: apps
variables:
my_job_id:
description: "ID of job to run app"
lookup:
job: "My Job"
databricks_name:
description: "Name for app user"
additional_flags:
description: "Additional flags to run command app"
default: ""
my_app_config:
type: complex
description: "Configuration for my Databricks App"
default:
command:
- flask
- --app
- hello
- run
- ${var.additional_flags}
env:
- name: DATABRICKS_NAME
value: ${var.databricks_name}
resources:
apps:
my_app:
name: "anester-app" # required and has to be unique
description: "My App"
source_code_path: ./app # required and points to location of app code
config: ${var.my_app_config}
resources:
- name: "my-job"
description: "A job for app to be able to run"
job:
id: ${var.my_job_id}
permission: "CAN_MANAGE_RUN"
permissions:
- user_name: "foo@bar.com"
level: "CAN_VIEW"
- service_principal_name: "my_sp"
level: "CAN_MANAGE"
targets:
dev:
variables:
databricks_name: "Andrew (from dev)"
additional_flags: --debug
prod:
variables:
databricks_name: "Andrew (from prod)"
```
### Execution
1. `databricks bundle deploy -t dev`
2. `databricks bundle run my_app -t dev`
**If app is started**
```
✓ Getting the status of the app my-app
✓ App is in RUNNING state
✓ Preparing source code for new app deployment.
✓ Deployment is pending
✓ Starting app with command: flask --app hello run --debug
✓ App started successfully
You can access the app at <app-url>
```
**If app is not started**
```
✓ Getting the status of the app my-app
✓ App is in UNAVAILABLE state
✓ Starting the app my-app
✓ App is starting...
....
✓ App is starting...
✓ App is started!
✓ Preparing source code for new app deployment.
✓ Downloading source code from /Workspace/Users/...
✓ Starting app with command: flask --app hello run --debug
✓ App started successfully
You can access the app at <app-url>
```
## Tests
Added unit and config tests + manual test.
```
--- PASS: TestAccDeployBundleWithApp (404.59s)
PASS
coverage: 36.8% of statements in ./...
ok github.com/databricks/cli/internal/bundle 405.035s coverage: 36.8% of statements in ./...
```
2025-01-13 16:43:48 +00:00
2591172370
fix: Circular types
2025-01-13 15:55:49 +01:00
3e0d2324b5
fix: Allow empty fields
2025-01-13 15:28:41 +01:00
ad81e1f0af
fix: Few links updates
2025-01-13 15:16:47 +01:00
5fb4fa000b
fix: Add field-name for some properties
2025-01-10 21:01:15 +01:00
fd88e4c97a
fix: Use markdown from original pages
2025-01-10 20:35:44 +01:00
90cafad8c8
fix: Allow nodes with only description
2025-01-10 19:57:02 +01:00
880a4cf49e
fix: Few extra links
2025-01-10 19:29:16 +01:00
0bd7b524db
fix: Invalid refrences
2025-01-10 19:26:45 +01:00
ee5db187c0
fix: Move logic to separate fiels
2025-01-10 18:33:58 +01:00
4b01f6b1bc
fix: Add links
2025-01-10 18:09:18 +01:00
151a6f86dd
fix: Multiple links
2025-01-10 17:51:22 +01:00
1fbec371e5
fix: Link
2025-01-10 17:47:06 +01:00
d5d433e9d7
fix: More descriptions
2025-01-10 16:37:22 +01:00
954ef76fe9
Merge branch 'main' of github.com:databricks/cli into feat/config-reference-doc-autogen
2025-01-10 16:07:13 +01:00
bad77bd358
fix: Sync annotations
2025-01-10 16:01:52 +01:00
6c5268aaaa
fix: Missing array types
2025-01-10 12:52:20 +01:00
f9278c2b4d
docs: Add override for volume spec
2025-01-08 16:11:58 +01:00
c355fbfcb2
fix: Description of root types with additional properties
2025-01-08 16:11:35 +01:00
02c7df39f6
Add 'experimental/python' support ( #2052 )
...
## Changes
Add `experimental/python` section replacing `experimental/pydabs`.
Add 2 new mutators into existing pipeline:
- `ApplyPythonMutator(load_resources)` - loads resources from Python
code
- `ApplyPythonMutator(apply_mutators)` - transforms existing resources
defined in Python/YAML
Example:
```yaml
experimental:
python:
resources:
- "resources:load_resources"
mutators:
- "mutators:add_email_notifications"
```
## Tests
Unit tests and manually
---------
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2025-01-08 09:29:45 +00:00
fe6ba76b7d
fix: Styling
2025-01-07 16:52:01 +01:00
c6703c1363
fix: Updated styles
2025-01-03 13:52:59 +01:00
39d1e8093f
Enable intrange linter and apply autofix ( #2069 )
...
New construct in Go1.22+ for integer iteration:
https://github.com/ckaznocha/intrange?tab=readme-ov-file#intrange
2025-01-03 09:25:07 +00:00
2aadfcbcb2
feat: Support for resources
2025-01-02 18:24:18 +01:00
541c3e3fb5
feat: More explicit type for arrays
2025-01-02 16:56:36 +01:00
460a4558e0
chore: Extract annotation package
2025-01-02 16:40:06 +01:00
820dd5f34c
feat: Add examples to the docs
2025-01-02 16:13:24 +01:00
0b80784df7
Enable testifylint and fix the issues ( #2065 )
...
## Changes
- Enable new linter: testifylint.
- Apply fixes with --fix.
- Fix remaining issues (mostly with aider).
There were 2 cases we --fix did the wrong thing - this seems to a be a
bug in linter: https://github.com/Antonboom/testifylint/issues/210
Nonetheless, I kept that check enabled, it seems useful, just need to be
fixed manually after autofix.
## Tests
Existing tests
2025-01-02 12:03:41 +01:00
793bf2b995
fix: Empty schema fields in OpenAPI spec ( #2045 )
...
## Changes
1. Removes default yaml-fields during schema generation, caused by [this
PR](https://github.com/databricks/cli/pull/2032 ) (current yaml package
can't read `json` annotations in struct fields)
2. Addresses missing annotations for fields from OpenAPI spec, which are
named differently in go SDK
3. Adds filtering for annotations.yaml to include only CLI package
fields
4. Implements alphabetical sort for yaml keys to avoid unnecessary diff
in PRs
## Tests
Manually tested
2024-12-23 12:08:01 +00:00
40c4b3a40a
Merge branch 'main' of github.com:databricks/cli into feat/config-reference-doc-autogen
2024-12-23 12:25:29 +01:00
dd9f59837e
Upgrade go to 1.23.4 ( #2038 )
...
## Changes
`git grep -l 1.23.2 | xargs -n 1 sed -i '' 's/1.23.2/1.23.4/'`
## Tests
Existing tests
2024-12-20 09:21:36 +00:00
965a3fcd53
Remove dependency on ghodss/yaml ( #2032 )
...
## Changes
I noticed that #1957 took a dep on this library even though we no longer
need it. This change removes the dep and cleans up other (unused) uses
of the library. We originally relied on this library to deserialize
bundle configuration and JSON payloads to non-bundle CLI commands.
Relevant commits:
* The YAML flag was added to support apps (very early), and is not
longer used: e408b701e47fa61987dd46a3
2024-12-19 08:23:05 +00:00
64fa2bfcb7
fix: Small readme
2024-12-18 21:49:42 +01:00
10e0d276c8
fix: Remove extra headings
2024-12-18 21:12:44 +01:00
Ilya Kuznetsov
Andrew Nester
dependabot[bot]
Gleb Kanterov
Denis Bilenko
Pieter Noordhuis