d5b4a595af
Prototype bundle run notebooks results
2024-07-15 17:32:56 +03:00
d30c4c730d
Add new template ( #1578 )
...
## Changes
Add a new hidden experimental template
## Tests
Tested manually
2024-07-08 13:32:56 +00:00
869576e144
Move bespoke status call to main workspace files filer ( #1570 )
...
## Changes
This consolidates the two separate status calls into one.
The extension-aware filer now doesn't need the direct API client anymore
and fully relies on the underlying filer.
## Tests
* Unit tests.
* Ran the filer integration tests manually.
2024-07-05 11:32:29 +00:00
3d8446bbdb
Rewrite local path for libraries in foreach tasks ( #1569 )
...
## Changes
Now local library path in `libraries` section of foreach each tasks are
correctly replaced with remote path for this library when it's uploaded
to Databricks
## Tests
Added unit test
2024-07-05 10:58:28 +00:00
040b374430
Override complex variables with target overrides instead of merging ( #1567 )
...
## Changes
At the moment we merge values of complex variables while more expected
behaviour is overriding the value with the target one.
## Tests
Added unit test
2024-07-04 11:57:29 +00:00
8c3be30093
Use different Go cache key for goreleaser jobs ( #1558 )
...
## Changes
The goreleaser jobs perform a cross-platform build of the main binary
without test files. It should use a different cache than the jobs that
run tests for a single platform.
This change also updates the `release-snapshot` job to use the latest
goreleaser action, as was done in #1477 .
## Tests
Ran `release-snapshot` job from this PR.
2024-07-04 11:39:55 +00:00
80136dea5f
Use Go 1.22 to build and test ( #1562 )
...
## Changes
This has been released for a while. Blog post:
https://go.dev/blog/go1.22 .
## Tests
None besides the unit tests.
2024-07-04 06:54:41 +00:00
324fa2e18b
Update actions/upload-artifact to v4 ( #1559 )
...
## Changes
This addresses a deprecation warning in our GHA output.
Full release notes of v4 at
https://github.com/actions/upload-artifact/releases/tag/v4.0.0
2024-07-04 06:54:30 +00:00
bf275428b6
Release v0.223.1 ( #1565 )
...
This bugfix release fixes missing error messages in v0.223.0.
CLI:
* Fix logic error in
[#1532 ](https://github.com/databricks/cli/pull/1532 )
([#1564 ](https://github.com/databricks/cli/pull/1564 )).
2024-07-03 16:41:55 +00:00
7d2aa35738
Fix logic error in #1532 ( #1564 )
...
## Changes
This snuck into #1532 right before merging. The result is that error
output is no longer logged. This includes actual execution errors as
well as help output if arguments or flags are incorrectly specified.
We don't have test coverage for the `root.Execute` function. This is to
be fixed later.
## Tests
Manually confirmed we observe error output again.
2024-07-03 16:23:19 +00:00
2a73d7788b
Release v0.223.0 ( #1557 )
...
Bundles:
As of this release you can interact with bundles when running the CLI on
DBR (e.g. via the Web Terminal).
* Fix non-default project names not working in dbt-sql template
([#1500 ](https://github.com/databricks/cli/pull/1500 )).
* Improve `bundle validate` output
([#1532 ](https://github.com/databricks/cli/pull/1532 )).
* Fixed resolving variable references inside slice variable
([#1550 ](https://github.com/databricks/cli/pull/1550 )).
* Fixed bundle not loading when empty variable is defined
([#1552 ](https://github.com/databricks/cli/pull/1552 )).
* Use `vfs.Path` for filesystem interaction
([#1554 ](https://github.com/databricks/cli/pull/1554 )).
* Replace `vfs.Path` with extension-aware filer when running on DBR
([#1556 ](https://github.com/databricks/cli/pull/1556 )).
Internal:
* merge.Override: Fix handling of dyn.NilValue
([#1530 ](https://github.com/databricks/cli/pull/1530 )).
* Compare `.Kind()` instead of direct equality checks on a `dyn.Value`
([#1520 ](https://github.com/databricks/cli/pull/1520 )).
* PythonMutator: register product in user agent extra
([#1533 ](https://github.com/databricks/cli/pull/1533 )).
* Ignore `dyn.NilValue` when traversing value from `dyn.Map`
([#1547 ](https://github.com/databricks/cli/pull/1547 )).
* Add extra tests for the sync block
([#1548 ](https://github.com/databricks/cli/pull/1548 )).
* PythonMutator: add diagnostics
([#1531 ](https://github.com/databricks/cli/pull/1531 )).
* PythonMutator: support omitempty in PyDABs
([#1513 ](https://github.com/databricks/cli/pull/1513 )).
* PythonMutator: allow insert 'resources' and 'resources.jobs'
([#1555 ](https://github.com/databricks/cli/pull/1555 )).
2024-07-03 12:24:42 +00:00
f14dded946
Replace `vfs.Path` with extension-aware filer when running on DBR ( #1556 )
...
## Changes
The FUSE mount of the workspace file system on DBR doesn't include file
extensions for notebooks. When these notebooks are checked into a
repository, they do have an extension. PR #1457 added a filer type that
is aware of this disparity and makes these notebooks show up as if they
do have these extensions.
This change swaps out the native `vfs.Path` with one that uses this
filer when running on DBR.
Follow up: consolidate between interfaces exported by `filer.Filer` and
`vfs.Path`.
## Tests
* Unit tests pass
* (Manually ran a snapshot build on DBR against a bundle with notebooks)
---------
Co-authored-by: Andrew Nester <andrew.nester@databricks.com>
2024-07-03 11:55:42 +00:00
b3c044c461
Use `vfs.Path` for filesystem interaction ( #1554 )
...
## Changes
Note: this doesn't cover _all_ filesystem interaction.
To intercept calls where read or stat files to determine their type, we
need a layer between our code and the `os` package calls that interact
with the local file system. Interception is necessary to accommodate
differences between a regular local file system and the FUSE-mounted
Workspace File System when running the CLI on DBR.
This change makes use of #1452 in the bundle struct.
It uses #1525 to access the bundle variable in path rewriting.
## Tests
* Unit tests pass.
* Integration tests pass.
2024-07-03 10:13:22 +00:00
4787edba36
PythonMutator: allow insert 'resources' and 'resources.jobs' ( #1555 )
...
## Changes
Allow insert 'resources' and 'resources.jobs' because they can be absent
in incoming bundle.
## Tests
Unit tests
2024-07-03 08:33:23 +00:00
b9e3c98723
PythonMutator: support omitempty in PyDABs ( #1513 )
...
## Changes
PyDABs output can omit empty sequences/mappings because we don't track
them as optional. There is no semantic difference between empty and
missing, which makes omitting correct. CLI detects that we falsely
modify input resources by deleting all empty collections.
To handle that, we extend `dyn.Override` to allow visitors to ignore
certain deletes. If we see that an empty sequence or mapping is deleted,
we revert such delete.
## Tests
Unit tests
---------
Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com>
2024-07-03 07:22:03 +00:00
5a0a6d7334
PythonMutator: add diagnostics ( #1531 )
...
## Changes
Allow PyDABs to report `dyn.Diagnostics` by writing to
`diagnostics.json` supplied as an argument, similar to `input.json` and
`output.json`
Such errors are not yet properly printed in `databricks bundle
validate`, which will be fixed in a follow-up PR.
## Tests
Unit tests
2024-07-02 15:10:53 +00:00
3d2f7622bc
Fixed bundle not loading when empty variable is defined ( #1552 )
...
## Changes
Fixes #1544
## Tests
Added regression test
2024-07-02 12:40:39 +00:00
0d64975d36
Fixed resolving variable references inside slice variable ( #1550 )
...
## Changes
Fixes #1541
## Tests
Added regression unit test
---------
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-07-02 11:45:16 +00:00
a0df54ac41
Add extra tests for the sync block ( #1548 )
...
## Changes
Issue #1545 describes how a nil entry in the sync block caused an error.
The fix for this issue is in #1547 . This change adds end-to-end test
coverage.
## Tests
New test passes on top of #1547 .
2024-07-01 13:08:50 +00:00
da603c6ead
Ignore `dyn.NilValue` when traversing value from `dyn.Map` ( #1547 )
...
## Changes
The map function ignores cases where either a key in a map is not
present or an index in a sequence is out of bounds. As of recently, we
retain nil values as valid values in a configuration tree. As such, it
makes sense to also ignore cases where a map or sequence is expected but
nil is found. This is semantically no different from an empty map where
a key is not found.
Without this fix, all calls to `dyn.Map` would need to be updated with
nil-checks at every path component.
Related PRs:
* #1507
* #1511
## Tests
Unit tests pass.
2024-07-01 13:00:31 +00:00
e8b76a7f13
Improve `bundle validate` output ( #1532 )
...
## Changes
This combination of changes allows pretty-printing errors happening
during the "load" and "init" phases, including their locations.
Move to render code into a separate module dedicated to rendering
`diag.Diagnostics` in a human-readable format. This will be used for the
`bundle deploy` command.
Preserve the "bundle" value if an error occurs in mutators. Rewrite the
go templates to handle the case when the bundle isn't yet loaded if an
error occurs during loading, that is possible now.
Improve rendering for errors and warnings:
- don't render empty locations
- render "details" for errors if they exist
Add `root.ErrAlreadyPrinted` indicating that the error was already
printed, and the CLI entry point shouldn't print it again.
## Tests
Add tests for output, that are especially handy to detect extra newlines
2024-07-01 09:01:10 +00:00
c7a36921b4
Fix non-default project names not working in dbt-sql template ( #1500 )
...
## Changes
Hello Team,
While tinkering with your solution, I've noticed that profiles provided
in dbt_project.yml and profiles.yml for generated dbt asset bundles. do
not align. This led to the following error, when deploying DAB:
```
+ dbt deps --target=dev
11:24:02 Running with dbt=1.8.2
11:24:02 Warning: No packages were found in packages.yml
11:24:02 Warning: No packages were found in packages.yml
+ dbt seed --target=dev --vars '{ dev_schema: mateusz_kijewski }'
11:24:05 Running with dbt=1.8.2
11:24:05 Encountered an error:
Runtime Error
Could not find profile named 'dbt_sql'
```
I have corrected profile name in profiles.yml.tmpl to the name used in
dbt_project.yml.tmpl. Using the opportunity of forking your repo, I've
also updated tests configuration in model config as starting of dbt v1.8
it's been raising warnings of configuration change from tests to
data_tests
```
11:31:34 [WARNING]: Deprecated functionality
The `tests` config has been renamed to `data_tests`. Please see
https://docs.getdbt.com/docs/build/data-tests#new-data_tests-syntax for more
information.
```
## Tests
<!-- How is this tested? -->
2024-07-01 07:52:22 +00:00
aee3910f3d
PythonMutator: register product in user agent extra ( #1533 )
...
## Changes
Register user agent product following RFC 9110.
See
https://github.com/databricks/terraform-provider-databricks/pull/3520
for Terraform change.
## Tests
Unit tests
2024-07-01 07:46:37 +00:00
4d8eba04cd
Compare `.Kind()` instead of direct equality checks on a `dyn.Value` ( #1520 )
...
## Changes
This PR makes two changes:
1. In https://github.com/databricks/cli/pull/1510 we'll be adding
multiple associated location metadata with a dyn.Value. The Go compiler
does not allow comparing structs if they contain slice values
(presumably due to multiple possible definitions for equality). In
anticipation for adding a `[]dyn.Location` type field to `dyn.Value`
this PR removes all direct comparisons of `dyn.Value` and instead relies
on the kind.
2. Retain location metadata for values in convert.FromTyped. The change
diff is exactly the same as https://github.com/databricks/cli/pull/1523 .
It's been combined with this PR because they both depend on each other
to prevent test failures (forming a test failure deadlock).
Go patch used:
```
@@
var x expression
@@
-x == dyn.InvalidValue
+x.Kind() == dyn.KindInvalid
@@
var x expression
@@
-x != dyn.InvalidValue
+x.Kind() != dyn.KindInvalid
@@
var x expression
@@
-x == dyn.NilValue
+x.Kind() == dyn.KindNil
@@
var x expression
@@
-x != dyn.NilValue
+x.Kind() != dyn.KindNil
```
## Tests
Unit tests and integration tests pass.
2024-06-27 13:28:19 +00:00
dba6164a4c
merge.Override: Fix handling of dyn.NilValue ( #1530 )
...
## Changes
Fix handling of `dyn.NilValue` in `merge.Override` in case `dyn.Value`
has location
## Tests
Unit tests
2024-06-27 09:47:58 +00:00
cdd6fe8cb9
Release v0.222.0 ( #1529 )
...
CLI:
* Add link to documentation for Homebrew installation to README
([#1505 ](https://github.com/databricks/cli/pull/1505 )).
* Fix `databricks configure` to use `DATABRICKS_CONFIG_FILE` environment
variable if exists as config file
([#1325 ](https://github.com/databricks/cli/pull/1325 )).
Bundles:
The Terraform upgrade to v1.48.0 includes a fix for library order not
being respected.
* Fix conditional in query in `default-sql` template
([#1479 ](https://github.com/databricks/cli/pull/1479 )).
* Remove user credentials specified in the Git origin URL
([#1494 ](https://github.com/databricks/cli/pull/1494 )).
* Serialize dynamic value for `bundle validate` output
([#1499 ](https://github.com/databricks/cli/pull/1499 )).
* Override variables with lookup value even if values has default value
set ([#1504 ](https://github.com/databricks/cli/pull/1504 )).
* Pause quality monitors when "mode: development" is used
([#1481 ](https://github.com/databricks/cli/pull/1481 )).
* Return `fs.ModeDir` for Git folders in the workspace
([#1521 ](https://github.com/databricks/cli/pull/1521 )).
* Upgrade TF provider to 1.48.0
([#1527 ](https://github.com/databricks/cli/pull/1527 )).
* Added support for complex variables
([#1467 ](https://github.com/databricks/cli/pull/1467 )).
Internal:
* Add randIntn function
([#1475 ](https://github.com/databricks/cli/pull/1475 )).
* Avoid multiple file tree traversals on bundle deploy
([#1493 ](https://github.com/databricks/cli/pull/1493 )).
* Clean up unused code
([#1502 ](https://github.com/databricks/cli/pull/1502 )).
* Use `dyn.InvalidValue` to indicate absence
([#1507 ](https://github.com/databricks/cli/pull/1507 )).
* Add ApplyPythonMutator
([#1430 ](https://github.com/databricks/cli/pull/1430 )).
* Set bool pointer to disable lock
([#1516 ](https://github.com/databricks/cli/pull/1516 )).
* Allow the any type to be set to nil in `convert.FromTyped`
([#1518 ](https://github.com/databricks/cli/pull/1518 )).
* Properly deal with nil values in `convert.FromTyped`
([#1511 ](https://github.com/databricks/cli/pull/1511 )).
* Return `dyn.InvalidValue` instead of `dyn.NilValue` when errors happen
([#1514 ](https://github.com/databricks/cli/pull/1514 )).
* PythonMutator: replace stdin/stdout with files
([#1512 ](https://github.com/databricks/cli/pull/1512 )).
* Add context type and value to path rewriting
([#1525 ](https://github.com/databricks/cli/pull/1525 )).
API Changes:
* Added schedule CRUD commands to `databricks lakeview`.
* Added subscription CRUD commands to `databricks lakeview`.
* Added `databricks apps start` command.
OpenAPI commit 7437dabb9dadee402c1fc060df4c1ce8cc5369f0 (2024-06-24)
Dependency updates:
* Bump golang.org/x/text from 0.15.0 to 0.16.0
([#1482 ](https://github.com/databricks/cli/pull/1482 )).
* Bump golang.org/x/term from 0.20.0 to 0.21.0
([#1483 ](https://github.com/databricks/cli/pull/1483 )).
* Bump golang.org/x/mod from 0.17.0 to 0.18.0
([#1484 ](https://github.com/databricks/cli/pull/1484 )).
* Bump golang.org/x/oauth2 from 0.20.0 to 0.21.0
([#1485 ](https://github.com/databricks/cli/pull/1485 )).
* Bump github.com/briandowns/spinner from 1.23.0 to 1.23.1
([#1495 ](https://github.com/databricks/cli/pull/1495 )).
* Bump github.com/spf13/cobra from 1.8.0 to 1.8.1
([#1496 ](https://github.com/databricks/cli/pull/1496 )).
* Bump github.com/databricks/databricks-sdk-go from 0.42.0 to 0.43.0
([#1522 ](https://github.com/databricks/cli/pull/1522 )).
2024-06-26 12:24:31 +00:00
5f42791609
Added support for complex variables ( #1467 )
...
## Changes
Added support for complex variables
Now it's possible to add and use complex variables as shown below
```
bundle:
name: complex-variables
resources:
jobs:
my_job:
job_clusters:
- job_cluster_key: key
new_cluster: ${var.cluster}
tasks:
- task_key: test
job_cluster_key: key
variables:
cluster:
description: "A cluster definition"
type: complex
default:
spark_version: "13.2.x-scala2.11"
node_type_id: "Standard_DS3_v2"
num_workers: 2
spark_conf:
spark.speculation: true
spark.databricks.delta.retentionDurationCheck.enabled: false
```
Fixes #1298
- [x] Support for complex variables
- [x] Allow variable overrides (with shortcut) in targets
- [x] Don't allow to provide complex variables via flag or env variable
- [x] Fail validation if complex value is used but not `type: complex`
provided
- [x] Support using variables inside complex variables
## Tests
Added unit tests
---------
Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com>
2024-06-26 10:25:32 +00:00
ce5a3f2ce6
Upgrade TF provider to 1.48.0 ( #1527 )
...
## Changes
This includes a fix for library order not being respected.
## Tests
Manually confirmed the fix works in
https://github.com/databricks/bundle-examples/pull/29 .
2024-06-26 09:29:46 +00:00
482d83cba8
Revert "Retain location metadata for values in `convert.FromTyped`" ( #1528 )
...
## Changes
This reverts commit dac5f09556#1523 ).
Retaining the location for nil values means equality checks no longer
pass.
We need #1520 to be merged first.
## Tests
Integration test `TestAccPythonWheelTaskDeployAndRunWithWrapper`.
2024-06-26 09:26:40 +00:00
dac5f09556
Retain location metadata for values in `convert.FromTyped` ( #1523 )
...
## Changes
There are four different treatments location metadata can receive in the
`convert.FromTyped` method.
1. Location metadata is **retained** for maps, structs and slices if the
value is **not nil**
2. Location metadata is **lost** for maps, structs and slices if the
value is **is nil**
3. Location metadata is **retained** if a scalar type (eg. bool, string
etc) does not change.
4. Location metadata is **lost** if the value for a scalar type changes.
This PR ensures that location metadata is not lost in any case; that is,
it's always preserved.
For (2), this serves as a bug fix so that location information is not
lost on conversion to and from typed for nil values of complex types
(struct, slices, and maps).
For (4) this is a change in semantics. For primitive values modified in
a `typed` mutator, any references to `.Location()` for computed
primitive fields will now return associated YAML location metadata (if
any) instead of an empty location.
While arguable, these semantics are OK since:
1. Situations like these will be rare.
2. Knowing the YAML location (if any) is better than not knowing the
location at all. These locations are typically visible to the user in
errors and warnings.
## Tests
Unit tests
2024-06-25 13:40:21 +00:00
8468878eed
Bump github.com/databricks/databricks-sdk-go from 0.42.0 to 0.43.0 ( #1522 )
...
Bumps
[github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go )
from 0.42.0 to 0.43.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/releases ">github.com/databricks/databricks-sdk-go's
releases</a>.</em></p>
<blockquote>
<h2>v0.43.0</h2>
<p>Major Changes and Improvements:</p>
<ul>
<li>Support partners in user agent for SDK (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/925 ">#925</a>).</li>
<li>Add <code>serverless_compute_id</code> field to the config (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/952 ">#952</a>).</li>
</ul>
<p>Other Changes:</p>
<ul>
<li>Generate from latest spec (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/944 ">#944</a>)
and (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/947 ">#947</a>).</li>
</ul>
<p>API Changes:</p>
<ul>
<li>Changed <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogInfo ">catalog.CatalogInfo</a>
to <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode ">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ExternalLocationInfo ">catalog.ExternalLocationInfo</a>.</li>
<li>Added <code>MaxResults</code> and <code>PageToken</code> fields for
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsRequest ">catalog.ListCatalogsRequest</a>.</li>
<li>Added <code>NextPageToken</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsResponse ">catalog.ListCatalogsResponse</a>.</li>
<li>Added <code>TableServingUrl</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#OnlineTable ">catalog.OnlineTable</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#StorageCredentialInfo ">catalog.StorageCredentialInfo</a>.</li>
<li>Changed <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateCatalog ">catalog.UpdateCatalog</a>
to <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode ">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateExternalLocation ">catalog.UpdateExternalLocation</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateStorageCredential ">catalog.UpdateStorageCredential</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode ">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>CreateSchedule</code>, <code>CreateSubscription</code>,
<code>DeleteSchedule</code>, <code>DeleteSubscription</code>,
<code>GetSchedule</code>, <code>GetSubscription</code>,
<code>List</code>, <code>ListSchedules</code>,
<code>ListSubscriptions</code> and <code>UpdateSchedule</code> methods
for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#LakeviewAPI ">w.Lakeview</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateScheduleRequest ">dashboards.CreateScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateSubscriptionRequest ">dashboards.CreateSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CronSchedule ">dashboards.CronSchedule</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DashboardView ">dashboards.DashboardView</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteScheduleRequest ">dashboards.DeleteScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteSubscriptionRequest ">dashboards.DeleteSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetScheduleRequest ">dashboards.GetScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetSubscriptionRequest ">dashboards.GetSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsRequest ">dashboards.ListDashboardsRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsResponse ">dashboards.ListDashboardsResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesRequest ">dashboards.ListSchedulesRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesResponse ">dashboards.ListSchedulesResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsRequest ">dashboards.ListSubscriptionsRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsResponse ">dashboards.ListSubscriptionsResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Schedule ">dashboards.Schedule</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SchedulePauseStatus ">dashboards.SchedulePauseStatus</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscriber ">dashboards.Subscriber</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscription ">dashboards.Subscription</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberDestination ">dashboards.SubscriptionSubscriberDestination</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberUser ">dashboards.SubscriptionSubscriberUser</a>
and <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#UpdateScheduleRequest ">dashboards.UpdateScheduleRequest</a>
structs.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobEmailNotifications ">jobs.JobEmailNotifications</a>.</li>
<li>Added <code>EnvironmentKey</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunTask ">jobs.RunTask</a>.</li>
<li>Removed <code>ConditionTask</code>, <code>DbtTask</code>,
<code>NotebookTask</code>, <code>PipelineTask</code>,
<code>PythonWheelTask</code>, <code>RunJobTask</code>,
<code>SparkJarTask</code>, <code>SparkPythonTask</code>,
<code>SparkSubmitTask</code>, <code>SqlTask</code> and
<code>Environments</code> fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun ">jobs.SubmitRun</a>.</li>
<li>Added <code>DbtTask</code> and <code>EnvironmentKey</code> field for
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask ">jobs.SubmitTask</a>.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TaskEmailNotifications ">jobs.TaskEmailNotifications</a>.</li>
<li>Added <code>Periodic</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TriggerSettings ">jobs.TriggerSettings</a>.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#WebhookNotifications ">jobs.WebhookNotifications</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfiguration ">jobs.PeriodicTriggerConfiguration</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfigurationTimeUnit ">jobs.PeriodicTriggerConfigurationTimeUnit</a>.</li>
<li>Added <code>ProviderSummary</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#Listing ">marketplace.Listing</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconFile ">marketplace.ProviderIconFile</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconType ">marketplace.ProviderIconType</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderListingSummaryInfo ">marketplace.ProviderListingSummaryInfo</a>.</li>
<li>Added <code>Start</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#AppsAPI ">w.Apps</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#ServingEndpointsDataPlaneAPI ">w.ServingEndpointsDataPlane</a>
workspace-level service.</li>
<li>Added <code>ServicePrincipalId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App ">serving.App</a>.</li>
<li>Added <code>ServicePrincipalName</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App ">serving.App</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#StartAppRequest ">serving.StartAppRequest</a>.</li>
<li>Added <code>QueryNextPage</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#VectorSearchIndexesAPI ">w.VectorSearchIndexes</a>
workspace-level service.</li>
<li>Added <code>QueryType</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexRequest ">vectorsearch.QueryVectorIndexRequest</a>.</li>
<li>Added <code>NextPageToken</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexResponse ">vectorsearch.QueryVectorIndexResponse</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexNextPageRequest ">vectorsearch.QueryVectorIndexNextPageRequest</a>.</li>
</ul>
<p>OpenAPI SHA: 7437dabb9dadee402c1fc060df4c1ce8cc5369f0, Date:
2024-06-25</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/blob/main/CHANGELOG.md ">github.com/databricks/databricks-sdk-go's
changelog</a>.</em></p>
<blockquote>
<h2>0.43.0</h2>
<p>Major Changes and Improvements:</p>
<ul>
<li>Support partners in user agent for SDK (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/925 ">#925</a>).</li>
<li>Add <code>serverless_compute_id</code> field to the config (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/952 ">#952</a>).</li>
</ul>
<p>Other Changes:</p>
<ul>
<li>Generate from latest spec (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/944 ">#944</a>)
and (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/947 ">#947</a>).</li>
</ul>
<p>API Changes:</p>
<ul>
<li>Changed <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogInfo ">catalog.CatalogInfo</a>
to <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode ">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ExternalLocationInfo ">catalog.ExternalLocationInfo</a>.</li>
<li>Added <code>MaxResults</code> and <code>PageToken</code> fields for
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsRequest ">catalog.ListCatalogsRequest</a>.</li>
<li>Added <code>NextPageToken</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsResponse ">catalog.ListCatalogsResponse</a>.</li>
<li>Added <code>TableServingUrl</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#OnlineTable ">catalog.OnlineTable</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#StorageCredentialInfo ">catalog.StorageCredentialInfo</a>.</li>
<li>Changed <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateCatalog ">catalog.UpdateCatalog</a>
to <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode ">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateExternalLocation ">catalog.UpdateExternalLocation</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateStorageCredential ">catalog.UpdateStorageCredential</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode ">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>CreateSchedule</code>, <code>CreateSubscription</code>,
<code>DeleteSchedule</code>, <code>DeleteSubscription</code>,
<code>GetSchedule</code>, <code>GetSubscription</code>,
<code>List</code>, <code>ListSchedules</code>,
<code>ListSubscriptions</code> and <code>UpdateSchedule</code> methods
for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#LakeviewAPI ">w.Lakeview</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateScheduleRequest ">dashboards.CreateScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateSubscriptionRequest ">dashboards.CreateSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CronSchedule ">dashboards.CronSchedule</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DashboardView ">dashboards.DashboardView</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteScheduleRequest ">dashboards.DeleteScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteSubscriptionRequest ">dashboards.DeleteSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetScheduleRequest ">dashboards.GetScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetSubscriptionRequest ">dashboards.GetSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsRequest ">dashboards.ListDashboardsRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsResponse ">dashboards.ListDashboardsResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesRequest ">dashboards.ListSchedulesRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesResponse ">dashboards.ListSchedulesResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsRequest ">dashboards.ListSubscriptionsRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsResponse ">dashboards.ListSubscriptionsResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Schedule ">dashboards.Schedule</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SchedulePauseStatus ">dashboards.SchedulePauseStatus</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscriber ">dashboards.Subscriber</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscription ">dashboards.Subscription</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberDestination ">dashboards.SubscriptionSubscriberDestination</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberUser ">dashboards.SubscriptionSubscriberUser</a>
and <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#UpdateScheduleRequest ">dashboards.UpdateScheduleRequest</a>
structs.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobEmailNotifications ">jobs.JobEmailNotifications</a>.</li>
<li>Added <code>EnvironmentKey</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunTask ">jobs.RunTask</a>.</li>
<li>Removed <code>ConditionTask</code>, <code>DbtTask</code>,
<code>NotebookTask</code>, <code>PipelineTask</code>,
<code>PythonWheelTask</code>, <code>RunJobTask</code>,
<code>SparkJarTask</code>, <code>SparkPythonTask</code>,
<code>SparkSubmitTask</code>, <code>SqlTask</code> and
<code>Environments</code> fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun ">jobs.SubmitRun</a>.</li>
<li>Added <code>DbtTask</code> and <code>EnvironmentKey</code> field for
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask ">jobs.SubmitTask</a>.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TaskEmailNotifications ">jobs.TaskEmailNotifications</a>.</li>
<li>Added <code>Periodic</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TriggerSettings ">jobs.TriggerSettings</a>.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#WebhookNotifications ">jobs.WebhookNotifications</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfiguration ">jobs.PeriodicTriggerConfiguration</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfigurationTimeUnit ">jobs.PeriodicTriggerConfigurationTimeUnit</a>.</li>
<li>Added <code>ProviderSummary</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#Listing ">marketplace.Listing</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconFile ">marketplace.ProviderIconFile</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconType ">marketplace.ProviderIconType</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderListingSummaryInfo ">marketplace.ProviderListingSummaryInfo</a>.</li>
<li>Added <code>Start</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#AppsAPI ">w.Apps</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#ServingEndpointsDataPlaneAPI ">w.ServingEndpointsDataPlane</a>
workspace-level service.</li>
<li>Added <code>ServicePrincipalId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App ">serving.App</a>.</li>
<li>Added <code>ServicePrincipalName</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App ">serving.App</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#StartAppRequest ">serving.StartAppRequest</a>.</li>
<li>Added <code>QueryNextPage</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#VectorSearchIndexesAPI ">w.VectorSearchIndexes</a>
workspace-level service.</li>
<li>Added <code>QueryType</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexRequest ">vectorsearch.QueryVectorIndexRequest</a>.</li>
<li>Added <code>NextPageToken</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexResponse ">vectorsearch.QueryVectorIndexResponse</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexNextPageRequest ">vectorsearch.QueryVectorIndexNextPageRequest</a>.</li>
</ul>
<p>OpenAPI SHA: 7437dabb9dadee402c1fc060df4c1ce8cc5369f0, Date:
2024-06-25</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="3e419132eahttps://redirect.github.com/databricks/databricks-sdk-go/issues/955 ">#955</a>)</li>
<li><a
href="ce3dc984f7https://redirect.github.com/databricks/databricks-sdk-go/issues/952 ">#952</a>)</li>
<li><a
href="00b1d09b24https://redirect.github.com/databricks/databricks-sdk-go/issues/947 ">#947</a>)</li>
<li><a
href="d098b1a3e7https://redirect.github.com/databricks/databricks-sdk-go/issues/925 ">#925</a>)</li>
<li><a
href="490bc13c0ehttps://redirect.github.com/databricks/databricks-sdk-go/issues/944 ">#944</a>)</li>
<li>See full diff in <a
href="https://github.com/databricks/databricks-sdk-go/compare/v0.42.0...v0.43.0 ">compare
view</a></li>
</ul>
</details>
<br />
<details>
<summary>Most Recent Ignore Conditions Applied to This Pull
Request</summary>
| Dependency Name | Ignore Conditions |
| --- | --- |
| github.com/databricks/databricks-sdk-go | [>= 0.28.a, < 0.29] |
</details>
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-06-25 12:51:17 +00:00
100a0516d4
Add context type and value to path rewriting ( #1525 )
...
## Changes
For a future change where the inner rewriting functions need access to
the underlying bundle, this change makes preparations.
All values were passed via the stack before and adding yet another value
would make the code less readable.
## Tests
Unit tests pass.
2024-06-25 10:04:22 +00:00
2ec6abf74e
Fix `databricks configure` to use DATABRICKS_CONFIG_FILE environment variable if exists as config file ( #1325 )
...
## Changes
added `ConfigFile: cfg.ConfigFile` for `databrickscfg.SaveToProfile` in
`cmd/configure/configure.go` to save the file in a specified path when
the value is not empty
## Tests
`TestConfigFileFromEnvNoInteractive` in
`cmd/configure/configure_test.go` sets a different config file path by
`DATABRICKS_CONFIG_FILE`, after execution, the overwrite config file is
generated, and the default path has no file.
2024-06-24 10:56:49 +00:00
8957f1e7cf
Return `fs.ModeDir` for Git folders in the workspace ( #1521 )
...
## Changes
Not doing this meant file system traversal ended upon reaching a Git
folder. By marking these objects as a directory globbing traverses into
these folders as well.
## Tests
Added a unit test for coverage.
2024-06-24 10:15:13 +00:00
5ff06578ac
PythonMutator: replace stdin/stdout with files ( #1512 )
...
## Changes
Replace stdin/stdout with files in `PythonMutator`. Files are created in
a temporary directory.
Rename `ApplyPythonMutator` to `PythonMutator`.
Add test for `dyn.Location` behavior during the "load" stage.
## Tests
Unit tests
2024-06-24 07:47:41 +00:00
068c7cfc2d
Return `dyn.InvalidValue` instead of `dyn.NilValue` when errors happen ( #1514 )
...
## Changes
With https://github.com/databricks/cli/pull/1507 and
https://github.com/databricks/cli/pull/1511 we are clarifying the
semantics associated with `dyn.InvalidValue` and `dyn.NilValue`. An
invalid value is the default zero value and is used to signals the
complete absence of the value.
A nil value, on the other hand, is a valid value for a piece of
configuration and signals explicitly setting a key to nil in the
configuration tree. In keeping with that theme, this PR returns
`dyn.InvalidValue` instead of `dyn.NilValue` at error sites. This change
is not expected to have a material change in behaviour and is being done
to set the right convention since we have well-defined semantics
associated with both `NilValue` and `InvalidValue`.
## Tests
Unit tests and integration tests pass. Also manually scanned the changes
and the associated call sites to verify the `NilValue` value itself was
not being relied upon.
2024-06-21 14:22:42 +00:00
446a9d0c52
Properly deal with nil values in `convert.FromTyped` ( #1511 )
...
## Changes
When a configuration defines:
```yaml
run_as:
```
It first showed up as `run_as -> nil` in the dynamic configuration only
to later be converted to `run_as -> {}` while going through typed
conversion. We were using the presence of a key to initialize an empty
value. This is incorrect and it should have remained a nil value.
This conversion was happening in `convert.FromTyped` where any struct
always returned a map value. Instead, it should only return a map value
in any one of these cases: 1) the struct has elements, 2) the struct was
originally a map in the dynamic configuration, or 3) the struct was
initialized to a non-empty pointer value.
Stacked on top of #1516 and #1518 .
## Tests
* Unit tests pass.
* Integration tests pass.
* Manually ran through bundle CRUD with a bundle without resources.
2024-06-21 13:43:21 +00:00
87bc583819
Allow the any type to be set to nil in `convert.FromTyped` ( #1518 )
...
## Changes
This came up in integration testing for #1511 . One of the tests
converted a `map[string]any` to a dynamic value and encountered a `nil`
and errored out. We can safely return a nil in this case.
## Tests
Unit test passes.
2024-06-21 11:19:48 +00:00
01adef666a
Set bool pointer to disable lock ( #1516 )
...
## Changes
This cherry-picks from #1490 to address an issue that came up in #1511 .
The function `dyn.SetByPath` requires intermediate values to be present.
If they are not, it returns an error that it cannot index a map. This is
not an issue on main, where the intermediate maps are always created,
even if they are not present in the dynamic configuration tree. As of
#1511 , we'll no longer populate empty maps for empty structs if they are
not explicitly set (i.e., a non-nil pointer). This change writes a bool
pointer to avoid this issue altogether.
## Tests
Unit tests pass.
2024-06-21 11:14:33 +00:00
57a5a65f87
Add ApplyPythonMutator ( #1430 )
...
## Changes
Add ApplyPythonMutator, which will fork the Python subprocess and
process pipe bundle configuration through it.
It's enabled through `experimental` section, for example:
```yaml
experimental:
pydabs:
enable: true
venv_path: .venv
```
For now, it's limited to two phases in the mutator pipeline:
- `load`: adds new jobs
- `init`: adds new jobs, or modifies existing ones
It's enforced that no jobs are modified in `load` and not jobs are
deleted in `load/init`, because, otherwise, it will break existing
assumptions.
## Tests
Unit tests
2024-06-20 08:43:08 +00:00
b2c03ea54c
Use `dyn.InvalidValue` to indicate absence ( #1507 )
...
## Changes
Previously, the functions `Get` and `Index` returned `dyn.NilValue` to
indicate that a map key or sequence index wasn't found. This is a valid
value, so we need to differentiate between actual absence and a real
`dyn.NilValue`. We do this with the zero value of a `dyn.Value` (also
captured in the constant `dyn.InvalidValue`).
## Tests
* Unit tests.
* Renamed `Get` and `Index` to find and update all call sites.
2024-06-19 15:24:57 +00:00
deb3e365cd
Pause quality monitors when "mode: development" is used ( #1481 )
...
## Changes
Similar to scheduled jobs, quality monitors should be paused when in
development mode (in line with the [behavior for scheduled
jobs](https://docs.databricks.com/en/dev-tools/bundles/deployment-modes.html )).
@aravind-segu @arpitjasa-db please take a look and verify this behavior.
- [x] Followup: documentation changes. If we make this change we should
update
https://docs.databricks.com/dev-tools/bundles/deployment-modes.html .
## Tests
Unit tests
2024-06-19 13:54:35 +00:00
cb4ab5007d
Add link to documentation for Homebrew installation to README ( #1505 )
...
Co-authored-by: Julia Crawford (Databricks) <julia.crawford@databricks.com>
2024-06-19 11:01:04 +00:00
663aa9ab8c
Override variables with lookup value even if values has default value set ( #1504 )
...
## Changes
This PR fixes the behaviour when variables were not overridden with
lookup value from targets if these variables had any default value set
in the default target.
Fixes #1449
## Tests
Added regression test
2024-06-19 08:03:06 +00:00
553fdd1e81
Serialize dynamic value for `bundle validate` output ( #1499 )
...
## Changes
Using dynamic values allows us to retain references like
`${resources.jobs...}` even when the type of field is not integer, eg:
`run_job_task`, or in general values that do not map to the Go types for
a field.
## Tests
Integration test
2024-06-18 15:04:20 +00:00
274688d8a2
Clean up unused code ( #1502 )
...
## Changes
1. Removes `DefaultMutatorsForTarget` which is no longer used anywhere
2. Makes SnapshotPath a private field. It's no longer needed by data
structures outside its package.
FYI, I also tried finding other instances of dead code but I could not
find anything else that was safe to remove. I used
https://go.dev/blog/deadcode to search for them, and the other instances
either implemented an interface, increased test coverage for some of our
other code paths or there was some other reason I could not remove them
(like autogenerated functions or used in tests).
Good sign our codebase is mostly clean (at least superficially).
2024-06-18 14:14:27 +00:00
533d357a71
Fix typo in DBT template ( #1498 )
...
## Changes
Found in https://github.com/databricks/bundle-examples/pull/26 .
## Tests
n/a
2024-06-17 15:56:49 +00:00
8d658589ed
Bump github.com/spf13/cobra from 1.8.0 to 1.8.1 ( #1496 )
...
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra ) from
1.8.0 to 1.8.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/spf13/cobra/releases ">github.com/spf13/cobra's
releases</a>.</em></p>
<blockquote>
<h2>v1.8.1</h2>
<h2>✨ Features</h2>
<ul>
<li>Add env variable to suppress completion descriptions on create by <a
href="https://github.com/scop "><code>@scop</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/1938 ">spf13/cobra#1938</a></li>
</ul>
<h2>🐛 Bug fixes</h2>
<ul>
<li>Micro-optimizations by <a
href="https://github.com/scop "><code>@scop</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/1957 ">spf13/cobra#1957</a></li>
</ul>
<h2>🔧 Maintenance</h2>
<ul>
<li>build(deps): bump github.com/cpuguy83/go-md2man/v2 from 2.0.3 to
2.0.4 by <a
href="https://github.com/dependabot "><code>@dependabot</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2127 ">spf13/cobra#2127</a></li>
<li>Consistent annotation names by <a
href="https://github.com/nirs "><code>@nirs</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2140 ">spf13/cobra#2140</a></li>
<li>Remove fully inactivated linters by <a
href="https://github.com/nirs "><code>@nirs</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2148 ">spf13/cobra#2148</a></li>
<li>Address golangci-lint deprecation warnings, enable some more linters
by <a href="https://github.com/scop "><code>@scop</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2152 ">spf13/cobra#2152</a></li>
</ul>
<h2>🧪 Testing & CI/CD</h2>
<ul>
<li>Add test for func in cobra.go by <a
href="https://github.com/korovindenis "><code>@korovindenis</code></a>
in <a
href="https://redirect.github.com/spf13/cobra/pull/2094 ">spf13/cobra#2094</a></li>
<li>ci: test golang 1.22 by <a
href="https://github.com/cyrilico "><code>@cyrilico</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2113 ">spf13/cobra#2113</a></li>
<li>Optimized and added more linting by <a
href="https://github.com/scop "><code>@scop</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2099 ">spf13/cobra#2099</a></li>
<li>build(deps): bump actions/setup-go from 4 to 5 by <a
href="https://github.com/dependabot "><code>@dependabot</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2087 ">spf13/cobra#2087</a></li>
<li>build(deps): bump actions/labeler from 4 to 5 by <a
href="https://github.com/dependabot "><code>@dependabot</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2086 ">spf13/cobra#2086</a></li>
<li>build(deps): bump golangci/golangci-lint-action from 3.7.0 to 4.0.0
by <a href="https://github.com/dependabot "><code>@dependabot</code></a>
in <a
href="https://redirect.github.com/spf13/cobra/pull/2108 ">spf13/cobra#2108</a></li>
<li>build(deps): bump actions/cache from 3 to 4 by <a
href="https://github.com/dependabot "><code>@dependabot</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2102 ">spf13/cobra#2102</a></li>
</ul>
<h2>✏️ Documentation</h2>
<ul>
<li>Fixes and docs for usage as plugin by <a
href="https://github.com/nirs "><code>@nirs</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2070 ">spf13/cobra#2070</a></li>
<li>flags: clarify documentation that LocalFlags related function do not
modify the state by <a
href="https://github.com/niamster "><code>@niamster</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2064 ">spf13/cobra#2064</a></li>
<li>chore: remove repetitive words by <a
href="https://github.com/racerole "><code>@racerole</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2122 ">spf13/cobra#2122</a></li>
<li>Add LXC to the list of projects using Cobra <a
href="https://github.com/VaradBelwalkar "><code>@VaradBelwalkar</code></a>
in <a
href="https://redirect.github.com/spf13/cobra/pull/2071 ">spf13/cobra#2071</a></li>
<li>Update projects_using_cobra.md by <a
href="https://github.com/marcuskohlberg "><code>@marcuskohlberg</code></a>
in <a
href="https://redirect.github.com/spf13/cobra/pull/2089 ">spf13/cobra#2089</a></li>
<li>[chore]: update projects using cobra by <a
href="https://github.com/cmwylie19 "><code>@cmwylie19</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2093 ">spf13/cobra#2093</a></li>
<li>Add Taikun CLI to list of projects by <a
href="https://github.com/Smidra "><code>@Smidra</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2098 ">spf13/cobra#2098</a></li>
<li>Add Incus to the list of projects using Cobra by <a
href="https://github.com/montag451 "><code>@montag451</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2118 ">spf13/cobra#2118</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="e94f6d0dd9https://redirect.github.com/spf13/cobra/issues/2152 ">#2152</a>)</li>
<li><a
href="8003b74a10https://redirect.github.com/spf13/cobra/issues/2148 ">#2148</a>)</li>
<li><a
href="5c2c1d627dhttps://redirect.github.com/spf13/cobra/issues/2140 ">#2140</a>)</li>
<li><a
href="5a1acea321https://redirect.github.com/spf13/cobra/issues/2127 ">#2127</a>)</li>
<li><a
href="0fc86c2ffdhttps://redirect.github.com/spf13/cobra/issues/2128 ">#2128</a>)</li>
<li><a
href="6b5f577ebchttps://redirect.github.com/spf13/cobra/issues/2099 ">#2099</a>)</li>
<li><a
href="bd914e58d6https://redirect.github.com/spf13/cobra/issues/2120 ">#2120</a>)</li>
<li><a
href="1f80fa2e23https://redirect.github.com/spf13/cobra/issues/2122 ">#2122</a>)</li>
<li><a
href="c69ae4c36bhttps://redirect.github.com/spf13/cobra/issues/2113 ">#2113</a>)</li>
<li><a
href="a30cee5e5ahttps://redirect.github.com/spf13/cobra/issues/2102 ">#2102</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/spf13/cobra/compare/v1.8.0...v1.8.1 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-17 12:29:50 +00:00
4904dfb047
Bump github.com/briandowns/spinner from 1.23.0 to 1.23.1 ( #1495 )
...
Bumps
[github.com/briandowns/spinner](https://github.com/briandowns/spinner )
from 1.23.0 to 1.23.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/briandowns/spinner/releases ">github.com/briandowns/spinner's
releases</a>.</em></p>
<blockquote>
<h2>v1.23.1</h2>
<h2>What's Changed</h2>
<ul>
<li>fix: CVE-2022-29526 by <a
href="https://github.com/testwill "><code>@testwill</code></a> in <a
href="https://redirect.github.com/briandowns/spinner/pull/152 ">briandowns/spinner#152</a></li>
<li>Consolidate the dependencies for the IsTerminal() API by <a
href="https://github.com/debarshiray "><code>@debarshiray</code></a> in
<a
href="https://redirect.github.com/briandowns/spinner/pull/156 ">briandowns/spinner#156</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/testwill "><code>@testwill</code></a>
made their first contribution in <a
href="https://redirect.github.com/briandowns/spinner/pull/152 ">briandowns/spinner#152</a></li>
<li><a
href="https://github.com/debarshiray "><code>@debarshiray</code></a>
made their first contribution in <a
href="https://redirect.github.com/briandowns/spinner/pull/156 ">briandowns/spinner#156</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/briandowns/spinner/compare/v1.23.0...v1.23.1 ">https://github.com/briandowns/spinner/compare/v1.23.0...v1.23.1 </a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="8f269dd04fhttps://redirect.github.com/briandowns/spinner/issues/156 ">#156</a>)</li>
<li><a
href="12e6c297dbhttps://redirect.github.com/briandowns/spinner/issues/152 ">#152</a>)</li>
<li>See full diff in <a
href="https://github.com/briandowns/spinner/compare/v1.23.0...v1.23.1 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-17 12:18:08 +00:00
ac6b80ed88
Remove user credentials specified in the Git origin URL ( #1494 )
...
## Changes
We set the origin URL as metadata in any jobs created by DABs. This PR
makes sure user credentials do not leak into the set metadata in the
job.
## Tests
Unit test
---------
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-06-17 09:49:00 +00:00
Anders Rex
Gleb Kanterov
Pieter Noordhuis
Andrew Nester
kijewskimateusz
shreyas-goenka
dependabot[bot]
Kai Zhu
Lennart Kats (databricks)