Commit Graph

1241 Commits

Author SHA1 Message Date
Andrew Nester 5f42791609
Added support for complex variables (#1467)
## Changes
Added support for complex variables

Now it's possible to add and use complex variables as shown below

```
bundle:
  name: complex-variables

resources:
  jobs:
    my_job:
      job_clusters:
        - job_cluster_key: key
          new_cluster: ${var.cluster}
      tasks:
      - task_key: test
        job_cluster_key: key

variables:
  cluster:
    description: "A cluster definition"
    type: complex
    default:
      spark_version: "13.2.x-scala2.11"
      node_type_id: "Standard_DS3_v2"
      num_workers: 2
      spark_conf:
        spark.speculation: true
        spark.databricks.delta.retentionDurationCheck.enabled: false
```

Fixes #1298

- [x] Support for complex variables
- [x] Allow variable overrides (with shortcut) in targets
- [x] Don't allow to provide complex variables via flag or env variable
- [x] Fail validation if complex value is used but not `type: complex`
provided
- [x] Support using variables inside complex variables 

## Tests
Added unit tests

---------

Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com>
2024-06-26 10:25:32 +00:00
Pieter Noordhuis ce5a3f2ce6
Upgrade TF provider to 1.48.0 (#1527)
## Changes

This includes a fix for library order not being respected.

## Tests

Manually confirmed the fix works in
https://github.com/databricks/bundle-examples/pull/29.
2024-06-26 09:29:46 +00:00
Pieter Noordhuis 482d83cba8
Revert "Retain location metadata for values in `convert.FromTyped`" (#1528)
## Changes

This reverts commit dac5f09556 (#1523).

Retaining the location for nil values means equality checks no longer
pass.

We need #1520 to be merged first.

## Tests

Integration test `TestAccPythonWheelTaskDeployAndRunWithWrapper`.
2024-06-26 09:26:40 +00:00
shreyas-goenka dac5f09556
Retain location metadata for values in `convert.FromTyped` (#1523)
## Changes

There are four different treatments location metadata can receive in the
`convert.FromTyped` method.

1. Location metadata is **retained** for maps, structs and slices if the
value is **not nil**
2. Location metadata is **lost** for maps, structs and slices if the
value is **is nil**
3. Location metadata is **retained** if a scalar type (eg. bool, string
etc) does not change.
4. Location metadata is **lost** if the value for a scalar type changes.

This PR ensures that location metadata is not lost in any case; that is,
it's always preserved.

For (2), this serves as a bug fix so that location information is not
lost on conversion to and from typed for nil values of complex types
(struct, slices, and maps).

For (4) this is a change in semantics. For primitive values modified in
a `typed` mutator, any references to `.Location()` for computed
primitive fields will now return associated YAML location metadata (if
any) instead of an empty location.

While arguable, these semantics are OK since:
1. Situations like these will be rare.
2. Knowing the YAML location (if any) is better than not knowing the
location at all. These locations are typically visible to the user in
errors and warnings.

## Tests

Unit tests
2024-06-25 13:40:21 +00:00
dependabot[bot] 8468878eed
Bump github.com/databricks/databricks-sdk-go from 0.42.0 to 0.43.0 (#1522)
Bumps
[github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go)
from 0.42.0 to 0.43.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/releases">github.com/databricks/databricks-sdk-go's
releases</a>.</em></p>
<blockquote>
<h2>v0.43.0</h2>
<p>Major Changes and Improvements:</p>
<ul>
<li>Support partners in user agent for SDK (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/925">#925</a>).</li>
<li>Add <code>serverless_compute_id</code> field to the config (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/952">#952</a>).</li>
</ul>
<p>Other Changes:</p>
<ul>
<li>Generate from latest spec (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/944">#944</a>)
and (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/947">#947</a>).</li>
</ul>
<p>API Changes:</p>
<ul>
<li>Changed <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogInfo">catalog.CatalogInfo</a>
to <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ExternalLocationInfo">catalog.ExternalLocationInfo</a>.</li>
<li>Added <code>MaxResults</code> and <code>PageToken</code> fields for
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsRequest">catalog.ListCatalogsRequest</a>.</li>
<li>Added <code>NextPageToken</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsResponse">catalog.ListCatalogsResponse</a>.</li>
<li>Added <code>TableServingUrl</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#OnlineTable">catalog.OnlineTable</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#StorageCredentialInfo">catalog.StorageCredentialInfo</a>.</li>
<li>Changed <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateCatalog">catalog.UpdateCatalog</a>
to <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateExternalLocation">catalog.UpdateExternalLocation</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateStorageCredential">catalog.UpdateStorageCredential</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>CreateSchedule</code>, <code>CreateSubscription</code>,
<code>DeleteSchedule</code>, <code>DeleteSubscription</code>,
<code>GetSchedule</code>, <code>GetSubscription</code>,
<code>List</code>, <code>ListSchedules</code>,
<code>ListSubscriptions</code> and <code>UpdateSchedule</code> methods
for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#LakeviewAPI">w.Lakeview</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateScheduleRequest">dashboards.CreateScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateSubscriptionRequest">dashboards.CreateSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CronSchedule">dashboards.CronSchedule</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DashboardView">dashboards.DashboardView</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteScheduleRequest">dashboards.DeleteScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteSubscriptionRequest">dashboards.DeleteSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetScheduleRequest">dashboards.GetScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetSubscriptionRequest">dashboards.GetSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsRequest">dashboards.ListDashboardsRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsResponse">dashboards.ListDashboardsResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesRequest">dashboards.ListSchedulesRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesResponse">dashboards.ListSchedulesResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsRequest">dashboards.ListSubscriptionsRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsResponse">dashboards.ListSubscriptionsResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Schedule">dashboards.Schedule</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SchedulePauseStatus">dashboards.SchedulePauseStatus</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscriber">dashboards.Subscriber</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscription">dashboards.Subscription</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberDestination">dashboards.SubscriptionSubscriberDestination</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberUser">dashboards.SubscriptionSubscriberUser</a>
and <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#UpdateScheduleRequest">dashboards.UpdateScheduleRequest</a>
structs.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobEmailNotifications">jobs.JobEmailNotifications</a>.</li>
<li>Added <code>EnvironmentKey</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunTask">jobs.RunTask</a>.</li>
<li>Removed <code>ConditionTask</code>, <code>DbtTask</code>,
<code>NotebookTask</code>, <code>PipelineTask</code>,
<code>PythonWheelTask</code>, <code>RunJobTask</code>,
<code>SparkJarTask</code>, <code>SparkPythonTask</code>,
<code>SparkSubmitTask</code>, <code>SqlTask</code> and
<code>Environments</code> fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun">jobs.SubmitRun</a>.</li>
<li>Added <code>DbtTask</code> and <code>EnvironmentKey</code> field for
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask">jobs.SubmitTask</a>.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TaskEmailNotifications">jobs.TaskEmailNotifications</a>.</li>
<li>Added <code>Periodic</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TriggerSettings">jobs.TriggerSettings</a>.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#WebhookNotifications">jobs.WebhookNotifications</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfiguration">jobs.PeriodicTriggerConfiguration</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfigurationTimeUnit">jobs.PeriodicTriggerConfigurationTimeUnit</a>.</li>
<li>Added <code>ProviderSummary</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#Listing">marketplace.Listing</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconFile">marketplace.ProviderIconFile</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconType">marketplace.ProviderIconType</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderListingSummaryInfo">marketplace.ProviderListingSummaryInfo</a>.</li>
<li>Added <code>Start</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#AppsAPI">w.Apps</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#ServingEndpointsDataPlaneAPI">w.ServingEndpointsDataPlane</a>
workspace-level service.</li>
<li>Added <code>ServicePrincipalId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App">serving.App</a>.</li>
<li>Added <code>ServicePrincipalName</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App">serving.App</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#StartAppRequest">serving.StartAppRequest</a>.</li>
<li>Added <code>QueryNextPage</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#VectorSearchIndexesAPI">w.VectorSearchIndexes</a>
workspace-level service.</li>
<li>Added <code>QueryType</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexRequest">vectorsearch.QueryVectorIndexRequest</a>.</li>
<li>Added <code>NextPageToken</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexResponse">vectorsearch.QueryVectorIndexResponse</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexNextPageRequest">vectorsearch.QueryVectorIndexNextPageRequest</a>.</li>
</ul>
<p>OpenAPI SHA: 7437dabb9dadee402c1fc060df4c1ce8cc5369f0, Date:
2024-06-25</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/blob/main/CHANGELOG.md">github.com/databricks/databricks-sdk-go's
changelog</a>.</em></p>
<blockquote>
<h2>0.43.0</h2>
<p>Major Changes and Improvements:</p>
<ul>
<li>Support partners in user agent for SDK (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/925">#925</a>).</li>
<li>Add <code>serverless_compute_id</code> field to the config (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/952">#952</a>).</li>
</ul>
<p>Other Changes:</p>
<ul>
<li>Generate from latest spec (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/944">#944</a>)
and (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/947">#947</a>).</li>
</ul>
<p>API Changes:</p>
<ul>
<li>Changed <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogInfo">catalog.CatalogInfo</a>
to <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ExternalLocationInfo">catalog.ExternalLocationInfo</a>.</li>
<li>Added <code>MaxResults</code> and <code>PageToken</code> fields for
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsRequest">catalog.ListCatalogsRequest</a>.</li>
<li>Added <code>NextPageToken</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsResponse">catalog.ListCatalogsResponse</a>.</li>
<li>Added <code>TableServingUrl</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#OnlineTable">catalog.OnlineTable</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#StorageCredentialInfo">catalog.StorageCredentialInfo</a>.</li>
<li>Changed <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateCatalog">catalog.UpdateCatalog</a>
to <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateExternalLocation">catalog.UpdateExternalLocation</a>.</li>
<li>Added <code>IsolationMode</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateStorageCredential">catalog.UpdateStorageCredential</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li>
<li>Added <code>CreateSchedule</code>, <code>CreateSubscription</code>,
<code>DeleteSchedule</code>, <code>DeleteSubscription</code>,
<code>GetSchedule</code>, <code>GetSubscription</code>,
<code>List</code>, <code>ListSchedules</code>,
<code>ListSubscriptions</code> and <code>UpdateSchedule</code> methods
for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#LakeviewAPI">w.Lakeview</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateScheduleRequest">dashboards.CreateScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateSubscriptionRequest">dashboards.CreateSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CronSchedule">dashboards.CronSchedule</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DashboardView">dashboards.DashboardView</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteScheduleRequest">dashboards.DeleteScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteSubscriptionRequest">dashboards.DeleteSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetScheduleRequest">dashboards.GetScheduleRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetSubscriptionRequest">dashboards.GetSubscriptionRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsRequest">dashboards.ListDashboardsRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsResponse">dashboards.ListDashboardsResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesRequest">dashboards.ListSchedulesRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesResponse">dashboards.ListSchedulesResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsRequest">dashboards.ListSubscriptionsRequest</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsResponse">dashboards.ListSubscriptionsResponse</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Schedule">dashboards.Schedule</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SchedulePauseStatus">dashboards.SchedulePauseStatus</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscriber">dashboards.Subscriber</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscription">dashboards.Subscription</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberDestination">dashboards.SubscriptionSubscriberDestination</a>,
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberUser">dashboards.SubscriptionSubscriberUser</a>
and <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#UpdateScheduleRequest">dashboards.UpdateScheduleRequest</a>
structs.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobEmailNotifications">jobs.JobEmailNotifications</a>.</li>
<li>Added <code>EnvironmentKey</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunTask">jobs.RunTask</a>.</li>
<li>Removed <code>ConditionTask</code>, <code>DbtTask</code>,
<code>NotebookTask</code>, <code>PipelineTask</code>,
<code>PythonWheelTask</code>, <code>RunJobTask</code>,
<code>SparkJarTask</code>, <code>SparkPythonTask</code>,
<code>SparkSubmitTask</code>, <code>SqlTask</code> and
<code>Environments</code> fields for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun">jobs.SubmitRun</a>.</li>
<li>Added <code>DbtTask</code> and <code>EnvironmentKey</code> field for
<a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask">jobs.SubmitTask</a>.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TaskEmailNotifications">jobs.TaskEmailNotifications</a>.</li>
<li>Added <code>Periodic</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TriggerSettings">jobs.TriggerSettings</a>.</li>
<li>Added <code>OnStreamingBacklogExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#WebhookNotifications">jobs.WebhookNotifications</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfiguration">jobs.PeriodicTriggerConfiguration</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfigurationTimeUnit">jobs.PeriodicTriggerConfigurationTimeUnit</a>.</li>
<li>Added <code>ProviderSummary</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#Listing">marketplace.Listing</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconFile">marketplace.ProviderIconFile</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconType">marketplace.ProviderIconType</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderListingSummaryInfo">marketplace.ProviderListingSummaryInfo</a>.</li>
<li>Added <code>Start</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#AppsAPI">w.Apps</a>
workspace-level service.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#ServingEndpointsDataPlaneAPI">w.ServingEndpointsDataPlane</a>
workspace-level service.</li>
<li>Added <code>ServicePrincipalId</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App">serving.App</a>.</li>
<li>Added <code>ServicePrincipalName</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App">serving.App</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#StartAppRequest">serving.StartAppRequest</a>.</li>
<li>Added <code>QueryNextPage</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#VectorSearchIndexesAPI">w.VectorSearchIndexes</a>
workspace-level service.</li>
<li>Added <code>QueryType</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexRequest">vectorsearch.QueryVectorIndexRequest</a>.</li>
<li>Added <code>NextPageToken</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexResponse">vectorsearch.QueryVectorIndexResponse</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexNextPageRequest">vectorsearch.QueryVectorIndexNextPageRequest</a>.</li>
</ul>
<p>OpenAPI SHA: 7437dabb9dadee402c1fc060df4c1ce8cc5369f0, Date:
2024-06-25</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="3e419132ea"><code>3e41913</code></a>
Release v0.43.0 (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/issues/955">#955</a>)</li>
<li><a
href="ce3dc984f7"><code>ce3dc98</code></a>
Add <code>serverless_compute_id</code> field to the config (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/issues/952">#952</a>)</li>
<li><a
href="00b1d09b24"><code>00b1d09</code></a>
Update OpenAPI spec (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/issues/947">#947</a>)</li>
<li><a
href="d098b1a3e7"><code>d098b1a</code></a>
Support partners in SDK (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/issues/925">#925</a>)</li>
<li><a
href="490bc13c0e"><code>490bc13</code></a>
Generate from latest spec (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/issues/944">#944</a>)</li>
<li>See full diff in <a
href="https://github.com/databricks/databricks-sdk-go/compare/v0.42.0...v0.43.0">compare
view</a></li>
</ul>
</details>
<br />

<details>
<summary>Most Recent Ignore Conditions Applied to This Pull
Request</summary>

| Dependency Name | Ignore Conditions |
| --- | --- |
| github.com/databricks/databricks-sdk-go | [>= 0.28.a, < 0.29] |
</details>


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/databricks/databricks-sdk-go&package-manager=go_modules&previous-version=0.42.0&new-version=0.43.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-06-25 12:51:17 +00:00
Pieter Noordhuis 100a0516d4
Add context type and value to path rewriting (#1525)
## Changes

For a future change where the inner rewriting functions need access to
the underlying bundle, this change makes preparations.

All values were passed via the stack before and adding yet another value
would make the code less readable.

## Tests

Unit tests pass.
2024-06-25 10:04:22 +00:00
Kai Zhu 2ec6abf74e
Fix `databricks configure` to use DATABRICKS_CONFIG_FILE environment variable if exists as config file (#1325)
## Changes
added `ConfigFile: cfg.ConfigFile` for `databrickscfg.SaveToProfile` in
`cmd/configure/configure.go` to save the file in a specified path when
the value is not empty

## Tests
`TestConfigFileFromEnvNoInteractive` in
`cmd/configure/configure_test.go` sets a different config file path by
`DATABRICKS_CONFIG_FILE`, after execution, the overwrite config file is
generated, and the default path has no file.
2024-06-24 10:56:49 +00:00
Pieter Noordhuis 8957f1e7cf
Return `fs.ModeDir` for Git folders in the workspace (#1521)
## Changes

Not doing this meant file system traversal ended upon reaching a Git
folder. By marking these objects as a directory globbing traverses into
these folders as well.

## Tests

Added a unit test for coverage.
2024-06-24 10:15:13 +00:00
Gleb Kanterov 5ff06578ac
PythonMutator: replace stdin/stdout with files (#1512)
## Changes
Replace stdin/stdout with files in `PythonMutator`. Files are created in
a temporary directory.

Rename `ApplyPythonMutator` to `PythonMutator`.

Add test for `dyn.Location` behavior during the "load" stage.

## Tests
Unit tests
2024-06-24 07:47:41 +00:00
shreyas-goenka 068c7cfc2d
Return `dyn.InvalidValue` instead of `dyn.NilValue` when errors happen (#1514)
## Changes
With https://github.com/databricks/cli/pull/1507 and
https://github.com/databricks/cli/pull/1511 we are clarifying the
semantics associated with `dyn.InvalidValue` and `dyn.NilValue`. An
invalid value is the default zero value and is used to signals the
complete absence of the value.

A nil value, on the other hand, is a valid value for a piece of
configuration and signals explicitly setting a key to nil in the
configuration tree. In keeping with that theme, this PR returns
`dyn.InvalidValue` instead of `dyn.NilValue` at error sites. This change
is not expected to have a material change in behaviour and is being done
to set the right convention since we have well-defined semantics
associated with both `NilValue` and `InvalidValue`.

## Tests
Unit tests and integration tests pass. Also manually scanned the changes
and the associated call sites to verify the `NilValue` value itself was
not being relied upon.
2024-06-21 14:22:42 +00:00
Pieter Noordhuis 446a9d0c52
Properly deal with nil values in `convert.FromTyped` (#1511)
## Changes

When a configuration defines:
```yaml
run_as:
```

It first showed up as `run_as -> nil` in the dynamic configuration only
to later be converted to `run_as -> {}` while going through typed
conversion. We were using the presence of a key to initialize an empty
value. This is incorrect and it should have remained a nil value.

This conversion was happening in `convert.FromTyped` where any struct
always returned a map value. Instead, it should only return a map value
in any one of these cases: 1) the struct has elements, 2) the struct was
originally a map in the dynamic configuration, or 3) the struct was
initialized to a non-empty pointer value.

Stacked on top of #1516 and #1518.

## Tests

* Unit tests pass.
* Integration tests pass.
* Manually ran through bundle CRUD with a bundle without resources.
2024-06-21 13:43:21 +00:00
Pieter Noordhuis 87bc583819
Allow the any type to be set to nil in `convert.FromTyped` (#1518)
## Changes

This came up in integration testing for #1511. One of the tests
converted a `map[string]any` to a dynamic value and encountered a `nil`
and errored out. We can safely return a nil in this case.

## Tests

Unit test passes.
2024-06-21 11:19:48 +00:00
Pieter Noordhuis 01adef666a
Set bool pointer to disable lock (#1516)
## Changes

This cherry-picks from #1490 to address an issue that came up in #1511.

The function `dyn.SetByPath` requires intermediate values to be present.
If they are not, it returns an error that it cannot index a map. This is
not an issue on main, where the intermediate maps are always created,
even if they are not present in the dynamic configuration tree. As of
#1511, we'll no longer populate empty maps for empty structs if they are
not explicitly set (i.e., a non-nil pointer). This change writes a bool
pointer to avoid this issue altogether.

## Tests

Unit tests pass.
2024-06-21 11:14:33 +00:00
Gleb Kanterov 57a5a65f87
Add ApplyPythonMutator (#1430)
## Changes
Add ApplyPythonMutator, which will fork the Python subprocess and
process pipe bundle configuration through it.

It's enabled through `experimental` section, for example:

```yaml
experimental:
  pydabs: 
    enable: true
    venv_path: .venv
```

For now, it's limited to two phases in the mutator pipeline:

- `load`: adds new jobs
- `init`: adds new jobs, or modifies existing ones

It's enforced that no jobs are modified in `load` and not jobs are
deleted in `load/init`, because, otherwise, it will break existing
assumptions.

## Tests
Unit tests
2024-06-20 08:43:08 +00:00
Pieter Noordhuis b2c03ea54c
Use `dyn.InvalidValue` to indicate absence (#1507)
## Changes

Previously, the functions `Get` and `Index` returned `dyn.NilValue` to
indicate that a map key or sequence index wasn't found. This is a valid
value, so we need to differentiate between actual absence and a real
`dyn.NilValue`. We do this with the zero value of a `dyn.Value` (also
captured in the constant `dyn.InvalidValue`).

## Tests

* Unit tests.
* Renamed `Get` and `Index` to find and update all call sites.
2024-06-19 15:24:57 +00:00
Lennart Kats (databricks) deb3e365cd
Pause quality monitors when "mode: development" is used (#1481)
## Changes

Similar to scheduled jobs, quality monitors should be paused when in
development mode (in line with the [behavior for scheduled
jobs](https://docs.databricks.com/en/dev-tools/bundles/deployment-modes.html)).
@aravind-segu @arpitjasa-db please take a look and verify this behavior.

- [x] Followup: documentation changes. If we make this change we should
update
https://docs.databricks.com/dev-tools/bundles/deployment-modes.html.

## Tests
Unit tests
2024-06-19 13:54:35 +00:00
shreyas-goenka cb4ab5007d
Add link to documentation for Homebrew installation to README (#1505)
Co-authored-by: Julia Crawford (Databricks) <julia.crawford@databricks.com>
2024-06-19 11:01:04 +00:00
Andrew Nester 663aa9ab8c
Override variables with lookup value even if values has default value set (#1504)
## Changes

This PR fixes the behaviour when variables were not overridden with
lookup value from targets if these variables had any default value set
in the default target.

Fixes #1449 

## Tests
Added regression test
2024-06-19 08:03:06 +00:00
shreyas-goenka 553fdd1e81
Serialize dynamic value for `bundle validate` output (#1499)
## Changes
Using dynamic values allows us to retain references like
`${resources.jobs...}` even when the type of field is not integer, eg:
`run_job_task`, or in general values that do not map to the Go types for
a field.

## Tests
Integration test
2024-06-18 15:04:20 +00:00
shreyas-goenka 274688d8a2
Clean up unused code (#1502)
## Changes
1. Removes `DefaultMutatorsForTarget` which is no longer used anywhere
2. Makes SnapshotPath a private field. It's no longer needed by data
structures outside its package.

FYI, I also tried finding other instances of dead code but I could not
find anything else that was safe to remove. I used
https://go.dev/blog/deadcode to search for them, and the other instances
either implemented an interface, increased test coverage for some of our
other code paths or there was some other reason I could not remove them
(like autogenerated functions or used in tests).

Good sign our codebase is mostly clean (at least superficially).
2024-06-18 14:14:27 +00:00
Pieter Noordhuis 533d357a71
Fix typo in DBT template (#1498)
## Changes

Found in https://github.com/databricks/bundle-examples/pull/26.

## Tests

n/a
2024-06-17 15:56:49 +00:00
dependabot[bot] 8d658589ed
Bump github.com/spf13/cobra from 1.8.0 to 1.8.1 (#1496)
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from
1.8.0 to 1.8.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/spf13/cobra/releases">github.com/spf13/cobra's
releases</a>.</em></p>
<blockquote>
<h2>v1.8.1</h2>
<h2> Features</h2>
<ul>
<li>Add env variable to suppress completion descriptions on create by <a
href="https://github.com/scop"><code>@​scop</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/1938">spf13/cobra#1938</a></li>
</ul>
<h2>🐛 Bug fixes</h2>
<ul>
<li>Micro-optimizations by <a
href="https://github.com/scop"><code>@​scop</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/1957">spf13/cobra#1957</a></li>
</ul>
<h2>🔧 Maintenance</h2>
<ul>
<li>build(deps): bump github.com/cpuguy83/go-md2man/v2 from 2.0.3 to
2.0.4 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2127">spf13/cobra#2127</a></li>
<li>Consistent annotation names by <a
href="https://github.com/nirs"><code>@​nirs</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2140">spf13/cobra#2140</a></li>
<li>Remove fully inactivated linters by <a
href="https://github.com/nirs"><code>@​nirs</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2148">spf13/cobra#2148</a></li>
<li>Address golangci-lint deprecation warnings, enable some more linters
by <a href="https://github.com/scop"><code>@​scop</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2152">spf13/cobra#2152</a></li>
</ul>
<h2>🧪 Testing &amp; CI/CD</h2>
<ul>
<li>Add test for func in cobra.go by <a
href="https://github.com/korovindenis"><code>@​korovindenis</code></a>
in <a
href="https://redirect.github.com/spf13/cobra/pull/2094">spf13/cobra#2094</a></li>
<li>ci: test golang 1.22 by <a
href="https://github.com/cyrilico"><code>@​cyrilico</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2113">spf13/cobra#2113</a></li>
<li>Optimized and added more linting by <a
href="https://github.com/scop"><code>@​scop</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2099">spf13/cobra#2099</a></li>
<li>build(deps): bump actions/setup-go from 4 to 5 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2087">spf13/cobra#2087</a></li>
<li>build(deps): bump actions/labeler from 4 to 5 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2086">spf13/cobra#2086</a></li>
<li>build(deps): bump golangci/golangci-lint-action from 3.7.0 to 4.0.0
by <a href="https://github.com/dependabot"><code>@​dependabot</code></a>
in <a
href="https://redirect.github.com/spf13/cobra/pull/2108">spf13/cobra#2108</a></li>
<li>build(deps): bump actions/cache from 3 to 4 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2102">spf13/cobra#2102</a></li>
</ul>
<h2>✏️ Documentation</h2>
<ul>
<li>Fixes and docs for usage as plugin by <a
href="https://github.com/nirs"><code>@​nirs</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2070">spf13/cobra#2070</a></li>
<li>flags: clarify documentation that LocalFlags related function do not
modify the state by <a
href="https://github.com/niamster"><code>@​niamster</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2064">spf13/cobra#2064</a></li>
<li>chore: remove repetitive words by <a
href="https://github.com/racerole"><code>@​racerole</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2122">spf13/cobra#2122</a></li>
<li>Add LXC to the list of projects using Cobra <a
href="https://github.com/VaradBelwalkar"><code>@​VaradBelwalkar</code></a>
in <a
href="https://redirect.github.com/spf13/cobra/pull/2071">spf13/cobra#2071</a></li>
<li>Update projects_using_cobra.md by <a
href="https://github.com/marcuskohlberg"><code>@​marcuskohlberg</code></a>
in <a
href="https://redirect.github.com/spf13/cobra/pull/2089">spf13/cobra#2089</a></li>
<li>[chore]: update projects using cobra by <a
href="https://github.com/cmwylie19"><code>@​cmwylie19</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2093">spf13/cobra#2093</a></li>
<li>Add Taikun CLI to list of projects by <a
href="https://github.com/Smidra"><code>@​Smidra</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2098">spf13/cobra#2098</a></li>
<li>Add Incus to the list of projects using Cobra by <a
href="https://github.com/montag451"><code>@​montag451</code></a> in <a
href="https://redirect.github.com/spf13/cobra/pull/2118">spf13/cobra#2118</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="e94f6d0dd9"><code>e94f6d0</code></a>
Address golangci-lint deprecation warnings, enable some more linters (<a
href="https://redirect.github.com/spf13/cobra/issues/2152">#2152</a>)</li>
<li><a
href="8003b74a10"><code>8003b74</code></a>
Remove fully inactivated linters (<a
href="https://redirect.github.com/spf13/cobra/issues/2148">#2148</a>)</li>
<li><a
href="5c2c1d627d"><code>5c2c1d6</code></a>
Consistent annotation names (<a
href="https://redirect.github.com/spf13/cobra/issues/2140">#2140</a>)</li>
<li><a
href="5a1acea321"><code>5a1acea</code></a>
build(deps): bump github.com/cpuguy83/go-md2man/v2 from 2.0.3 to 2.0.4
(<a
href="https://redirect.github.com/spf13/cobra/issues/2127">#2127</a>)</li>
<li><a
href="0fc86c2ffd"><code>0fc86c2</code></a>
docs: update user guide (<a
href="https://redirect.github.com/spf13/cobra/issues/2128">#2128</a>)</li>
<li><a
href="6b5f577ebc"><code>6b5f577</code></a>
More linting (<a
href="https://redirect.github.com/spf13/cobra/issues/2099">#2099</a>)</li>
<li><a
href="bd914e58d6"><code>bd914e5</code></a>
fix: remove deprecated io/ioutils package (<a
href="https://redirect.github.com/spf13/cobra/issues/2120">#2120</a>)</li>
<li><a
href="1f80fa2e23"><code>1f80fa2</code></a>
chore: remove repetitive words (<a
href="https://redirect.github.com/spf13/cobra/issues/2122">#2122</a>)</li>
<li><a
href="c69ae4c36b"><code>c69ae4c</code></a>
ci: test golang 1.22 (<a
href="https://redirect.github.com/spf13/cobra/issues/2113">#2113</a>)</li>
<li><a
href="a30cee5e5a"><code>a30cee5</code></a>
build(deps): bump actions/cache from 3 to 4 (<a
href="https://redirect.github.com/spf13/cobra/issues/2102">#2102</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/spf13/cobra/compare/v1.8.0...v1.8.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/spf13/cobra&package-manager=go_modules&previous-version=1.8.0&new-version=1.8.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-17 12:29:50 +00:00
dependabot[bot] 4904dfb047
Bump github.com/briandowns/spinner from 1.23.0 to 1.23.1 (#1495)
Bumps
[github.com/briandowns/spinner](https://github.com/briandowns/spinner)
from 1.23.0 to 1.23.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/briandowns/spinner/releases">github.com/briandowns/spinner's
releases</a>.</em></p>
<blockquote>
<h2>v1.23.1</h2>
<h2>What's Changed</h2>
<ul>
<li>fix: CVE-2022-29526 by <a
href="https://github.com/testwill"><code>@​testwill</code></a> in <a
href="https://redirect.github.com/briandowns/spinner/pull/152">briandowns/spinner#152</a></li>
<li>Consolidate the dependencies for the IsTerminal() API by <a
href="https://github.com/debarshiray"><code>@​debarshiray</code></a> in
<a
href="https://redirect.github.com/briandowns/spinner/pull/156">briandowns/spinner#156</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/testwill"><code>@​testwill</code></a>
made their first contribution in <a
href="https://redirect.github.com/briandowns/spinner/pull/152">briandowns/spinner#152</a></li>
<li><a
href="https://github.com/debarshiray"><code>@​debarshiray</code></a>
made their first contribution in <a
href="https://redirect.github.com/briandowns/spinner/pull/156">briandowns/spinner#156</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/briandowns/spinner/compare/v1.23.0...v1.23.1">https://github.com/briandowns/spinner/compare/v1.23.0...v1.23.1</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="8f269dd04f"><code>8f269dd</code></a>
Consolidate the dependencies for the IsTerminal() API (<a
href="https://redirect.github.com/briandowns/spinner/issues/156">#156</a>)</li>
<li><a
href="12e6c297db"><code>12e6c29</code></a>
fix: CVE-2022-29526 (<a
href="https://redirect.github.com/briandowns/spinner/issues/152">#152</a>)</li>
<li>See full diff in <a
href="https://github.com/briandowns/spinner/compare/v1.23.0...v1.23.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/briandowns/spinner&package-manager=go_modules&previous-version=1.23.0&new-version=1.23.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-17 12:18:08 +00:00
shreyas-goenka ac6b80ed88
Remove user credentials specified in the Git origin URL (#1494)
## Changes
We set the origin URL as metadata in any jobs created by DABs. This PR
makes sure user credentials do not leak into the set metadata in the
job.
 
## Tests
Unit test

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-06-17 09:49:00 +00:00
shreyas-goenka 44e3928d6a
Avoid multiple file tree traversals on bundle deploy (#1493)
## Changes
To run bundle deploy from DBR we use an abstraction over the workspace
import / export APIs to create a `filer.Filer` and abstract the file
system. Walking the file tree in such a filer is expensive and requires
multiple API calls. This PR remove the two duplicate file tree walks
that happen by caching the result.
2024-06-17 09:48:52 +00:00
dependabot[bot] a5e89fd382
Bump golang.org/x/oauth2 from 0.20.0 to 0.21.0 (#1485)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from
0.20.0 to 0.21.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="5fd42413ed"><code>5fd4241</code></a>
google: update compute token refresh</li>
<li>See full diff in <a
href="https://github.com/golang/oauth2/compare/v0.20.0...v0.21.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/oauth2&package-manager=go_modules&previous-version=0.20.0&new-version=0.21.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-10 12:19:21 +00:00
dependabot[bot] 645e9ba8c4
Bump golang.org/x/mod from 0.17.0 to 0.18.0 (#1484)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.17.0 to
0.18.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c0bdc7bd01"><code>c0bdc7b</code></a>
modfile: add API for godebug lines</li>
<li><a
href="6686f41697"><code>6686f41</code></a>
module: add COM0 and LPT0 to badWindowsNames</li>
<li>See full diff in <a
href="https://github.com/golang/mod/compare/v0.17.0...v0.18.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/mod&package-manager=go_modules&previous-version=0.17.0&new-version=0.18.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-10 12:03:29 +00:00
dependabot[bot] 3d3ab50ff9
Bump golang.org/x/term from 0.20.0 to 0.21.0 (#1483)
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.20.0 to
0.21.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="5f0bb72315"><code>5f0bb72</code></a>
go.mod: update golang.org/x dependencies</li>
<li>See full diff in <a
href="https://github.com/golang/term/compare/v0.20.0...v0.21.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/term&package-manager=go_modules&previous-version=0.20.0&new-version=0.21.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-10 12:02:59 +00:00
dependabot[bot] b92e072add
Bump golang.org/x/text from 0.15.0 to 0.16.0 (#1482)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.15.0 to
0.16.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9c2f3a2135"><code>9c2f3a2</code></a>
cmd/gotext: fix segfault in extract &amp; rewrite commands</li>
<li><a
href="59e1219a5f"><code>59e1219</code></a>
message: optimize lookupAndFormat function for better performance</li>
<li><a
href="a20a3e2496"><code>a20a3e2</code></a>
x/text: update x/tools for go/ssa range-over-func fix</li>
<li>See full diff in <a
href="https://github.com/golang/text/compare/v0.15.0...v0.16.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/text&package-manager=go_modules&previous-version=0.15.0&new-version=0.16.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-10 11:47:22 +00:00
Lennart Kats (databricks) 99c7d136d6
Fix conditional in query in `default-sql` template (#1479)
## Changes

This corrects a mistake in the sample SQL identified by @pietern
2024-06-06 07:40:15 +00:00
Arpit Jasapara 35186d5ddb
Add randIntn function (#1475)
## Changes
<!-- Summary of your changes that are easy to understand -->
Add support for `math/rand.Intn` to DAB templates.

## Tests
<!-- How is this tested? -->
Unit tests.
2024-06-06 07:11:23 +00:00
Pieter Noordhuis 8c9fff3cb9
Release v0.221.1 (#1478)
Bundles:

This releases fixes an issue introduced in v0.221.0 where managing jobs
with a single-node cluster would fail.

* Fix SQL schema selection in default-sql template
([#1471](https://github.com/databricks/cli/pull/1471)).
* Copy-editing for SQL templates
([#1474](https://github.com/databricks/cli/pull/1474)).
* Upgrade TF provider to 1.47.0
([#1476](https://github.com/databricks/cli/pull/1476)).

Internal:
* Use latest version of goreleaser action
([#1477](https://github.com/databricks/cli/pull/1477)).
2024-06-05 12:29:20 +00:00
Pieter Noordhuis 1451361c9f
Use latest version of goreleaser action (#1477)
## Changes

Same as
https://github.com/databricks/terraform-provider-databricks/pull/3645.

## Tests

n/a
2024-06-05 11:33:51 +00:00
Pieter Noordhuis 311dfa4642
Upgrade TF provider to 1.47.0 (#1476)
## Changes

This includes a bugfix for provisioning jobs with `num_workers = 0`.

Fixes #1472.

## Tests

Manually tested this fixes the issue.
2024-06-05 11:33:43 +00:00
Lennart Kats (databricks) 41678fa695
Copy-editing for SQL templates (#1474)
## Changes

This applies changes suggested by @juliacrawf-db
2024-06-05 11:13:32 +00:00
Lennart Kats (databricks) 4bc0ea0af3
Fix SQL schema selection in default-sql template (#1471)
## Changes

This fixes a last-minute regression that snuck into
https://github.com/databricks/cli/pull/1463: unfortunately we need to
use `USE IDENTIFIER('schema')` to select a schema for now. In the future
we expect we can just use `USE SCHEMA 'schema'`.
2024-06-04 15:40:40 +00:00
Pieter Noordhuis f8b2cb89d5
Release v0.221.0 (#1470)
CLI:
* Update OpenAPI spec
([#1466](https://github.com/databricks/cli/pull/1466)).

Bundles:
* Upgrade TF provider to 1.46.0
([#1460](https://github.com/databricks/cli/pull/1460)).
* Add support for Lakehouse monitoring
([#1307](https://github.com/databricks/cli/pull/1307)).
* Make dbt-sql and default-sql templates public
([#1463](https://github.com/databricks/cli/pull/1463)).

Internal:
* Abstract over filesystem interaction with libs/vfs
([#1452](https://github.com/databricks/cli/pull/1452)).
* Add `filer.Filer` to read notebooks from WSFS without omitting their
extension ([#1457](https://github.com/databricks/cli/pull/1457)).
* Fix listing notebooks in a subdirectory
([#1468](https://github.com/databricks/cli/pull/1468)).

API Changes:
* Changed `databricks account storage-credentials list` command to
return .
 * Added `databricks consumer-listings batch-get` command.
 * Added `databricks consumer-providers batch-get` command.
 * Removed `databricks apps create-deployment` command.
 * Added `databricks apps deploy` command.

OpenAPI commit 37b925eba37dfb3d7e05b6ba2d458454ce62d3a0 (2024-06-03)

Dependency updates:
* Bump github.com/hashicorp/go-version from 1.6.0 to 1.7.0
([#1454](https://github.com/databricks/cli/pull/1454)).
* Bump github.com/hashicorp/hc-install from 0.6.4 to 0.7.0
([#1453](https://github.com/databricks/cli/pull/1453)).
2024-06-04 11:03:19 +00:00
Pieter Noordhuis 448d41027d
Fix listing notebooks in a subdirectory (#1468)
## Changes

This worked fine if the notebooks are located in the filer's root and
didn't if they are nested in a directory.

This change adds test coverage and fixes the underlying issue.

## Tests

Ran integration test manually.
2024-06-04 09:53:14 +00:00
Lennart Kats (databricks) aa36aee159
Make dbt-sql and default-sql templates public (#1463)
## Changes

This makes the dbt-sql and default-sql templates public.

These templates were previously not listed and marked "experimental"
since structured streaming tables were still in gated preview and would
result in weird error messages when a workspace wasn't enabled for the
preview.

This PR also incorporates some of the feedback and learnings for these
templates so far.
2024-06-04 08:57:13 +00:00
Pieter Noordhuis 70fd8ad3d7
Update OpenAPI spec (#1466)
## Changes

Notable changes:

* Pagination of account-level storage credentials
* Rename app deployment method

Go SDK release notes:
https://github.com/databricks/databricks-sdk-go/releases/tag/v0.42.0

## Tests

* Nightlies pass.
2024-06-03 14:14:48 +00:00
Pieter Noordhuis c9b4f11947
Update error checks that use the `os` package to use `errors.Is` (#1461)
## Changes

From the [documentation](https://pkg.go.dev/os#IsNotExist) on the
functions in the `os` package:
> This function predates errors.Is. It only supports errors returned by
the os package.
> New code should use errors.Is(err, fs.ErrNotExist).

This issue surfaced while working on using a different `vfs.Path`
implementation that uses errors from the `fs` package. Calls to
`os.IsNotExist` didn't return true for errors that wrap
`fs.ErrNotExist`.

## Tests

n/a
2024-06-03 12:39:36 +00:00
Pieter Noordhuis 30fd84893f
Generate bundle schema placeholder for quality monitors (#1465)
## Changes

Generated with default generation command.

The team is making a fix to ensure the proper comments are included
later.

## Tests

n/a
2024-06-03 12:39:27 +00:00
Aravind Segu a33d0c8bf9
Add support for Lakehouse monitoring in bundles (#1307)
## Changes

This change adds support for Lakehouse monitoring in bundles.

The associated resource type name is "quality monitor".

## Testing

Unit tests.

---------

Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com>
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
Co-authored-by: Arpit Jasapara <87999496+arpitjasa-db@users.noreply.github.com>
2024-05-31 09:42:25 +00:00
Pieter Noordhuis 364a609ea7
Upgrade TF provider to 1.46.0 (#1460)
## Changes

Release notes in
https://github.com/databricks/terraform-provider-databricks/releases/tag/v1.46.0

Notable changes since 1.43.0:
* The job resource has been migrated to the Go SDK. More fields are now
passed through from DABs into TF.
* Improved zero-value handling.

## Tests

n/a
2024-05-31 07:13:43 +00:00
shreyas-goenka ec33a7c059
Add `filer.Filer` to read notebooks from WSFS without omitting their extension (#1457)
## Changes
This PR adds a filer that'll allow us to read notebooks from the WSFS
using their full paths (with the extension included). The filer relies
on the existing workspace filer (and consequently the workspace
import/export/list APIs).

Using this filer along with a virtual filesystem layer
(https://github.com/databricks/cli/pull/1452/files) will allow us to use
our custom implementation (which preserves the notebook extensions)
rather than the default mount available via DBR when the CLI is run from
DBR.

## Tests
Integration tests.

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-05-30 11:59:27 +00:00
Pieter Noordhuis 424499ec1d
Abstract over filesystem interaction with libs/vfs (#1452)
## Changes

Introduce `libs/vfs` for an implementation of `fs.FS` and friends that
_includes_ the absolute path it is anchored to.

This is needed for:
1. Intercepting file operations to inject custom logic (e.g., logging,
access control).
2. Traversing directories to find specific leaf directories (e.g.,
`.git`).
3. Converting virtual paths to OS-native paths.

Options 2 and 3 are not possible with the standard `fs.FS` interface.
They are needed such that we can provide an instance to the sync package
and still detect the containing `.git` directory and convert paths to
native paths.

This change focuses on making the following packages use `vfs.Path`:
* libs/fileset
* libs/git
* libs/sync

All entries returned by `fileset.All` are now slash-separated. This has
2 consequences:
* The sync snapshot now always uses slash-separated paths
* We don't need to call `filepath.FromSlash` as much as we did

## Tests

* All unit tests pass
* All integration tests pass
* Manually confirmed that a deployment made on Windows by a previous
version of the CLI can be deployed by a new version of the CLI while
retaining the validity of the local sync snapshot as well as the remote
deployment state.
2024-05-30 07:41:50 +00:00
Pieter Noordhuis b2ea9dd971
Remove unnecessary `filepath.FromSlash` calls (#1458)
## Changes

The prior join call calls `filepath.Join` which returns a cleaned
result.

Path cleaning, in turn, calls `filepath.FromSlash`.

## Tests

* Unit tests.
2024-05-29 15:30:26 +00:00
dependabot[bot] 13b937cea8
Bump github.com/hashicorp/hc-install from 0.6.4 to 0.7.0 (#1453)
Bumps
[github.com/hashicorp/hc-install](https://github.com/hashicorp/hc-install)
from 0.6.4 to 0.7.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/hashicorp/hc-install/releases">github.com/hashicorp/hc-install's
releases</a>.</em></p>
<blockquote>
<h2>v0.7.0</h2>
<p>ENHANCEMENTS:</p>
<ul>
<li>cmd/hc-install: Enable logging into a file by <a
href="https://github.com/radeksimko"><code>@​radeksimko</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/193">hashicorp/hc-install#193</a></li>
<li>Ensure license file gets packaged along w/ the CLI binary by <a
href="https://github.com/radeksimko"><code>@​radeksimko</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/205">hashicorp/hc-install#205</a></li>
<li>Ensure license files are tracked during installation so they can be
removed by <a
href="https://github.com/radeksimko"><code>@​radeksimko</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/206">hashicorp/hc-install#206</a></li>
<li>Add support for custom download URLs by <a
href="https://github.com/james0209"><code>@​james0209</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/203">hashicorp/hc-install#203</a></li>
</ul>
<p>BUG FIXES:</p>
<ul>
<li>releasesjson: Fix missing <code>Version</code> in unmarshaled
response by <a
href="https://github.com/radeksimko"><code>@​radeksimko</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/192">hashicorp/hc-install#192</a></li>
</ul>
<p>DEPENDENCIES:</p>
<ul>
<li>build(deps): bump golang.org/x/mod from 0.16.0 to 0.17.0 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/194">hashicorp/hc-install#194</a></li>
<li>build(deps): bump golang.org/x/net from 0.22.0 to 0.23.0 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/195">hashicorp/hc-install#195</a></li>
<li>build(deps): bump hashicorp/action-setup-bob from 1.0.20 to 2 in the
github-actions-breaking group by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/208">hashicorp/hc-install#208</a></li>
<li>build(deps): bump hashicorp/setup-copywrite from 1.1.2 to 1.1.3 in
the github-actions-backward-compatible group by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/212">hashicorp/hc-install#212</a></li>
<li>build(deps): bump the github-actions-breaking group with 2 updates
by <a href="https://github.com/dependabot"><code>@​dependabot</code></a>
in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/211">hashicorp/hc-install#211</a></li>
<li>build(deps): Bump workflows to latest trusted versions by <a
href="https://github.com/hashicorp-tsccr"><code>@​hashicorp-tsccr</code></a>
in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/199">hashicorp/hc-install#199</a></li>
<li>build(deps): Bump workflows to latest trusted versions by <a
href="https://github.com/hashicorp-tsccr"><code>@​hashicorp-tsccr</code></a>
in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/204">hashicorp/hc-install#204</a></li>
<li>build(deps): Bump workflows to latest trusted versions by <a
href="https://github.com/hashicorp-tsccr"><code>@​hashicorp-tsccr</code></a>
in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/209">hashicorp/hc-install#209</a></li>
</ul>
<p>INTERNAL:</p>
<ul>
<li>[chore] Fix minor lint issue by <a
href="https://github.com/james0209"><code>@​james0209</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/202">hashicorp/hc-install#202</a></li>
<li>github: Create CODEOWNERS by <a
href="https://github.com/xiehan"><code>@​xiehan</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/210">hashicorp/hc-install#210</a></li>
<li>github: Set up Dependabot to manage HashiCorp-owned Actions
versioning by <a
href="https://github.com/xiehan"><code>@​xiehan</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/207">hashicorp/hc-install#207</a></li>
<li>Lint Markdown by <a
href="https://github.com/james0209"><code>@​james0209</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/200">hashicorp/hc-install#200</a></li>
<li>Replace deprecated <code>ioutil</code> usage with the equivalent
replacements by <a
href="https://github.com/james0209"><code>@​james0209</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/198">hashicorp/hc-install#198</a></li>
<li>Support darwin/arm64 in tests by <a
href="https://github.com/radeksimko"><code>@​radeksimko</code></a> in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/196">hashicorp/hc-install#196</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/james0209"><code>@​james0209</code></a>
made their first contribution in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/198">hashicorp/hc-install#198</a></li>
<li><a href="https://github.com/xiehan"><code>@​xiehan</code></a> made
their first contribution in <a
href="https://redirect.github.com/hashicorp/hc-install/pull/207">hashicorp/hc-install#207</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/hashicorp/hc-install/compare/v0.6.4...v0.7.0">https://github.com/hashicorp/hc-install/compare/v0.6.4...v0.7.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="152a3b6753"><code>152a3b6</code></a>
Release v0.7.0</li>
<li><a
href="237ac6f0aa"><code>237ac6f</code></a>
Ensure license files are tracked during installation so they can be
removed (...</li>
<li><a
href="5a74938d4a"><code>5a74938</code></a>
github: Create CODEOWNERS (<a
href="https://redirect.github.com/hashicorp/hc-install/issues/210">#210</a>)</li>
<li><a
href="40acb8c900"><code>40acb8c</code></a>
build(deps): bump the github-actions-breaking group with 2 updates (<a
href="https://redirect.github.com/hashicorp/hc-install/issues/211">#211</a>)</li>
<li><a
href="b19d1fcd87"><code>b19d1fc</code></a>
build(deps): bump hashicorp/setup-copywrite from 1.1.2 to 1.1.3 in the
github...</li>
<li><a
href="e0945975dc"><code>e094597</code></a>
Result of tsccr-helper -log-level=info gha update -latest . (<a
href="https://redirect.github.com/hashicorp/hc-install/issues/209">#209</a>)</li>
<li><a
href="b5c313e484"><code>b5c313e</code></a>
build(deps): bump hashicorp/action-setup-bob (<a
href="https://redirect.github.com/hashicorp/hc-install/issues/208">#208</a>)</li>
<li><a
href="35884ef0d1"><code>35884ef</code></a>
github: Set up Dependabot to manage HashiCorp-owned Actions versioning
(<a
href="https://redirect.github.com/hashicorp/hc-install/issues/207">#207</a>)</li>
<li><a
href="704a29eb54"><code>704a29e</code></a>
Add support for custom download URLs (<a
href="https://redirect.github.com/hashicorp/hc-install/issues/203">#203</a>)</li>
<li><a
href="7de7b37ef8"><code>7de7b37</code></a>
Ensure license file gets packaged along w/ the CLI binary (<a
href="https://redirect.github.com/hashicorp/hc-install/issues/205">#205</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/hashicorp/hc-install/compare/v0.6.4...v0.7.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/hashicorp/hc-install&package-manager=go_modules&previous-version=0.6.4&new-version=0.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-28 12:41:20 +00:00
dependabot[bot] 9a452f38ee
Bump github.com/hashicorp/go-version from 1.6.0 to 1.7.0 (#1454)
Bumps
[github.com/hashicorp/go-version](https://github.com/hashicorp/go-version)
from 1.6.0 to 1.7.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/hashicorp/go-version/releases">github.com/hashicorp/go-version's
releases</a>.</em></p>
<blockquote>
<h2>v1.7.0</h2>
<p>ENHANCEMENTS:</p>
<ul>
<li>Remove <code>reflect</code> dependency (<a
href="https://redirect.github.com/hashicorp/go-version/pull/91">#91</a>)</li>
<li>Implement the <code>database/sql.Scanner</code> and
<code>database/sql/driver.Value</code> interfaces for
<code>Version</code> (<a
href="https://redirect.github.com/hashicorp/go-version/pull/133">#133</a>)</li>
</ul>
<p>INTERNAL:</p>
<ul>
<li>[COMPLIANCE] Add Copyright and License Headers (<a
href="https://redirect.github.com/hashicorp/go-version/pull/115">#115</a>)</li>
<li>[COMPLIANCE] Update MPL-2.0 LICENSE (<a
href="https://redirect.github.com/hashicorp/go-version/pull/105">#105</a>)</li>
<li>Bump actions/cache from 3.0.11 to 3.2.5 (<a
href="https://redirect.github.com/hashicorp/go-version/pull/116">#116</a>)</li>
<li>Bump actions/checkout from 3.2.0 to 3.3.0 (<a
href="https://redirect.github.com/hashicorp/go-version/pull/111">#111</a>)</li>
<li>Bump actions/upload-artifact from 3.1.1 to 3.1.2 (<a
href="https://redirect.github.com/hashicorp/go-version/pull/112">#112</a>)</li>
<li>GHA Migration (<a
href="https://redirect.github.com/hashicorp/go-version/pull/103">#103</a>)</li>
<li>github: Pin external GitHub Actions to hashes (<a
href="https://redirect.github.com/hashicorp/go-version/pull/107">#107</a>)</li>
<li>SEC-090: Automated trusted workflow pinning (2023-04-05) (<a
href="https://redirect.github.com/hashicorp/go-version/pull/124">#124</a>)</li>
<li>update readme (<a
href="https://redirect.github.com/hashicorp/go-version/pull/104">#104</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/hashicorp/go-version/blob/main/CHANGELOG.md">github.com/hashicorp/go-version's
changelog</a>.</em></p>
<blockquote>
<h1>1.7.0 (May 24, 2024)</h1>
<p>ENHANCEMENTS:</p>
<ul>
<li>Remove <code>reflect</code> dependency (<a
href="https://redirect.github.com/hashicorp/go-version/pull/91">#91</a>)</li>
<li>Implement the <code>database/sql.Scanner</code> and
<code>database/sql/driver.Value</code> interfaces for
<code>Version</code> (<a
href="https://redirect.github.com/hashicorp/go-version/pull/133">#133</a>)</li>
</ul>
<p>INTERNAL:</p>
<ul>
<li>[COMPLIANCE] Add Copyright and License Headers (<a
href="https://redirect.github.com/hashicorp/go-version/pull/115">#115</a>)</li>
<li>[COMPLIANCE] Update MPL-2.0 LICENSE (<a
href="https://redirect.github.com/hashicorp/go-version/pull/105">#105</a>)</li>
<li>Bump actions/cache from 3.0.11 to 3.2.5 (<a
href="https://redirect.github.com/hashicorp/go-version/pull/116">#116</a>)</li>
<li>Bump actions/checkout from 3.2.0 to 3.3.0 (<a
href="https://redirect.github.com/hashicorp/go-version/pull/111">#111</a>)</li>
<li>Bump actions/upload-artifact from 3.1.1 to 3.1.2 (<a
href="https://redirect.github.com/hashicorp/go-version/pull/112">#112</a>)</li>
<li>GHA Migration (<a
href="https://redirect.github.com/hashicorp/go-version/pull/103">#103</a>)</li>
<li>github: Pin external GitHub Actions to hashes (<a
href="https://redirect.github.com/hashicorp/go-version/pull/107">#107</a>)</li>
<li>SEC-090: Automated trusted workflow pinning (2023-04-05) (<a
href="https://redirect.github.com/hashicorp/go-version/pull/124">#124</a>)</li>
<li>update readme (<a
href="https://redirect.github.com/hashicorp/go-version/pull/104">#104</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="fcaa532679"><code>fcaa532</code></a>
Update CHANGELOG.md</li>
<li><a
href="b85381af95"><code>b85381a</code></a>
Update CHANGELOG.md</li>
<li><a
href="d55f21493f"><code>d55f214</code></a>
Implement the Scan and driver.Value SQL interfaces (<a
href="https://redirect.github.com/hashicorp/go-version/issues/133">#133</a>)</li>
<li><a
href="e04a8660de"><code>e04a866</code></a>
remove reflection dependency (<a
href="https://redirect.github.com/hashicorp/go-version/issues/91">#91</a>)</li>
<li><a
href="94bab9eaa8"><code>94bab9e</code></a>
[COMPLIANCE] Add Copyright and License Headers (<a
href="https://redirect.github.com/hashicorp/go-version/issues/115">#115</a>)</li>
<li><a
href="73ddc6313d"><code>73ddc63</code></a>
github: Change Dependabot to only manage HashiCorp-owned Actions</li>
<li><a
href="bf1144e59f"><code>bf1144e</code></a>
SEC-090: Automated trusted workflow pinning (2023-04-05) (<a
href="https://redirect.github.com/hashicorp/go-version/issues/124">#124</a>)</li>
<li><a
href="644291d140"><code>644291d</code></a>
Bump actions/cache from 3.0.11 to 3.2.5 (<a
href="https://redirect.github.com/hashicorp/go-version/issues/116">#116</a>)</li>
<li><a
href="8f6487b06e"><code>8f6487b</code></a>
Bump actions/upload-artifact from 3.1.1 to 3.1.2 (<a
href="https://redirect.github.com/hashicorp/go-version/issues/112">#112</a>)</li>
<li><a
href="7f856b8ccc"><code>7f856b8</code></a>
Bump actions/checkout from 3.2.0 to 3.3.0 (<a
href="https://redirect.github.com/hashicorp/go-version/issues/111">#111</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/hashicorp/go-version/compare/v1.6.0...v1.7.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/hashicorp/go-version&package-manager=go_modules&previous-version=1.6.0&new-version=1.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-28 12:20:11 +00:00
Pieter Noordhuis 46f6cbcfc3
Release v0.220.0 (#1446)
CLI:
* Add line about Docker installation to README.md
([#1363](https://github.com/databricks/cli/pull/1363)).
* Improve token refresh flow
([#1434](https://github.com/databricks/cli/pull/1434)).

Bundles:
* Upgrade Terraform provider to v1.42.0
([#1418](https://github.com/databricks/cli/pull/1418)).
* Upgrade Terraform provider to v1.43.0
([#1429](https://github.com/databricks/cli/pull/1429)).
* Don't merge-in remote resources during deployments
([#1432](https://github.com/databricks/cli/pull/1432)).
* Remove dependency on `ConfigFilePath` from path translation mutator
([#1437](https://github.com/databricks/cli/pull/1437)).
* Add `merge.Override` transform
([#1428](https://github.com/databricks/cli/pull/1428)).
* Fixed panic when loading incorrectly defined jobs
([#1402](https://github.com/databricks/cli/pull/1402)).
* Add more tests for `merge.Override`
([#1439](https://github.com/databricks/cli/pull/1439)).
* Fixed seg fault when specifying environment key for tasks
([#1443](https://github.com/databricks/cli/pull/1443)).
* Fix conversion of zero valued scalar pointers to a dynamic value
([#1433](https://github.com/databricks/cli/pull/1433)).

Internal:
* Don't hide commands of services that are already hidden
([#1438](https://github.com/databricks/cli/pull/1438)).

API Changes:
 * Renamed `lakehouse-monitors` command group to `quality-monitors`.
 * Added `apps` command group.
* Renamed `csp-enablement` command group to
`compliance-security-profile`.
* Renamed `esm-enablement` command group to
`enhanced-security-monitoring`.
 * Added `databricks vector-search-indexes scan-index` command.

OpenAPI commit 7eb5ad9a2ed3e3f1055968a2d1014ac92c06fe92 (2024-05-21)

Dependency updates:
* Bump golang.org/x/text from 0.14.0 to 0.15.0
([#1419](https://github.com/databricks/cli/pull/1419)).
* Bump golang.org/x/oauth2 from 0.19.0 to 0.20.0
([#1421](https://github.com/databricks/cli/pull/1421)).
* Bump golang.org/x/term from 0.19.0 to 0.20.0
([#1422](https://github.com/databricks/cli/pull/1422)).
* Bump github.com/databricks/databricks-sdk-go from 0.39.0 to 0.40.1
([#1431](https://github.com/databricks/cli/pull/1431)).
* Bump github.com/fatih/color from 1.16.0 to 1.17.0
([#1441](https://github.com/databricks/cli/pull/1441)).
* Bump github.com/hashicorp/terraform-json from 0.21.0 to 0.22.1
([#1440](https://github.com/databricks/cli/pull/1440)).
* Bump github.com/hashicorp/terraform-exec from 0.20.0 to 0.21.0
([#1442](https://github.com/databricks/cli/pull/1442)).
* Update Go SDK to v0.41.0
([#1445](https://github.com/databricks/cli/pull/1445)).
2024-05-22 09:08:27 +00:00