mirror of https://github.com/databricks/cli.git
1241 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
Andrew Nester
|
5f42791609
|
Added support for complex variables (#1467)
## Changes
Added support for complex variables
Now it's possible to add and use complex variables as shown below
```
bundle:
name: complex-variables
resources:
jobs:
my_job:
job_clusters:
- job_cluster_key: key
new_cluster: ${var.cluster}
tasks:
- task_key: test
job_cluster_key: key
variables:
cluster:
description: "A cluster definition"
type: complex
default:
spark_version: "13.2.x-scala2.11"
node_type_id: "Standard_DS3_v2"
num_workers: 2
spark_conf:
spark.speculation: true
spark.databricks.delta.retentionDurationCheck.enabled: false
```
Fixes #1298
- [x] Support for complex variables
- [x] Allow variable overrides (with shortcut) in targets
- [x] Don't allow to provide complex variables via flag or env variable
- [x] Fail validation if complex value is used but not `type: complex`
provided
- [x] Support using variables inside complex variables
## Tests
Added unit tests
---------
Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com>
|
|
Pieter Noordhuis
|
ce5a3f2ce6
|
Upgrade TF provider to 1.48.0 (#1527)
## Changes This includes a fix for library order not being respected. ## Tests Manually confirmed the fix works in https://github.com/databricks/bundle-examples/pull/29. |
|
|
Pieter Noordhuis
|
482d83cba8
|
Revert "Retain location metadata for values in `convert.FromTyped`" (#1528)
## Changes
This reverts commit
|
|
shreyas-goenka
|
dac5f09556
|
Retain location metadata for values in `convert.FromTyped` (#1523)
## Changes There are four different treatments location metadata can receive in the `convert.FromTyped` method. 1. Location metadata is **retained** for maps, structs and slices if the value is **not nil** 2. Location metadata is **lost** for maps, structs and slices if the value is **is nil** 3. Location metadata is **retained** if a scalar type (eg. bool, string etc) does not change. 4. Location metadata is **lost** if the value for a scalar type changes. This PR ensures that location metadata is not lost in any case; that is, it's always preserved. For (2), this serves as a bug fix so that location information is not lost on conversion to and from typed for nil values of complex types (struct, slices, and maps). For (4) this is a change in semantics. For primitive values modified in a `typed` mutator, any references to `.Location()` for computed primitive fields will now return associated YAML location metadata (if any) instead of an empty location. While arguable, these semantics are OK since: 1. Situations like these will be rare. 2. Knowing the YAML location (if any) is better than not knowing the location at all. These locations are typically visible to the user in errors and warnings. ## Tests Unit tests |
|
dependabot[bot]
|
8468878eed
|
Bump github.com/databricks/databricks-sdk-go from 0.42.0 to 0.43.0 (#1522)
Bumps [github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go) from 0.42.0 to 0.43.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/databricks/databricks-sdk-go/releases">github.com/databricks/databricks-sdk-go's releases</a>.</em></p> <blockquote> <h2>v0.43.0</h2> <p>Major Changes and Improvements:</p> <ul> <li>Support partners in user agent for SDK (<a href="https://redirect.github.com/databricks/databricks-sdk-go/pull/925">#925</a>).</li> <li>Add <code>serverless_compute_id</code> field to the config (<a href="https://redirect.github.com/databricks/databricks-sdk-go/pull/952">#952</a>).</li> </ul> <p>Other Changes:</p> <ul> <li>Generate from latest spec (<a href="https://redirect.github.com/databricks/databricks-sdk-go/pull/944">#944</a>) and (<a href="https://redirect.github.com/databricks/databricks-sdk-go/pull/947">#947</a>).</li> </ul> <p>API Changes:</p> <ul> <li>Changed <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogInfo">catalog.CatalogInfo</a> to <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li> <li>Added <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ExternalLocationInfo">catalog.ExternalLocationInfo</a>.</li> <li>Added <code>MaxResults</code> and <code>PageToken</code> fields for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsRequest">catalog.ListCatalogsRequest</a>.</li> <li>Added <code>NextPageToken</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsResponse">catalog.ListCatalogsResponse</a>.</li> <li>Added <code>TableServingUrl</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#OnlineTable">catalog.OnlineTable</a>.</li> <li>Added <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#StorageCredentialInfo">catalog.StorageCredentialInfo</a>.</li> <li>Changed <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateCatalog">catalog.UpdateCatalog</a> to <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li> <li>Added <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateExternalLocation">catalog.UpdateExternalLocation</a>.</li> <li>Added <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateStorageCredential">catalog.UpdateStorageCredential</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li> <li>Added <code>CreateSchedule</code>, <code>CreateSubscription</code>, <code>DeleteSchedule</code>, <code>DeleteSubscription</code>, <code>GetSchedule</code>, <code>GetSubscription</code>, <code>List</code>, <code>ListSchedules</code>, <code>ListSubscriptions</code> and <code>UpdateSchedule</code> methods for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#LakeviewAPI">w.Lakeview</a> workspace-level service.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateScheduleRequest">dashboards.CreateScheduleRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateSubscriptionRequest">dashboards.CreateSubscriptionRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CronSchedule">dashboards.CronSchedule</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DashboardView">dashboards.DashboardView</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteScheduleRequest">dashboards.DeleteScheduleRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteSubscriptionRequest">dashboards.DeleteSubscriptionRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetScheduleRequest">dashboards.GetScheduleRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetSubscriptionRequest">dashboards.GetSubscriptionRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsRequest">dashboards.ListDashboardsRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsResponse">dashboards.ListDashboardsResponse</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesRequest">dashboards.ListSchedulesRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesResponse">dashboards.ListSchedulesResponse</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsRequest">dashboards.ListSubscriptionsRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsResponse">dashboards.ListSubscriptionsResponse</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Schedule">dashboards.Schedule</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SchedulePauseStatus">dashboards.SchedulePauseStatus</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscriber">dashboards.Subscriber</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscription">dashboards.Subscription</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberDestination">dashboards.SubscriptionSubscriberDestination</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberUser">dashboards.SubscriptionSubscriberUser</a> and <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#UpdateScheduleRequest">dashboards.UpdateScheduleRequest</a> structs.</li> <li>Added <code>OnStreamingBacklogExceeded</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobEmailNotifications">jobs.JobEmailNotifications</a>.</li> <li>Added <code>EnvironmentKey</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunTask">jobs.RunTask</a>.</li> <li>Removed <code>ConditionTask</code>, <code>DbtTask</code>, <code>NotebookTask</code>, <code>PipelineTask</code>, <code>PythonWheelTask</code>, <code>RunJobTask</code>, <code>SparkJarTask</code>, <code>SparkPythonTask</code>, <code>SparkSubmitTask</code>, <code>SqlTask</code> and <code>Environments</code> fields for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun">jobs.SubmitRun</a>.</li> <li>Added <code>DbtTask</code> and <code>EnvironmentKey</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask">jobs.SubmitTask</a>.</li> <li>Added <code>OnStreamingBacklogExceeded</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TaskEmailNotifications">jobs.TaskEmailNotifications</a>.</li> <li>Added <code>Periodic</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TriggerSettings">jobs.TriggerSettings</a>.</li> <li>Added <code>OnStreamingBacklogExceeded</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#WebhookNotifications">jobs.WebhookNotifications</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfiguration">jobs.PeriodicTriggerConfiguration</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfigurationTimeUnit">jobs.PeriodicTriggerConfigurationTimeUnit</a>.</li> <li>Added <code>ProviderSummary</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#Listing">marketplace.Listing</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconFile">marketplace.ProviderIconFile</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconType">marketplace.ProviderIconType</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderListingSummaryInfo">marketplace.ProviderListingSummaryInfo</a>.</li> <li>Added <code>Start</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#AppsAPI">w.Apps</a> workspace-level service.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#ServingEndpointsDataPlaneAPI">w.ServingEndpointsDataPlane</a> workspace-level service.</li> <li>Added <code>ServicePrincipalId</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App">serving.App</a>.</li> <li>Added <code>ServicePrincipalName</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App">serving.App</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#StartAppRequest">serving.StartAppRequest</a>.</li> <li>Added <code>QueryNextPage</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#VectorSearchIndexesAPI">w.VectorSearchIndexes</a> workspace-level service.</li> <li>Added <code>QueryType</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexRequest">vectorsearch.QueryVectorIndexRequest</a>.</li> <li>Added <code>NextPageToken</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexResponse">vectorsearch.QueryVectorIndexResponse</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexNextPageRequest">vectorsearch.QueryVectorIndexNextPageRequest</a>.</li> </ul> <p>OpenAPI SHA: 7437dabb9dadee402c1fc060df4c1ce8cc5369f0, Date: 2024-06-25</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/databricks/databricks-sdk-go/blob/main/CHANGELOG.md">github.com/databricks/databricks-sdk-go's changelog</a>.</em></p> <blockquote> <h2>0.43.0</h2> <p>Major Changes and Improvements:</p> <ul> <li>Support partners in user agent for SDK (<a href="https://redirect.github.com/databricks/databricks-sdk-go/pull/925">#925</a>).</li> <li>Add <code>serverless_compute_id</code> field to the config (<a href="https://redirect.github.com/databricks/databricks-sdk-go/pull/952">#952</a>).</li> </ul> <p>Other Changes:</p> <ul> <li>Generate from latest spec (<a href="https://redirect.github.com/databricks/databricks-sdk-go/pull/944">#944</a>) and (<a href="https://redirect.github.com/databricks/databricks-sdk-go/pull/947">#947</a>).</li> </ul> <p>API Changes:</p> <ul> <li>Changed <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogInfo">catalog.CatalogInfo</a> to <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li> <li>Added <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ExternalLocationInfo">catalog.ExternalLocationInfo</a>.</li> <li>Added <code>MaxResults</code> and <code>PageToken</code> fields for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsRequest">catalog.ListCatalogsRequest</a>.</li> <li>Added <code>NextPageToken</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#ListCatalogsResponse">catalog.ListCatalogsResponse</a>.</li> <li>Added <code>TableServingUrl</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#OnlineTable">catalog.OnlineTable</a>.</li> <li>Added <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#StorageCredentialInfo">catalog.StorageCredentialInfo</a>.</li> <li>Changed <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateCatalog">catalog.UpdateCatalog</a> to <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li> <li>Added <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateExternalLocation">catalog.UpdateExternalLocation</a>.</li> <li>Added <code>IsolationMode</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateStorageCredential">catalog.UpdateStorageCredential</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#CatalogIsolationMode">catalog.CatalogIsolationMode</a>.</li> <li>Added <code>CreateSchedule</code>, <code>CreateSubscription</code>, <code>DeleteSchedule</code>, <code>DeleteSubscription</code>, <code>GetSchedule</code>, <code>GetSubscription</code>, <code>List</code>, <code>ListSchedules</code>, <code>ListSubscriptions</code> and <code>UpdateSchedule</code> methods for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#LakeviewAPI">w.Lakeview</a> workspace-level service.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateScheduleRequest">dashboards.CreateScheduleRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CreateSubscriptionRequest">dashboards.CreateSubscriptionRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#CronSchedule">dashboards.CronSchedule</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DashboardView">dashboards.DashboardView</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteScheduleRequest">dashboards.DeleteScheduleRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#DeleteSubscriptionRequest">dashboards.DeleteSubscriptionRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetScheduleRequest">dashboards.GetScheduleRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GetSubscriptionRequest">dashboards.GetSubscriptionRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsRequest">dashboards.ListDashboardsRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListDashboardsResponse">dashboards.ListDashboardsResponse</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesRequest">dashboards.ListSchedulesRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSchedulesResponse">dashboards.ListSchedulesResponse</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsRequest">dashboards.ListSubscriptionsRequest</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#ListSubscriptionsResponse">dashboards.ListSubscriptionsResponse</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Schedule">dashboards.Schedule</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SchedulePauseStatus">dashboards.SchedulePauseStatus</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscriber">dashboards.Subscriber</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#Subscription">dashboards.Subscription</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberDestination">dashboards.SubscriptionSubscriberDestination</a>, <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#SubscriptionSubscriberUser">dashboards.SubscriptionSubscriberUser</a> and <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#UpdateScheduleRequest">dashboards.UpdateScheduleRequest</a> structs.</li> <li>Added <code>OnStreamingBacklogExceeded</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobEmailNotifications">jobs.JobEmailNotifications</a>.</li> <li>Added <code>EnvironmentKey</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunTask">jobs.RunTask</a>.</li> <li>Removed <code>ConditionTask</code>, <code>DbtTask</code>, <code>NotebookTask</code>, <code>PipelineTask</code>, <code>PythonWheelTask</code>, <code>RunJobTask</code>, <code>SparkJarTask</code>, <code>SparkPythonTask</code>, <code>SparkSubmitTask</code>, <code>SqlTask</code> and <code>Environments</code> fields for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun">jobs.SubmitRun</a>.</li> <li>Added <code>DbtTask</code> and <code>EnvironmentKey</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask">jobs.SubmitTask</a>.</li> <li>Added <code>OnStreamingBacklogExceeded</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TaskEmailNotifications">jobs.TaskEmailNotifications</a>.</li> <li>Added <code>Periodic</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TriggerSettings">jobs.TriggerSettings</a>.</li> <li>Added <code>OnStreamingBacklogExceeded</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#WebhookNotifications">jobs.WebhookNotifications</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfiguration">jobs.PeriodicTriggerConfiguration</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#PeriodicTriggerConfigurationTimeUnit">jobs.PeriodicTriggerConfigurationTimeUnit</a>.</li> <li>Added <code>ProviderSummary</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#Listing">marketplace.Listing</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconFile">marketplace.ProviderIconFile</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderIconType">marketplace.ProviderIconType</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/marketplace#ProviderListingSummaryInfo">marketplace.ProviderListingSummaryInfo</a>.</li> <li>Added <code>Start</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#AppsAPI">w.Apps</a> workspace-level service.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#ServingEndpointsDataPlaneAPI">w.ServingEndpointsDataPlane</a> workspace-level service.</li> <li>Added <code>ServicePrincipalId</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App">serving.App</a>.</li> <li>Added <code>ServicePrincipalName</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#App">serving.App</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/serving#StartAppRequest">serving.StartAppRequest</a>.</li> <li>Added <code>QueryNextPage</code> method for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#VectorSearchIndexesAPI">w.VectorSearchIndexes</a> workspace-level service.</li> <li>Added <code>QueryType</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexRequest">vectorsearch.QueryVectorIndexRequest</a>.</li> <li>Added <code>NextPageToken</code> field for <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexResponse">vectorsearch.QueryVectorIndexResponse</a>.</li> <li>Added <a href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/vectorsearch#QueryVectorIndexNextPageRequest">vectorsearch.QueryVectorIndexNextPageRequest</a>.</li> </ul> <p>OpenAPI SHA: 7437dabb9dadee402c1fc060df4c1ce8cc5369f0, Date: 2024-06-25</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
|
|
Pieter Noordhuis
|
100a0516d4
|
Add context type and value to path rewriting (#1525)
## Changes For a future change where the inner rewriting functions need access to the underlying bundle, this change makes preparations. All values were passed via the stack before and adding yet another value would make the code less readable. ## Tests Unit tests pass. |
|
Kai Zhu
|
2ec6abf74e
|
Fix `databricks configure` to use DATABRICKS_CONFIG_FILE environment variable if exists as config file (#1325)
## Changes added `ConfigFile: cfg.ConfigFile` for `databrickscfg.SaveToProfile` in `cmd/configure/configure.go` to save the file in a specified path when the value is not empty ## Tests `TestConfigFileFromEnvNoInteractive` in `cmd/configure/configure_test.go` sets a different config file path by `DATABRICKS_CONFIG_FILE`, after execution, the overwrite config file is generated, and the default path has no file. |
|
|
Pieter Noordhuis
|
8957f1e7cf
|
Return `fs.ModeDir` for Git folders in the workspace (#1521)
## Changes Not doing this meant file system traversal ended upon reaching a Git folder. By marking these objects as a directory globbing traverses into these folders as well. ## Tests Added a unit test for coverage. |
|
Gleb Kanterov
|
5ff06578ac
|
PythonMutator: replace stdin/stdout with files (#1512)
## Changes Replace stdin/stdout with files in `PythonMutator`. Files are created in a temporary directory. Rename `ApplyPythonMutator` to `PythonMutator`. Add test for `dyn.Location` behavior during the "load" stage. ## Tests Unit tests |
|
|
shreyas-goenka
|
068c7cfc2d
|
Return `dyn.InvalidValue` instead of `dyn.NilValue` when errors happen (#1514)
## Changes With https://github.com/databricks/cli/pull/1507 and https://github.com/databricks/cli/pull/1511 we are clarifying the semantics associated with `dyn.InvalidValue` and `dyn.NilValue`. An invalid value is the default zero value and is used to signals the complete absence of the value. A nil value, on the other hand, is a valid value for a piece of configuration and signals explicitly setting a key to nil in the configuration tree. In keeping with that theme, this PR returns `dyn.InvalidValue` instead of `dyn.NilValue` at error sites. This change is not expected to have a material change in behaviour and is being done to set the right convention since we have well-defined semantics associated with both `NilValue` and `InvalidValue`. ## Tests Unit tests and integration tests pass. Also manually scanned the changes and the associated call sites to verify the `NilValue` value itself was not being relied upon. |
|
|
Pieter Noordhuis
|
446a9d0c52
|
Properly deal with nil values in `convert.FromTyped` (#1511)
## Changes
When a configuration defines:
```yaml
run_as:
```
It first showed up as `run_as -> nil` in the dynamic configuration only
to later be converted to `run_as -> {}` while going through typed
conversion. We were using the presence of a key to initialize an empty
value. This is incorrect and it should have remained a nil value.
This conversion was happening in `convert.FromTyped` where any struct
always returned a map value. Instead, it should only return a map value
in any one of these cases: 1) the struct has elements, 2) the struct was
originally a map in the dynamic configuration, or 3) the struct was
initialized to a non-empty pointer value.
Stacked on top of #1516 and #1518.
## Tests
* Unit tests pass.
* Integration tests pass.
* Manually ran through bundle CRUD with a bundle without resources.
|
|
|
Pieter Noordhuis
|
87bc583819
|
Allow the any type to be set to nil in `convert.FromTyped` (#1518)
## Changes This came up in integration testing for #1511. One of the tests converted a `map[string]any` to a dynamic value and encountered a `nil` and errored out. We can safely return a nil in this case. ## Tests Unit test passes. |
|
|
Pieter Noordhuis
|
01adef666a
|
Set bool pointer to disable lock (#1516)
## Changes This cherry-picks from #1490 to address an issue that came up in #1511. The function `dyn.SetByPath` requires intermediate values to be present. If they are not, it returns an error that it cannot index a map. This is not an issue on main, where the intermediate maps are always created, even if they are not present in the dynamic configuration tree. As of #1511, we'll no longer populate empty maps for empty structs if they are not explicitly set (i.e., a non-nil pointer). This change writes a bool pointer to avoid this issue altogether. ## Tests Unit tests pass. |
|
|
Gleb Kanterov
|
57a5a65f87
|
Add ApplyPythonMutator (#1430)
## Changes
Add ApplyPythonMutator, which will fork the Python subprocess and
process pipe bundle configuration through it.
It's enabled through `experimental` section, for example:
```yaml
experimental:
pydabs:
enable: true
venv_path: .venv
```
For now, it's limited to two phases in the mutator pipeline:
- `load`: adds new jobs
- `init`: adds new jobs, or modifies existing ones
It's enforced that no jobs are modified in `load` and not jobs are
deleted in `load/init`, because, otherwise, it will break existing
assumptions.
## Tests
Unit tests
|
|
|
Pieter Noordhuis
|
b2c03ea54c
|
Use `dyn.InvalidValue` to indicate absence (#1507)
## Changes Previously, the functions `Get` and `Index` returned `dyn.NilValue` to indicate that a map key or sequence index wasn't found. This is a valid value, so we need to differentiate between actual absence and a real `dyn.NilValue`. We do this with the zero value of a `dyn.Value` (also captured in the constant `dyn.InvalidValue`). ## Tests * Unit tests. * Renamed `Get` and `Index` to find and update all call sites. |
|
Lennart Kats (databricks)
|
deb3e365cd
|
Pause quality monitors when "mode: development" is used (#1481)
## Changes Similar to scheduled jobs, quality monitors should be paused when in development mode (in line with the [behavior for scheduled jobs](https://docs.databricks.com/en/dev-tools/bundles/deployment-modes.html)). @aravind-segu @arpitjasa-db please take a look and verify this behavior. - [x] Followup: documentation changes. If we make this change we should update https://docs.databricks.com/dev-tools/bundles/deployment-modes.html. ## Tests Unit tests |
|
|
shreyas-goenka
|
cb4ab5007d
|
Add link to documentation for Homebrew installation to README (#1505)
Co-authored-by: Julia Crawford (Databricks) <julia.crawford@databricks.com> |
|
|
Andrew Nester
|
663aa9ab8c
|
Override variables with lookup value even if values has default value set (#1504)
## Changes This PR fixes the behaviour when variables were not overridden with lookup value from targets if these variables had any default value set in the default target. Fixes #1449 ## Tests Added regression test |
|
|
shreyas-goenka
|
553fdd1e81
|
Serialize dynamic value for `bundle validate` output (#1499)
## Changes
Using dynamic values allows us to retain references like
`${resources.jobs...}` even when the type of field is not integer, eg:
`run_job_task`, or in general values that do not map to the Go types for
a field.
## Tests
Integration test
|
|
|
shreyas-goenka
|
274688d8a2
|
Clean up unused code (#1502)
## Changes 1. Removes `DefaultMutatorsForTarget` which is no longer used anywhere 2. Makes SnapshotPath a private field. It's no longer needed by data structures outside its package. FYI, I also tried finding other instances of dead code but I could not find anything else that was safe to remove. I used https://go.dev/blog/deadcode to search for them, and the other instances either implemented an interface, increased test coverage for some of our other code paths or there was some other reason I could not remove them (like autogenerated functions or used in tests). Good sign our codebase is mostly clean (at least superficially). |
|
|
Pieter Noordhuis
|
533d357a71
|
Fix typo in DBT template (#1498)
## Changes Found in https://github.com/databricks/bundle-examples/pull/26. ## Tests n/a |
|
|
dependabot[bot]
|
8d658589ed
|
Bump github.com/spf13/cobra from 1.8.0 to 1.8.1 (#1496)
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.8.0 to 1.8.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/spf13/cobra/releases">github.com/spf13/cobra's releases</a>.</em></p> <blockquote> <h2>v1.8.1</h2> <h2>✨ Features</h2> <ul> <li>Add env variable to suppress completion descriptions on create by <a href="https://github.com/scop"><code>@scop</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/1938">spf13/cobra#1938</a></li> </ul> <h2>🐛 Bug fixes</h2> <ul> <li>Micro-optimizations by <a href="https://github.com/scop"><code>@scop</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/1957">spf13/cobra#1957</a></li> </ul> <h2>🔧 Maintenance</h2> <ul> <li>build(deps): bump github.com/cpuguy83/go-md2man/v2 from 2.0.3 to 2.0.4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2127">spf13/cobra#2127</a></li> <li>Consistent annotation names by <a href="https://github.com/nirs"><code>@nirs</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2140">spf13/cobra#2140</a></li> <li>Remove fully inactivated linters by <a href="https://github.com/nirs"><code>@nirs</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2148">spf13/cobra#2148</a></li> <li>Address golangci-lint deprecation warnings, enable some more linters by <a href="https://github.com/scop"><code>@scop</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2152">spf13/cobra#2152</a></li> </ul> <h2>🧪 Testing & CI/CD</h2> <ul> <li>Add test for func in cobra.go by <a href="https://github.com/korovindenis"><code>@korovindenis</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2094">spf13/cobra#2094</a></li> <li>ci: test golang 1.22 by <a href="https://github.com/cyrilico"><code>@cyrilico</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2113">spf13/cobra#2113</a></li> <li>Optimized and added more linting by <a href="https://github.com/scop"><code>@scop</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2099">spf13/cobra#2099</a></li> <li>build(deps): bump actions/setup-go from 4 to 5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2087">spf13/cobra#2087</a></li> <li>build(deps): bump actions/labeler from 4 to 5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2086">spf13/cobra#2086</a></li> <li>build(deps): bump golangci/golangci-lint-action from 3.7.0 to 4.0.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2108">spf13/cobra#2108</a></li> <li>build(deps): bump actions/cache from 3 to 4 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2102">spf13/cobra#2102</a></li> </ul> <h2>✏️ Documentation</h2> <ul> <li>Fixes and docs for usage as plugin by <a href="https://github.com/nirs"><code>@nirs</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2070">spf13/cobra#2070</a></li> <li>flags: clarify documentation that LocalFlags related function do not modify the state by <a href="https://github.com/niamster"><code>@niamster</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2064">spf13/cobra#2064</a></li> <li>chore: remove repetitive words by <a href="https://github.com/racerole"><code>@racerole</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2122">spf13/cobra#2122</a></li> <li>Add LXC to the list of projects using Cobra <a href="https://github.com/VaradBelwalkar"><code>@VaradBelwalkar</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2071">spf13/cobra#2071</a></li> <li>Update projects_using_cobra.md by <a href="https://github.com/marcuskohlberg"><code>@marcuskohlberg</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2089">spf13/cobra#2089</a></li> <li>[chore]: update projects using cobra by <a href="https://github.com/cmwylie19"><code>@cmwylie19</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2093">spf13/cobra#2093</a></li> <li>Add Taikun CLI to list of projects by <a href="https://github.com/Smidra"><code>@Smidra</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2098">spf13/cobra#2098</a></li> <li>Add Incus to the list of projects using Cobra by <a href="https://github.com/montag451"><code>@montag451</code></a> in <a href="https://redirect.github.com/spf13/cobra/pull/2118">spf13/cobra#2118</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
|
|
dependabot[bot]
|
4904dfb047
|
Bump github.com/briandowns/spinner from 1.23.0 to 1.23.1 (#1495)
Bumps [github.com/briandowns/spinner](https://github.com/briandowns/spinner) from 1.23.0 to 1.23.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/briandowns/spinner/releases">github.com/briandowns/spinner's releases</a>.</em></p> <blockquote> <h2>v1.23.1</h2> <h2>What's Changed</h2> <ul> <li>fix: CVE-2022-29526 by <a href="https://github.com/testwill"><code>@testwill</code></a> in <a href="https://redirect.github.com/briandowns/spinner/pull/152">briandowns/spinner#152</a></li> <li>Consolidate the dependencies for the IsTerminal() API by <a href="https://github.com/debarshiray"><code>@debarshiray</code></a> in <a href="https://redirect.github.com/briandowns/spinner/pull/156">briandowns/spinner#156</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/testwill"><code>@testwill</code></a> made their first contribution in <a href="https://redirect.github.com/briandowns/spinner/pull/152">briandowns/spinner#152</a></li> <li><a href="https://github.com/debarshiray"><code>@debarshiray</code></a> made their first contribution in <a href="https://redirect.github.com/briandowns/spinner/pull/156">briandowns/spinner#156</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/briandowns/spinner/compare/v1.23.0...v1.23.1">https://github.com/briandowns/spinner/compare/v1.23.0...v1.23.1</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
|
|
shreyas-goenka
|
ac6b80ed88
|
Remove user credentials specified in the Git origin URL (#1494)
## Changes We set the origin URL as metadata in any jobs created by DABs. This PR makes sure user credentials do not leak into the set metadata in the job. ## Tests Unit test --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> |
|
|
shreyas-goenka
|
44e3928d6a
|
Avoid multiple file tree traversals on bundle deploy (#1493)
## Changes To run bundle deploy from DBR we use an abstraction over the workspace import / export APIs to create a `filer.Filer` and abstract the file system. Walking the file tree in such a filer is expensive and requires multiple API calls. This PR remove the two duplicate file tree walks that happen by caching the result. |
|
|
dependabot[bot]
|
a5e89fd382
|
Bump golang.org/x/oauth2 from 0.20.0 to 0.21.0 (#1485)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from
0.20.0 to 0.21.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="
|
|
|
dependabot[bot]
|
645e9ba8c4
|
Bump golang.org/x/mod from 0.17.0 to 0.18.0 (#1484)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.17.0 to 0.18.0. <details> <summary>Commits</summary> <ul> <li><a href=" |
|
|
dependabot[bot]
|
3d3ab50ff9
|
Bump golang.org/x/term from 0.20.0 to 0.21.0 (#1483)
Bumps [golang.org/x/term](https://github.com/golang/term) from 0.20.0 to
0.21.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="
|
|
|
dependabot[bot]
|
b92e072add
|
Bump golang.org/x/text from 0.15.0 to 0.16.0 (#1482)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.15.0 to 0.16.0. <details> <summary>Commits</summary> <ul> <li><a href=" |
|
|
Lennart Kats (databricks)
|
99c7d136d6
|
Fix conditional in query in `default-sql` template (#1479)
## Changes This corrects a mistake in the sample SQL identified by @pietern |
|
Arpit Jasapara
|
35186d5ddb
|
Add randIntn function (#1475)
## Changes <!-- Summary of your changes that are easy to understand --> Add support for `math/rand.Intn` to DAB templates. ## Tests <!-- How is this tested? --> Unit tests. |
|
|
Pieter Noordhuis
|
8c9fff3cb9
|
Release v0.221.1 (#1478)
Bundles: This releases fixes an issue introduced in v0.221.0 where managing jobs with a single-node cluster would fail. * Fix SQL schema selection in default-sql template ([#1471](https://github.com/databricks/cli/pull/1471)). * Copy-editing for SQL templates ([#1474](https://github.com/databricks/cli/pull/1474)). * Upgrade TF provider to 1.47.0 ([#1476](https://github.com/databricks/cli/pull/1476)). Internal: * Use latest version of goreleaser action ([#1477](https://github.com/databricks/cli/pull/1477)). |
|
|
Pieter Noordhuis
|
1451361c9f
|
Use latest version of goreleaser action (#1477)
## Changes Same as https://github.com/databricks/terraform-provider-databricks/pull/3645. ## Tests n/a |
|
|
Pieter Noordhuis
|
311dfa4642
|
Upgrade TF provider to 1.47.0 (#1476)
## Changes This includes a bugfix for provisioning jobs with `num_workers = 0`. Fixes #1472. ## Tests Manually tested this fixes the issue. |
|
|
Lennart Kats (databricks)
|
41678fa695
|
Copy-editing for SQL templates (#1474)
## Changes This applies changes suggested by @juliacrawf-db |
|
|
Lennart Kats (databricks)
|
4bc0ea0af3
|
Fix SQL schema selection in default-sql template (#1471)
## Changes This fixes a last-minute regression that snuck into https://github.com/databricks/cli/pull/1463: unfortunately we need to use `USE IDENTIFIER('schema')` to select a schema for now. In the future we expect we can just use `USE SCHEMA 'schema'`. |
|
|
Pieter Noordhuis
|
f8b2cb89d5
|
Release v0.221.0 (#1470)
CLI: * Update OpenAPI spec ([#1466](https://github.com/databricks/cli/pull/1466)). Bundles: * Upgrade TF provider to 1.46.0 ([#1460](https://github.com/databricks/cli/pull/1460)). * Add support for Lakehouse monitoring ([#1307](https://github.com/databricks/cli/pull/1307)). * Make dbt-sql and default-sql templates public ([#1463](https://github.com/databricks/cli/pull/1463)). Internal: * Abstract over filesystem interaction with libs/vfs ([#1452](https://github.com/databricks/cli/pull/1452)). * Add `filer.Filer` to read notebooks from WSFS without omitting their extension ([#1457](https://github.com/databricks/cli/pull/1457)). * Fix listing notebooks in a subdirectory ([#1468](https://github.com/databricks/cli/pull/1468)). API Changes: * Changed `databricks account storage-credentials list` command to return . * Added `databricks consumer-listings batch-get` command. * Added `databricks consumer-providers batch-get` command. * Removed `databricks apps create-deployment` command. * Added `databricks apps deploy` command. OpenAPI commit 37b925eba37dfb3d7e05b6ba2d458454ce62d3a0 (2024-06-03) Dependency updates: * Bump github.com/hashicorp/go-version from 1.6.0 to 1.7.0 ([#1454](https://github.com/databricks/cli/pull/1454)). * Bump github.com/hashicorp/hc-install from 0.6.4 to 0.7.0 ([#1453](https://github.com/databricks/cli/pull/1453)). |
|
|
Pieter Noordhuis
|
448d41027d
|
Fix listing notebooks in a subdirectory (#1468)
## Changes This worked fine if the notebooks are located in the filer's root and didn't if they are nested in a directory. This change adds test coverage and fixes the underlying issue. ## Tests Ran integration test manually. |
|
|
Lennart Kats (databricks)
|
aa36aee159
|
Make dbt-sql and default-sql templates public (#1463)
## Changes This makes the dbt-sql and default-sql templates public. These templates were previously not listed and marked "experimental" since structured streaming tables were still in gated preview and would result in weird error messages when a workspace wasn't enabled for the preview. This PR also incorporates some of the feedback and learnings for these templates so far. |
|
|
Pieter Noordhuis
|
70fd8ad3d7
|
Update OpenAPI spec (#1466)
## Changes Notable changes: * Pagination of account-level storage credentials * Rename app deployment method Go SDK release notes: https://github.com/databricks/databricks-sdk-go/releases/tag/v0.42.0 ## Tests * Nightlies pass. |
|
|
Pieter Noordhuis
|
c9b4f11947
|
Update error checks that use the `os` package to use `errors.Is` (#1461)
## Changes From the [documentation](https://pkg.go.dev/os#IsNotExist) on the functions in the `os` package: > This function predates errors.Is. It only supports errors returned by the os package. > New code should use errors.Is(err, fs.ErrNotExist). This issue surfaced while working on using a different `vfs.Path` implementation that uses errors from the `fs` package. Calls to `os.IsNotExist` didn't return true for errors that wrap `fs.ErrNotExist`. ## Tests n/a |
|
|
Pieter Noordhuis
|
30fd84893f
|
Generate bundle schema placeholder for quality monitors (#1465)
## Changes Generated with default generation command. The team is making a fix to ensure the proper comments are included later. ## Tests n/a |
|
Aravind Segu
|
a33d0c8bf9
|
Add support for Lakehouse monitoring in bundles (#1307)
## Changes This change adds support for Lakehouse monitoring in bundles. The associated resource type name is "quality monitor". ## Testing Unit tests. --------- Co-authored-by: Pieter Noordhuis <pcnoordhuis@gmail.com> Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> Co-authored-by: Arpit Jasapara <87999496+arpitjasa-db@users.noreply.github.com> |
|
|
Pieter Noordhuis
|
364a609ea7
|
Upgrade TF provider to 1.46.0 (#1460)
## Changes Release notes in https://github.com/databricks/terraform-provider-databricks/releases/tag/v1.46.0 Notable changes since 1.43.0: * The job resource has been migrated to the Go SDK. More fields are now passed through from DABs into TF. * Improved zero-value handling. ## Tests n/a |
|
|
shreyas-goenka
|
ec33a7c059
|
Add `filer.Filer` to read notebooks from WSFS without omitting their extension (#1457)
## Changes This PR adds a filer that'll allow us to read notebooks from the WSFS using their full paths (with the extension included). The filer relies on the existing workspace filer (and consequently the workspace import/export/list APIs). Using this filer along with a virtual filesystem layer (https://github.com/databricks/cli/pull/1452/files) will allow us to use our custom implementation (which preserves the notebook extensions) rather than the default mount available via DBR when the CLI is run from DBR. ## Tests Integration tests. --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> |
|
|
Pieter Noordhuis
|
424499ec1d
|
Abstract over filesystem interaction with libs/vfs (#1452)
## Changes Introduce `libs/vfs` for an implementation of `fs.FS` and friends that _includes_ the absolute path it is anchored to. This is needed for: 1. Intercepting file operations to inject custom logic (e.g., logging, access control). 2. Traversing directories to find specific leaf directories (e.g., `.git`). 3. Converting virtual paths to OS-native paths. Options 2 and 3 are not possible with the standard `fs.FS` interface. They are needed such that we can provide an instance to the sync package and still detect the containing `.git` directory and convert paths to native paths. This change focuses on making the following packages use `vfs.Path`: * libs/fileset * libs/git * libs/sync All entries returned by `fileset.All` are now slash-separated. This has 2 consequences: * The sync snapshot now always uses slash-separated paths * We don't need to call `filepath.FromSlash` as much as we did ## Tests * All unit tests pass * All integration tests pass * Manually confirmed that a deployment made on Windows by a previous version of the CLI can be deployed by a new version of the CLI while retaining the validity of the local sync snapshot as well as the remote deployment state. |
|
|
Pieter Noordhuis
|
b2ea9dd971
|
Remove unnecessary `filepath.FromSlash` calls (#1458)
## Changes The prior join call calls `filepath.Join` which returns a cleaned result. Path cleaning, in turn, calls `filepath.FromSlash`. ## Tests * Unit tests. |
|
|
dependabot[bot]
|
13b937cea8
|
Bump github.com/hashicorp/hc-install from 0.6.4 to 0.7.0 (#1453)
Bumps [github.com/hashicorp/hc-install](https://github.com/hashicorp/hc-install) from 0.6.4 to 0.7.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/hashicorp/hc-install/releases">github.com/hashicorp/hc-install's releases</a>.</em></p> <blockquote> <h2>v0.7.0</h2> <p>ENHANCEMENTS:</p> <ul> <li>cmd/hc-install: Enable logging into a file by <a href="https://github.com/radeksimko"><code>@radeksimko</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/193">hashicorp/hc-install#193</a></li> <li>Ensure license file gets packaged along w/ the CLI binary by <a href="https://github.com/radeksimko"><code>@radeksimko</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/205">hashicorp/hc-install#205</a></li> <li>Ensure license files are tracked during installation so they can be removed by <a href="https://github.com/radeksimko"><code>@radeksimko</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/206">hashicorp/hc-install#206</a></li> <li>Add support for custom download URLs by <a href="https://github.com/james0209"><code>@james0209</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/203">hashicorp/hc-install#203</a></li> </ul> <p>BUG FIXES:</p> <ul> <li>releasesjson: Fix missing <code>Version</code> in unmarshaled response by <a href="https://github.com/radeksimko"><code>@radeksimko</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/192">hashicorp/hc-install#192</a></li> </ul> <p>DEPENDENCIES:</p> <ul> <li>build(deps): bump golang.org/x/mod from 0.16.0 to 0.17.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/194">hashicorp/hc-install#194</a></li> <li>build(deps): bump golang.org/x/net from 0.22.0 to 0.23.0 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/195">hashicorp/hc-install#195</a></li> <li>build(deps): bump hashicorp/action-setup-bob from 1.0.20 to 2 in the github-actions-breaking group by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/208">hashicorp/hc-install#208</a></li> <li>build(deps): bump hashicorp/setup-copywrite from 1.1.2 to 1.1.3 in the github-actions-backward-compatible group by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/212">hashicorp/hc-install#212</a></li> <li>build(deps): bump the github-actions-breaking group with 2 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/211">hashicorp/hc-install#211</a></li> <li>build(deps): Bump workflows to latest trusted versions by <a href="https://github.com/hashicorp-tsccr"><code>@hashicorp-tsccr</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/199">hashicorp/hc-install#199</a></li> <li>build(deps): Bump workflows to latest trusted versions by <a href="https://github.com/hashicorp-tsccr"><code>@hashicorp-tsccr</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/204">hashicorp/hc-install#204</a></li> <li>build(deps): Bump workflows to latest trusted versions by <a href="https://github.com/hashicorp-tsccr"><code>@hashicorp-tsccr</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/209">hashicorp/hc-install#209</a></li> </ul> <p>INTERNAL:</p> <ul> <li>[chore] Fix minor lint issue by <a href="https://github.com/james0209"><code>@james0209</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/202">hashicorp/hc-install#202</a></li> <li>github: Create CODEOWNERS by <a href="https://github.com/xiehan"><code>@xiehan</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/210">hashicorp/hc-install#210</a></li> <li>github: Set up Dependabot to manage HashiCorp-owned Actions versioning by <a href="https://github.com/xiehan"><code>@xiehan</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/207">hashicorp/hc-install#207</a></li> <li>Lint Markdown by <a href="https://github.com/james0209"><code>@james0209</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/200">hashicorp/hc-install#200</a></li> <li>Replace deprecated <code>ioutil</code> usage with the equivalent replacements by <a href="https://github.com/james0209"><code>@james0209</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/198">hashicorp/hc-install#198</a></li> <li>Support darwin/arm64 in tests by <a href="https://github.com/radeksimko"><code>@radeksimko</code></a> in <a href="https://redirect.github.com/hashicorp/hc-install/pull/196">hashicorp/hc-install#196</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/james0209"><code>@james0209</code></a> made their first contribution in <a href="https://redirect.github.com/hashicorp/hc-install/pull/198">hashicorp/hc-install#198</a></li> <li><a href="https://github.com/xiehan"><code>@xiehan</code></a> made their first contribution in <a href="https://redirect.github.com/hashicorp/hc-install/pull/207">hashicorp/hc-install#207</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/hashicorp/hc-install/compare/v0.6.4...v0.7.0">https://github.com/hashicorp/hc-install/compare/v0.6.4...v0.7.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
|
|
dependabot[bot]
|
9a452f38ee
|
Bump github.com/hashicorp/go-version from 1.6.0 to 1.7.0 (#1454)
Bumps [github.com/hashicorp/go-version](https://github.com/hashicorp/go-version) from 1.6.0 to 1.7.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/hashicorp/go-version/releases">github.com/hashicorp/go-version's releases</a>.</em></p> <blockquote> <h2>v1.7.0</h2> <p>ENHANCEMENTS:</p> <ul> <li>Remove <code>reflect</code> dependency (<a href="https://redirect.github.com/hashicorp/go-version/pull/91">#91</a>)</li> <li>Implement the <code>database/sql.Scanner</code> and <code>database/sql/driver.Value</code> interfaces for <code>Version</code> (<a href="https://redirect.github.com/hashicorp/go-version/pull/133">#133</a>)</li> </ul> <p>INTERNAL:</p> <ul> <li>[COMPLIANCE] Add Copyright and License Headers (<a href="https://redirect.github.com/hashicorp/go-version/pull/115">#115</a>)</li> <li>[COMPLIANCE] Update MPL-2.0 LICENSE (<a href="https://redirect.github.com/hashicorp/go-version/pull/105">#105</a>)</li> <li>Bump actions/cache from 3.0.11 to 3.2.5 (<a href="https://redirect.github.com/hashicorp/go-version/pull/116">#116</a>)</li> <li>Bump actions/checkout from 3.2.0 to 3.3.0 (<a href="https://redirect.github.com/hashicorp/go-version/pull/111">#111</a>)</li> <li>Bump actions/upload-artifact from 3.1.1 to 3.1.2 (<a href="https://redirect.github.com/hashicorp/go-version/pull/112">#112</a>)</li> <li>GHA Migration (<a href="https://redirect.github.com/hashicorp/go-version/pull/103">#103</a>)</li> <li>github: Pin external GitHub Actions to hashes (<a href="https://redirect.github.com/hashicorp/go-version/pull/107">#107</a>)</li> <li>SEC-090: Automated trusted workflow pinning (2023-04-05) (<a href="https://redirect.github.com/hashicorp/go-version/pull/124">#124</a>)</li> <li>update readme (<a href="https://redirect.github.com/hashicorp/go-version/pull/104">#104</a>)</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/hashicorp/go-version/blob/main/CHANGELOG.md">github.com/hashicorp/go-version's changelog</a>.</em></p> <blockquote> <h1>1.7.0 (May 24, 2024)</h1> <p>ENHANCEMENTS:</p> <ul> <li>Remove <code>reflect</code> dependency (<a href="https://redirect.github.com/hashicorp/go-version/pull/91">#91</a>)</li> <li>Implement the <code>database/sql.Scanner</code> and <code>database/sql/driver.Value</code> interfaces for <code>Version</code> (<a href="https://redirect.github.com/hashicorp/go-version/pull/133">#133</a>)</li> </ul> <p>INTERNAL:</p> <ul> <li>[COMPLIANCE] Add Copyright and License Headers (<a href="https://redirect.github.com/hashicorp/go-version/pull/115">#115</a>)</li> <li>[COMPLIANCE] Update MPL-2.0 LICENSE (<a href="https://redirect.github.com/hashicorp/go-version/pull/105">#105</a>)</li> <li>Bump actions/cache from 3.0.11 to 3.2.5 (<a href="https://redirect.github.com/hashicorp/go-version/pull/116">#116</a>)</li> <li>Bump actions/checkout from 3.2.0 to 3.3.0 (<a href="https://redirect.github.com/hashicorp/go-version/pull/111">#111</a>)</li> <li>Bump actions/upload-artifact from 3.1.1 to 3.1.2 (<a href="https://redirect.github.com/hashicorp/go-version/pull/112">#112</a>)</li> <li>GHA Migration (<a href="https://redirect.github.com/hashicorp/go-version/pull/103">#103</a>)</li> <li>github: Pin external GitHub Actions to hashes (<a href="https://redirect.github.com/hashicorp/go-version/pull/107">#107</a>)</li> <li>SEC-090: Automated trusted workflow pinning (2023-04-05) (<a href="https://redirect.github.com/hashicorp/go-version/pull/124">#124</a>)</li> <li>update readme (<a href="https://redirect.github.com/hashicorp/go-version/pull/104">#104</a>)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
|
|
Pieter Noordhuis
|
46f6cbcfc3
|
Release v0.220.0 (#1446)
CLI: * Add line about Docker installation to README.md ([#1363](https://github.com/databricks/cli/pull/1363)). * Improve token refresh flow ([#1434](https://github.com/databricks/cli/pull/1434)). Bundles: * Upgrade Terraform provider to v1.42.0 ([#1418](https://github.com/databricks/cli/pull/1418)). * Upgrade Terraform provider to v1.43.0 ([#1429](https://github.com/databricks/cli/pull/1429)). * Don't merge-in remote resources during deployments ([#1432](https://github.com/databricks/cli/pull/1432)). * Remove dependency on `ConfigFilePath` from path translation mutator ([#1437](https://github.com/databricks/cli/pull/1437)). * Add `merge.Override` transform ([#1428](https://github.com/databricks/cli/pull/1428)). * Fixed panic when loading incorrectly defined jobs ([#1402](https://github.com/databricks/cli/pull/1402)). * Add more tests for `merge.Override` ([#1439](https://github.com/databricks/cli/pull/1439)). * Fixed seg fault when specifying environment key for tasks ([#1443](https://github.com/databricks/cli/pull/1443)). * Fix conversion of zero valued scalar pointers to a dynamic value ([#1433](https://github.com/databricks/cli/pull/1433)). Internal: * Don't hide commands of services that are already hidden ([#1438](https://github.com/databricks/cli/pull/1438)). API Changes: * Renamed `lakehouse-monitors` command group to `quality-monitors`. * Added `apps` command group. * Renamed `csp-enablement` command group to `compliance-security-profile`. * Renamed `esm-enablement` command group to `enhanced-security-monitoring`. * Added `databricks vector-search-indexes scan-index` command. OpenAPI commit 7eb5ad9a2ed3e3f1055968a2d1014ac92c06fe92 (2024-05-21) Dependency updates: * Bump golang.org/x/text from 0.14.0 to 0.15.0 ([#1419](https://github.com/databricks/cli/pull/1419)). * Bump golang.org/x/oauth2 from 0.19.0 to 0.20.0 ([#1421](https://github.com/databricks/cli/pull/1421)). * Bump golang.org/x/term from 0.19.0 to 0.20.0 ([#1422](https://github.com/databricks/cli/pull/1422)). * Bump github.com/databricks/databricks-sdk-go from 0.39.0 to 0.40.1 ([#1431](https://github.com/databricks/cli/pull/1431)). * Bump github.com/fatih/color from 1.16.0 to 1.17.0 ([#1441](https://github.com/databricks/cli/pull/1441)). * Bump github.com/hashicorp/terraform-json from 0.21.0 to 0.22.1 ([#1440](https://github.com/databricks/cli/pull/1440)). * Bump github.com/hashicorp/terraform-exec from 0.20.0 to 0.21.0 ([#1442](https://github.com/databricks/cli/pull/1442)). * Update Go SDK to v0.41.0 ([#1445](https://github.com/databricks/cli/pull/1445)). |