f9e521b43e
databricks bundle init template v2: optional stubs, DLT support ( #700 )
...
## Changes
This follows up on https://github.com/databricks/cli/pull/686 . This PR
makes our stubs optional + it adds DLT stubs:
```
$ databricks bundle init
Template to use [default-python]: default-python
Unique name for this project [my_project]: my_project
Include a stub (sample) notebook in 'my_project/src' [yes]: yes
Include a stub (sample) DLT pipeline in 'my_project/src' [yes]: yes
Include a stub (sample) Python package 'my_project/src' [yes]: yes
✨ Successfully initialized template
```
## Tests
Manual testing, matrix tests.
---------
Co-authored-by: Andrew Nester <andrew.nester@databricks.com>
Co-authored-by: PaulCornellDB <paul.cornell@databricks.com>
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2023-09-06 09:52:31 +00:00
fabe8e88b8
Include $PATH in set of environment variables to pass along. ( #736 )
...
## Changes
This is necessary to ensure that our Terraform provider can use the same
auxiliary programs (e.g. `az`, or `gcloud`) as the CLI.
## Tests
Unit test and manual verification.
2023-09-06 07:54:35 +00:00
46b999ed42
Pin Terraform binary version to 1.5.5 ( #715 )
...
## Changes
The installer doesn't respect the version constraints if they are
specified.
Source: [the vc argument is not
used](850464c601/releases/latest_version.go (L158-L177)
2023-08-30 14:08:37 +00:00
56dcd3f0a7
Renamed `environments` to `targets` in bundle configuration ( #670 )
...
## Changes
Renamed Environments to Targets in bundle.yml.
The change is backward-compatible and customers can continue to use
`environments` in the time being.
## Tests
Added tests which checks that both `environments` and `targets` sections
in bundle.yml works correctly
2023-08-17 15:22:32 +00:00
61b103318f
Use custom prompter for bundle template inputs ( #663 )
...
## Changes
Prompt UI glitches often. We are switching to a custom implementation of
a simple prompter which is much more stable.
This also allows new lines in prompts which has been an ask by the
mlflow team.
## Tests
Tested manually
2023-08-15 14:50:20 +00:00
5cdaacacc3
Locked terraform binary version to <= 1.5.5 ( #666 )
...
## Changes
Locked terraform binary version to <= 1.5.5
2023-08-15 13:39:32 +00:00
6430d23453
Print y/n options when displaying prompts using cmdio.Ask ( #650 )
...
## Changes
Adds `[y/n]` in `cmdio.Ask` to make the options obvious in all question
prompts
## Tests
Test manually. Works.
2023-08-09 09:22:42 +00:00
65d8fe13e9
Bump github.com/databricks/databricks-sdk-go from 0.12.0 to 0.13.0 ( #585 )
...
Bumps
[github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go )
from 0.12.0 to 0.13.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/releases ">github.com/databricks/databricks-sdk-go's
releases</a>.</em></p>
<blockquote>
<h2>v0.13.0</h2>
<ul>
<li>Add issue templates (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/539 ">#539</a>).</li>
<li>Added HasRequiredNonBodyField method (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/536 ">#536</a>).</li>
<li>Make Azure MSI auth account compatible (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/544 ">#544</a>).</li>
<li>Refactor Handling of Name<!-- raw HTML omitted -->ID Mapping in
OpenAPI Generator (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/547 ">#547</a>).</li>
<li>Regenerate Go SDK from current OpenAPI Specification (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/549 ">#549</a>).</li>
<li>Parse Camel Case and Pascal Case Enum Values (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/550 ">#550</a>).</li>
<li>Prepare for auto-releaser infra (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/554 ">#554</a>).</li>
<li>Added SCIM Patch Acceptance Tests (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/540 ">#540</a>).</li>
</ul>
<p>API Changes:</p>
<ul>
<li>Removed <code>Maintenance</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#MetastoresAPI ">w.Metastores</a>
workspace-level service.</li>
<li>Added <code>EnableOptimization</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#MetastoresAPI ">w.Metastores</a>
workspace-level service.</li>
<li>Added <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#TablesAPI ">w.Tables</a>
workspace-level service.</li>
<li>Added <code>Force</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#DeleteAccountMetastoreRequest ">catalog.DeleteAccountMetastoreRequest</a>.</li>
<li>Added <code>Force</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#DeleteAccountStorageCredentialRequest ">catalog.DeleteAccountStorageCredentialRequest</a>.</li>
<li>Removed <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateAutoMaintenance ">catalog.UpdateAutoMaintenance</a>.</li>
<li>Removed <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateAutoMaintenanceResponse ">catalog.UpdateAutoMaintenanceResponse</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdatePredictiveOptimization ">catalog.UpdatePredictiveOptimization</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdatePredictiveOptimizationResponse ">catalog.UpdatePredictiveOptimizationResponse</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateTableRequest ">catalog.UpdateTableRequest</a>.</li>
<li>Added <code>Schema</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/iam#PartialUpdate ">iam.PartialUpdate</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/iam#PatchSchema ">iam.PatchSchema</a>.</li>
<li>Added <code>TriggerInfo</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#BaseRun ">jobs.BaseRun</a>.</li>
<li>Added <code>Health</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#CreateJob ">jobs.CreateJob</a>.</li>
<li>Added <code>JobSource</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#GitSource ">jobs.GitSource</a>.</li>
<li>Added <code>OnDurationWarningThresholdExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobEmailNotifications ">jobs.JobEmailNotifications</a>.</li>
<li>Added <code>Health</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobSettings ">jobs.JobSettings</a>.</li>
<li>Added <code>TriggerInfo</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Run ">jobs.Run</a>.</li>
<li>Added <code>RunJobOutput</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunOutput ">jobs.RunOutput</a>.</li>
<li>Added <code>RunJobTask</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunTask ">jobs.RunTask</a>.</li>
<li>Added <code>EmailNotifications</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun ">jobs.SubmitRun</a>.</li>
<li>Added <code>Health</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun ">jobs.SubmitRun</a>.</li>
<li>Added <code>EmailNotifications</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask ">jobs.SubmitTask</a>.</li>
<li>Added <code>Health</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask ">jobs.SubmitTask</a>.</li>
<li>Added <code>NotificationSettings</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask ">jobs.SubmitTask</a>.</li>
<li>Added <code>Health</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Task ">jobs.Task</a>.</li>
<li>Added <code>RunJobTask</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Task ">jobs.Task</a>.</li>
<li>Added <code>OnDurationWarningThresholdExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TaskEmailNotifications ">jobs.TaskEmailNotifications</a>.</li>
<li>Added <code>OnDurationWarningThresholdExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#WebhookNotifications ">jobs.WebhookNotifications</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobSource ">jobs.JobSource</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobSourceDirtyState ">jobs.JobSourceDirtyState</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobsHealthMetric ">jobs.JobsHealthMetric</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobsHealthOperator ">jobs.JobsHealthOperator</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobsHealthRule ">jobs.JobsHealthRule</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobsHealthRules ">jobs.JobsHealthRules</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunJobOutput ">jobs.RunJobOutput</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunJobTask ">jobs.RunJobTask</a>.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-go/blob/main/CHANGELOG.md ">github.com/databricks/databricks-sdk-go's
changelog</a>.</em></p>
<blockquote>
<h2>0.13.0</h2>
<ul>
<li>Add issue templates (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/539 ">#539</a>).</li>
<li>Added HasRequiredNonBodyField method (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/536 ">#536</a>).</li>
<li>Make Azure MSI auth account compatible (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/544 ">#544</a>).</li>
<li>Refactor Handling of Name<!-- raw HTML omitted -->ID Mapping in
OpenAPI Generator (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/547 ">#547</a>).</li>
<li>Regenerate Go SDK from current OpenAPI Specification (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/549 ">#549</a>).</li>
<li>Parse Camel Case and Pascal Case Enum Values (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/550 ">#550</a>).</li>
<li>Prepare for auto-releaser infra (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/554 ">#554</a>).</li>
<li>Added SCIM Patch Acceptance Tests (<a
href="https://redirect.github.com/databricks/databricks-sdk-go/pull/540 ">#540</a>).</li>
</ul>
<p>API Changes:</p>
<ul>
<li>Removed <code>Maintenance</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#MetastoresAPI ">w.Metastores</a>
workspace-level service.</li>
<li>Added <code>EnableOptimization</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#MetastoresAPI ">w.Metastores</a>
workspace-level service.</li>
<li>Added <code>Update</code> method for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#TablesAPI ">w.Tables</a>
workspace-level service.</li>
<li>Added <code>Force</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#DeleteAccountMetastoreRequest ">catalog.DeleteAccountMetastoreRequest</a>.</li>
<li>Added <code>Force</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#DeleteAccountStorageCredentialRequest ">catalog.DeleteAccountStorageCredentialRequest</a>.</li>
<li>Removed <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateAutoMaintenance ">catalog.UpdateAutoMaintenance</a>.</li>
<li>Removed <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateAutoMaintenanceResponse ">catalog.UpdateAutoMaintenanceResponse</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdatePredictiveOptimization ">catalog.UpdatePredictiveOptimization</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdatePredictiveOptimizationResponse ">catalog.UpdatePredictiveOptimizationResponse</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#UpdateTableRequest ">catalog.UpdateTableRequest</a>.</li>
<li>Added <code>Schema</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/iam#PartialUpdate ">iam.PartialUpdate</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/iam#PatchSchema ">iam.PatchSchema</a>.</li>
<li>Added <code>TriggerInfo</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#BaseRun ">jobs.BaseRun</a>.</li>
<li>Added <code>Health</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#CreateJob ">jobs.CreateJob</a>.</li>
<li>Added <code>JobSource</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#GitSource ">jobs.GitSource</a>.</li>
<li>Added <code>OnDurationWarningThresholdExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobEmailNotifications ">jobs.JobEmailNotifications</a>.</li>
<li>Added <code>Health</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobSettings ">jobs.JobSettings</a>.</li>
<li>Added <code>TriggerInfo</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Run ">jobs.Run</a>.</li>
<li>Added <code>RunJobOutput</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunOutput ">jobs.RunOutput</a>.</li>
<li>Added <code>RunJobTask</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunTask ">jobs.RunTask</a>.</li>
<li>Added <code>EmailNotifications</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun ">jobs.SubmitRun</a>.</li>
<li>Added <code>Health</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitRun ">jobs.SubmitRun</a>.</li>
<li>Added <code>EmailNotifications</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask ">jobs.SubmitTask</a>.</li>
<li>Added <code>Health</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask ">jobs.SubmitTask</a>.</li>
<li>Added <code>NotificationSettings</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#SubmitTask ">jobs.SubmitTask</a>.</li>
<li>Added <code>Health</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Task ">jobs.Task</a>.</li>
<li>Added <code>RunJobTask</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#Task ">jobs.Task</a>.</li>
<li>Added <code>OnDurationWarningThresholdExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#TaskEmailNotifications ">jobs.TaskEmailNotifications</a>.</li>
<li>Added <code>OnDurationWarningThresholdExceeded</code> field for <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#WebhookNotifications ">jobs.WebhookNotifications</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobSource ">jobs.JobSource</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobSourceDirtyState ">jobs.JobSourceDirtyState</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobsHealthMetric ">jobs.JobsHealthMetric</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobsHealthOperator ">jobs.JobsHealthOperator</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobsHealthRule ">jobs.JobsHealthRule</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#JobsHealthRules ">jobs.JobsHealthRules</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunJobOutput ">jobs.RunJobOutput</a>.</li>
<li>Added <a
href="https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/jobs#RunJobTask ">jobs.RunJobTask</a>.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b4fb746b3bhttps://redirect.github.com/databricks/databricks-sdk-go/issues/555 ">#555</a>)</li>
<li><a
href="180c7eea4chttps://redirect.github.com/databricks/databricks-sdk-go/issues/540 ">#540</a>)</li>
<li><a
href="546814a272https://redirect.github.com/databricks/databricks-sdk-go/issues/554 ">#554</a>)</li>
<li><a
href="7e680c5ba8https://redirect.github.com/databricks/databricks-sdk-go/issues/550 ">#550</a>)</li>
<li><a
href="e71ece4ccdhttps://redirect.github.com/databricks/databricks-sdk-go/issues/551 ">#551</a>)</li>
<li><a
href="3b4492b6d6https://redirect.github.com/databricks/databricks-sdk-go/issues/549 ">#549</a>)</li>
<li><a
href="f84de6111ahttps://redirect.github.com/databricks/databricks-sdk-go/issues/547 ">#547</a>)</li>
<li><a
href="c37a894872https://redirect.github.com/databricks/databricks-sdk-go/issues/542 ">#542</a>)</li>
<li><a
href="4f2aa38e75https://redirect.github.com/databricks/databricks-sdk-go/issues/541 ">#541</a>)</li>
<li><a
href="e80f6e16ffhttps://redirect.github.com/databricks/databricks-sdk-go/issues/544 ">#544</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/databricks/databricks-sdk-go/compare/v0.12.0...v0.13.0 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Serge Smertin <serge.smertin@databricks.com>
2023-07-18 15:30:00 +00:00
179154477e
Propagate TF_CLI_CONFIG_FILE env variable ( #555 )
...
## Changes
Propagate `TF_CLI_CONFIG_FILE` env variable.
From Terraform documentation:
> The location of the Terraform CLI configuration file can also be
specified using the TF_CLI_CONFIG_FILE [environment
variable](https://developer.hashicorp.com/terraform/cli/config/environment-variables )
It allows using custom builds of terraform-provider-databricks, using
config files like:
```tf
provider_installation {
dev_overrides {
"databricks/databricks" = "/Users/gleb.kanterov/terraform-provider-databricks"
}
direct {}
}
```
## Tests
I added unit tests.
2023-07-07 13:20:37 +02:00
ad8183d7a9
Bump Go SDK to v0.12.0 ( #540 )
...
## Changes
* Regenerate CLI commands
* Ignore `account-access-control-proxy` (see #505 )
## Tests
Unit and integration tests pass.
2023-07-03 11:46:45 +02:00
1875908b59
Pass through proxy related environment variables ( #465 )
...
## Changes
If set on the host, we must pass them through to Terraform.
## Tests
Unit tests pass.
2023-06-14 21:58:26 +02:00
349e2aff40
Allow equivalence checking of filer errors to fs errors ( #416 )
...
## Changes
The pattern `errors.Is(err, fs.ErrNotExist)` is common to check for an
error type.
Errors can implement `Is(error) bool` with a custom equivalence checker.
## Tests
New asserts all pass in the integration test.
2023-05-31 20:47:00 +02:00
e4ab455ea1
Don't pass synthesized TMPDIR if not already set ( #409 )
...
## Changes
On Unix systems, the default of `/tmp` always works. No need to
synthesize a path for it.
The custom TMPDIR was causing issues when used from GitHub Actions
runners.
## Tests
Confirmed manually this fixes the issue on GitHub Actions runners.
2023-05-26 13:05:30 +02:00
6141476ca2
Added support for bundle.Seq, simplified Mutator.Apply interface ( #403 )
...
## Changes
Added support for `bundle.Seq`, simplified `Mutator.Apply` interface by
removing list of mutators from return values/
## Tests
1. Ran `cli bundle deploy` and interrupted it with Cmd + C mid execution
so lock is not released
2. Ran `cli bundle deploy` top make sure that CLI is not trying to
release lock when it fail to acquire it
```
andrew.nester@HFW9Y94129 multiples-tasks % cli bundle deploy
Starting upload of bundle files
Uploaded bundle files at /Users/andrew.nester@databricks.com/.bundle/simple-task/development/files!
^C
andrew.nester@HFW9Y94129 multiples-tasks % cli bundle deploy
Error: deploy lock acquired by andrew.nester@databricks.com at 2023-05-24 12:10:23.050343 +0200 CEST. Use --force to override
```
2023-05-24 14:45:19 +02:00
c53ad860e6
Create tmp files in the cache dir in terraform command runs ( #395 )
...
## Changes
Passes through tmp dir related env vars to the terraform process. Incase
any of them are not set, we assign temp dir inside bundle cache dir as
the location terraform should use.
## Tests
Manually checked that these env vars do override location where
os.CreateTemp files are created
2023-05-23 13:51:15 +02:00
98ebb78c9b
Rename bricks -> databricks ( #389 )
...
## Changes
Rename all instances of "bricks" to "databricks".
## Tests
* Confirmed the goreleaser build works, uses the correct new binary
name, and produces the right archives.
* Help output is confirmed to be correct.
* Output of `git grep -w bricks` is minimal with a couple changes
remaining for after the repository rename.
2023-05-16 18:35:39 +02:00
33fb0b3c40
Do not truncate local state file when pulling remote changes ( #382 )
...
## Changes
When local state file exists it won't be override by remote state file
## Tests
Running `bricks bundle deploy` after state push failed does not override
local state file
Use cases verified:
1. Local state file is newer than remote
2. Local state file is older than remote
3. Local state file does not exist
4. Local state file corrupted
2023-05-16 17:02:33 +02:00
473d2bf503
Improved error message when 'bricks bundle run' is executed before 'bricks bundle deploy' ( #378 )
...
## Changes
Improved error message when 'bricks bundle run' is executed before
'bricks bundle deploy'
The error happens when we attempt to load terraform state when it does
not exist.
The best way to check if terraform state actually exists is to call
`terraform show -json` and that's what already happens here
https://github.com/databricks/bricks/compare/main...error-before-deploy#diff-8c50f8c04e568397bc865b7e02d1f4ec5b18379d8d32daddfeb041035d804f5fL28
Absence of `state.Values` indicates that there is no state and likely
bundle was just never deployed.
## Tests
Ran `bricks bundle run test_job` on a new non-deployed bundle.
**Output:**
`Error: terraform show: No state. Did you forget to run 'bricks bundle
deploy'?`
Running `bricks bundle deploy && bricks bundle run test_job` succeeds.
---------
Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2023-05-10 11:02:25 +02:00
9581187c9e
Update to Go SDK v0.8.0 ( #351 )
...
## Changes
- Update to Go SDK v0.8.0
- Fix all breaking changes
## Tests
- make test
2023-04-21 10:30:20 +02:00
598ad62688
Log mutator messages using progress logger ( #312 )
...
This PR uses progress logger to log messages inside mutators
2023-04-18 16:55:06 +02:00
85889dffb1
Move state to event for whether they support inplace progress logging ( #339 )
...
## Changes
Adds a IsInplaceSupported() function to the event interface. Any event
that now uses the progress logger has to declare whether they support in
place logging
## Tests
Manually
2023-04-18 14:20:35 +02:00
b388f4a0dc
Make all workspace paths string fields ( #327 )
...
## Changes
These are unlikely to ever be DBFS paths so we can remove this level of indirection to simplify.
**Note:** this is a breaking change. Downstream usage of these fields must be updated.
## Tests
Existing tests pass.
2023-04-12 16:54:36 +02:00
42d29f92c9
Pass through $HOME when invoking Terraform ( #319 )
...
## Changes
This is useful when developing the Databricks Terraform provider where
you keep a local-only build of the provider and refer to it using $HOME
from `~/.terraformrc`, for example like this:
```
plugin_cache_dir = "$HOME/.terraform.d/plugin-cache"
```
## Tests
That $HOME is passed through cannot be tested as is because the
`tfexec.Terraform` struct doesn't expose it through public fields or
methods. What can be tested is a successful run of the initialize
mutator and this is included in this commit.
2023-04-11 13:11:31 +02:00
4871f7bc8a
Add bundle destroy command ( #300 )
...
Adds bundle destroy capability to bricks
2023-04-06 12:54:58 +02:00
d7ac265536
Allow use of file library in pipeline ( #308 )
...
## Changes
This requires databricks/databricks-sdk-go#359 .
## Tests
Tests pass and ran manual verification of deployment with files.
2023-04-05 16:29:42 +02:00
04e77102c9
Add mutators to pull and push Terraform state ( #288 )
...
## Changes
Pull state before deploying and push state after deploying.
Note: the run command was missing mutators to initialize Terraform. This
is necessary if the cache directory is removed between running "deploy"
and "run" (which is valid now that we synchronize state).
## Tests
Manually.
2023-03-30 12:01:09 +02:00
0ea0e81c8a
Ignore databricks_permissions resource when loading Terraform state ( #291 )
...
## Changes
The databricks_permissions resource may be generated if a bundle
resource includes a `permissions` block. There's no need to incorporate
details from the materialization into the bundle configuration struct.
## Tests
Confirmed that this fixes `bricks bundle run` when dealing with a bundle
with permission configuration.
2023-03-29 21:14:52 +02:00
87207bba78
Configure Terraform provider auth through env vars ( #290 )
...
## Changes
Auth relied on setting a profile. In this change we enumerate all
configuration properties and export all non-empty ones as a map with
environment variables. We then pass this map to the Terraform execution
wrapper.
This results in Terraform using the bundle's authentication
configuration.
This change is needed to make #287 work.
## Tests
Manually.
2023-03-29 20:46:09 +02:00
66ca9ec266
Add permissions block to each resource ( #264 )
...
Example:
```yaml
resources:
jobs:
my_job:
name: "[${bundle.environment}] My job"
permissions:
- level: CAN_VIEW
group_name: users
```
2023-03-21 10:58:16 +01:00
58563b1ea9
Add resources for mlflow models and experiments ( #263 )
...
Manually confirmed that both can be deployed.
2023-03-20 21:28:43 +01:00
ad666ff796
Use new logger throughout codebase ( #256 )
2023-03-17 15:17:31 +01:00
414ea4f891
Bump databricks-sdk-go to 0.3.2 ( #215 )
2023-02-20 16:00:20 +01:00
35243db33c
Automatically install Terraform if needed ( #141 )
...
Users can opt out and use the system-installed version with the
following configuration:
```
bundle:
terraform:
exec_path: terraform
```
This will find the binary in $PATH and replace it with the found value.
If this is not set, the initialize phase will install Terraform in the
bundle's cache directory.
2022-12-15 17:30:33 +01:00
b111416fe5
Add `bricks bundle run` command ( #134 )
2022-12-15 15:12:47 +01:00
72e89bf33c
Use pointers to resources in bundle configuration ( #140 )
...
Avoid copy-by-value when iterating over these maps.
2022-12-15 13:00:41 +01:00
d713521d63
Convert job task libraries to TF JSON ( #132 )
2022-12-12 16:36:59 +01:00
8640696b4b
Add minimal test for conversion to TF JSON format ( #130 )
2022-12-12 11:31:28 +01:00
4f668fc58b
Mutators to work with Terraform ( #124 )
...
This includes 3 mutators:
* Interpolate resources references to TF compatible format
* Convert resources struct to TF JSON format and write it to disk
* Run TF apply
2022-12-09 08:57:30 +01:00
Lennart Kats (databricks)
Pieter Noordhuis
Andrew Nester
shreyas-goenka
dependabot[bot]
Gleb Kanterov
Serge Smertin