From 91e04cc4441d011a568eaf6b064b85ff6c190d6b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 3 Feb 2025 12:36:30 +0100 Subject: [PATCH 01/16] Bump golangci/golangci-lint-action from 6.1.1 to 6.2.0 (#2273) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 6.1.1 to 6.2.0.
Release notes

Sourced from golangci/golangci-lint-action's releases.

v6.2.0

What's Changed

Changes

Documentation

Dependencies

New Contributors

Full Changelog: https://github.com/golangci/golangci-lint-action/compare/v6.1.1...v6.2.0

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golangci/golangci-lint-action&package-manager=github_actions&previous-version=6.1.1&new-version=6.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/push.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml index 2a8a68862..5921b4e5f 100644 --- a/.github/workflows/push.yml +++ b/.github/workflows/push.yml @@ -95,7 +95,7 @@ jobs: # Exit with status code 1 if there are differences (i.e. unformatted files) git diff --exit-code - name: golangci-lint - uses: golangci/golangci-lint-action@971e284b6050e8a5849b72094c50ab08da042db8 # v6.1.1 + uses: golangci/golangci-lint-action@ec5d18412c0aeab7936cb16880d708ba2a64e1ae # v6.2.0 with: version: v1.63.4 args: --timeout=15m From 75932198f72a2e0e658d36148b539ef545e8d828 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 3 Feb 2025 12:36:42 +0100 Subject: [PATCH 02/16] Bump astral-sh/ruff-action from 3.0.1 to 3.1.0 (#2274) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [astral-sh/ruff-action](https://github.com/astral-sh/ruff-action) from 3.0.1 to 3.1.0.
Release notes

Sourced from astral-sh/ruff-action's releases.

v3.1.0 🌈 Determine ruff version from optional or dependency groups

Changes

Big thank you to @​AA-Turner for expanding the pyproject.toml parsing to also find the ruff version to use in the following scenarios:

[dependency-groups]
dev = [
    { include-group = "docs" },
    { include-group = "lint" },
]
docs = [
    "sphinx",
]
lint = [
    "ruff==0.8.3",
]

[project.optional-dependencies]
lint = [
    "ruff==0.8.3",
]

🚀 Enhancements

  • Read the [project.optional-dependencies] and [dependency-groups] tables @​AA-Turner (#66)

v3.0.2 🌈 Full support for GHES

Changes

This release fixes some issues that prevented use with GitHub Enterprise Server instances. Parsing the ruff version from pyproject.toml now also uses a library that is fully TOML 1.0.0 compliant.

🐛 Bug fixes

🧰 Maintenance

📚 Documentation

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=astral-sh/ruff-action&package-manager=github_actions&previous-version=3.0.1&new-version=3.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/push.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml index 5921b4e5f..f27459baa 100644 --- a/.github/workflows/push.yml +++ b/.github/workflows/push.yml @@ -100,7 +100,7 @@ jobs: version: v1.63.4 args: --timeout=15m - name: Run ruff - uses: astral-sh/ruff-action@31a518504640beb4897d0b9f9e50a2a9196e75ba # v3.0.1 + uses: astral-sh/ruff-action@f14634c415d3e63ffd4d550a22f037df4c734a60 # v3.1.0 with: version: "0.9.1" args: "format --check" From 4f3a289333094951fe7b55b00a7f3bc70cd138eb Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 3 Feb 2025 12:36:50 +0100 Subject: [PATCH 03/16] Bump actions/stale from 9.0.0 to 9.1.0 (#2275) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0.
Release notes

Sourced from actions/stale's releases.

v9.1.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/stale/compare/v9...v9.1.0

Commits
  • 5bef64f build(deps): bump @​actions/cache from 3.2.2 to 4.0.0 (#1194)
  • fa77dfd build(deps-dev): bump @​types/jest from 29.5.11 to 29.5.14 (#1193)
  • f04443d build(deps): bump @​actions/core from 1.10.1 to 1.11.1 (#1191)
  • 5c715b0 build(deps-dev): bump ts-jest from 29.1.1 to 29.2.5 (#1175)
  • f691222 build(deps): bump actions/publish-action from 0.2.2 to 0.3.0 (#1147)
  • df990c2 build(deps): bump actions/checkout from 3 to 4 (#1091)
  • 6e472ce Merge pull request #1179 from actions/Jcambass-patch-1
  • d10ba64 Merge pull request #1150 from actions/dependabot/npm_and_yarn/undici-5.28.4
  • bbf3da5 resolve check failures
  • 6a2e61d Add workflow file for publishing releases to immutable action package
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/stale&package-manager=github_actions&previous-version=9.0.0&new-version=9.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/close-stale-issues.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/close-stale-issues.yml b/.github/workflows/close-stale-issues.yml index ea9558caf..fc764fb0d 100644 --- a/.github/workflows/close-stale-issues.yml +++ b/.github/workflows/close-stale-issues.yml @@ -18,7 +18,7 @@ jobs: pull-requests: write steps: - - uses: actions/stale@28ca1036281a5e5922ead5184a1bbf96e5fc984e # v9.0.0 + - uses: actions/stale@5bef64f19d7facfb25b37b414482c7164d639639 # v9.1.0 with: stale-issue-message: This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled. stale-pr-message: This PR has not received an update in a while. If you want to keep this PR open, please leave a comment below or push a new commit and auto-close will be canceled. From 75db82ae1f5f3fc12716cb7441d0374121dcf6b4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 3 Feb 2025 12:37:17 +0100 Subject: [PATCH 04/16] Bump actions/create-github-app-token from 1.11.1 to 1.11.2 (#2276) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/create-github-app-token](https://github.com/actions/create-github-app-token) from 1.11.1 to 1.11.2.
Release notes

Sourced from actions/create-github-app-token's releases.

v1.11.2

1.11.2 (2025-01-30)

Bug Fixes

Commits
  • 136412a build(release): 1.11.2 [skip ci]
  • b4192a5 fix(deps): bump @​octokit/request from 9.1.3 to 9.1.4 in the production-depend...
  • 29aa051 fix(deps): bump undici from 6.19.8 to 7.2.0 (#198)
  • a5f8600 build(deps-dev): bump @​sinonjs/fake-timers from 13.0.2 to 14.0.0 (#199)
  • 0edddd7 build(deps-dev): bump the development-dependencies group with 2 updates (#197)
  • bb3ca76 docs(README): remove extra space in variable syntax in README example (#201)
  • See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/create-github-app-token&package-manager=github_actions&previous-version=1.11.1&new-version=1.11.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/integration-main.yml | 2 +- .github/workflows/integration-pr.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/integration-main.yml b/.github/workflows/integration-main.yml index 84dd7263a..f737c48e6 100644 --- a/.github/workflows/integration-main.yml +++ b/.github/workflows/integration-main.yml @@ -20,7 +20,7 @@ jobs: steps: - name: Generate GitHub App Token id: generate-token - uses: actions/create-github-app-token@c1a285145b9d317df6ced56c09f525b5c2b6f755 # v1.11.1 + uses: actions/create-github-app-token@136412a57a7081aa63c935a2cc2918f76c34f514 # v1.11.2 with: app-id: ${{ secrets.DECO_WORKFLOW_TRIGGER_APP_ID }} private-key: ${{ secrets.DECO_WORKFLOW_TRIGGER_PRIVATE_KEY }} diff --git a/.github/workflows/integration-pr.yml b/.github/workflows/integration-pr.yml index 7a62113cd..bf096c863 100644 --- a/.github/workflows/integration-pr.yml +++ b/.github/workflows/integration-pr.yml @@ -23,7 +23,7 @@ jobs: steps: - name: Generate GitHub App Token id: generate-token - uses: actions/create-github-app-token@c1a285145b9d317df6ced56c09f525b5c2b6f755 # v1.11.1 + uses: actions/create-github-app-token@136412a57a7081aa63c935a2cc2918f76c34f514 # v1.11.2 with: app-id: ${{ secrets.DECO_WORKFLOW_TRIGGER_APP_ID }} private-key: ${{ secrets.DECO_WORKFLOW_TRIGGER_PRIVATE_KEY }} From 838de2fde23b2234843671ff38f021d1356edf67 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 3 Feb 2025 12:38:16 +0100 Subject: [PATCH 05/16] Bump github.com/hashicorp/terraform-exec from 0.21.0 to 0.22.0 (#2237) Bumps [github.com/hashicorp/terraform-exec](https://github.com/hashicorp/terraform-exec) from 0.21.0 to 0.22.0.
Release notes

Sourced from github.com/hashicorp/terraform-exec's releases.

v0.22.0

ENHANCEMENTS:

  • tfexec: Add support for terraform init --json via InitJSON (#478)

INTERNAL:

  • go: Require Go 1.22 (previously 1.18) (#499)
Changelog

Sourced from github.com/hashicorp/terraform-exec's changelog.

0.22.0 (January 21, 2025)

ENHANCEMENTS:

  • tfexec: Add support for terraform init --json via InitJSON (#478)

INTERNAL:

  • go: Require Go 1.22 (previously 1.18) (#499)
Commits
  • 6801a6e v0.22.0 [skip ci]
  • dd2bc9a Update CHANGELOG.md (#501)
  • b5e5740 build(deps): bump github.com/hashicorp/hc-install from 0.8.0 to 0.9.1 (#494)
  • abfb5ba tfexec: add InitJSON (#478)
  • 840ecad ci/e2etests: Add latest major Terraform versions (#498)
  • 4497f9e go: Require Go 1.22 (previously 1.18) (#499)
  • b13b10b build(deps): bump github.com/zclconf/go-cty from 1.16.0 to 1.16.1 (#496)
  • 6b0d5eb build(deps): bump github.com/zclconf/go-cty from 1.15.1 to 1.16.0 (#495)
  • ef0b6c3 build(deps): Bump workflows to latest trusted versions (#493)
  • c75d998 build(deps): bump github.com/hashicorp/terraform-json from 0.23.0 to 0.24.0 (...
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/hashicorp/terraform-exec&package-manager=go_modules&previous-version=0.21.0&new-version=0.22.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 6 +++--- go.sum | 12 ++++++------ 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/go.mod b/go.mod index bd8997190..151133944 100644 --- a/go.mod +++ b/go.mod @@ -13,8 +13,8 @@ require ( github.com/google/uuid v1.6.0 // BSD-3-Clause github.com/hashicorp/go-version v1.7.0 // MPL 2.0 github.com/hashicorp/hc-install v0.9.1 // MPL 2.0 - github.com/hashicorp/terraform-exec v0.21.0 // MPL 2.0 - github.com/hashicorp/terraform-json v0.23.0 // MPL 2.0 + github.com/hashicorp/terraform-exec v0.22.0 // MPL 2.0 + github.com/hashicorp/terraform-json v0.24.0 // MPL 2.0 github.com/hexops/gotextdiff v1.0.3 // BSD 3-Clause "New" or "Revised" License github.com/manifoldco/promptui v0.9.0 // BSD-3-Clause github.com/mattn/go-isatty v0.0.20 // MIT @@ -62,7 +62,7 @@ require ( github.com/tidwall/match v1.1.1 // indirect github.com/tidwall/pretty v1.2.1 // indirect github.com/tidwall/sjson v1.2.5 // indirect - github.com/zclconf/go-cty v1.15.0 // indirect + github.com/zclconf/go-cty v1.16.1 // indirect go.opencensus.io v0.24.0 // indirect go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect go.opentelemetry.io/otel v1.24.0 // indirect diff --git a/go.sum b/go.sum index dec1d40b2..3c7f20937 100644 --- a/go.sum +++ b/go.sum @@ -107,10 +107,10 @@ github.com/hashicorp/go-version v1.7.0 h1:5tqGy27NaOTB8yJKUZELlFAS/LTKJkrmONwQKe github.com/hashicorp/go-version v1.7.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= github.com/hashicorp/hc-install v0.9.1 h1:gkqTfE3vVbafGQo6VZXcy2v5yoz2bE0+nhZXruCuODQ= github.com/hashicorp/hc-install v0.9.1/go.mod h1:pWWvN/IrfeBK4XPeXXYkL6EjMufHkCK5DvwxeLKuBf0= -github.com/hashicorp/terraform-exec v0.21.0 h1:uNkLAe95ey5Uux6KJdua6+cv8asgILFVWkd/RG0D2XQ= -github.com/hashicorp/terraform-exec v0.21.0/go.mod h1:1PPeMYou+KDUSSeRE9szMZ/oHf4fYUmB923Wzbq1ICg= -github.com/hashicorp/terraform-json v0.23.0 h1:sniCkExU4iKtTADReHzACkk8fnpQXrdD2xoR+lppBkI= -github.com/hashicorp/terraform-json v0.23.0/go.mod h1:MHdXbBAbSg0GvzuWazEGKAn/cyNfIB7mN6y7KJN6y2c= +github.com/hashicorp/terraform-exec v0.22.0 h1:G5+4Sz6jYZfRYUCg6eQgDsqTzkNXV+fP8l+uRmZHj64= +github.com/hashicorp/terraform-exec v0.22.0/go.mod h1:bjVbsncaeh8jVdhttWYZuBGj21FcYw6Ia/XfHcNO7lQ= +github.com/hashicorp/terraform-json v0.24.0 h1:rUiyF+x1kYawXeRth6fKFm/MdfBS6+lW4NbeATsYz8Q= +github.com/hashicorp/terraform-json v0.24.0/go.mod h1:Nfj5ubo9xbu9uiAoZVBsNOjvNKB66Oyrvtit74kC7ow= github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM= github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= @@ -174,8 +174,8 @@ github.com/wI2L/jsondiff v0.6.1 h1:ISZb9oNWbP64LHnu4AUhsMF5W0FIj5Ok3Krip9Shqpw= github.com/wI2L/jsondiff v0.6.1/go.mod h1:KAEIojdQq66oJiHhDyQez2x+sRit0vIzC9KeK0yizxM= github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM= github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw= -github.com/zclconf/go-cty v1.15.0 h1:tTCRWxsexYUmtt/wVxgDClUe+uQusuI443uL6e+5sXQ= -github.com/zclconf/go-cty v1.15.0/go.mod h1:VvMs5i0vgZdhYawQNq5kePSpLAoz8u1xvZgrPIxfnZE= +github.com/zclconf/go-cty v1.16.1 h1:a5TZEPzBFFR53udlIKApXzj8JIF4ZNQ6abH79z5R1S0= +github.com/zclconf/go-cty v1.16.1/go.mod h1:VvMs5i0vgZdhYawQNq5kePSpLAoz8u1xvZgrPIxfnZE= go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0= go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo= go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0 h1:4Pp6oUg3+e/6M4C0A/3kJ2VYa++dsWVTtGgLVj5xtHg= From 9320bd1682a22df57950765d71d21b44852084a1 Mon Sep 17 00:00:00 2001 From: Denis Bilenko Date: Mon, 3 Feb 2025 15:10:19 +0100 Subject: [PATCH 06/16] acc: Use [VARNAME] instead of $VARNAME (#2282) $VARNAME is what we use for environment variables, it's good to separate. Some people use envsubst for homemade variable interpolation, it's also good to have separation there. --- acceptance/acceptance_test.go | 12 ++-- acceptance/bundle/git-permerror/output.txt | 18 +++--- .../bundle/help/bundle-deploy/output.txt | 2 +- .../bundle/help/bundle-deployment/output.txt | 2 +- .../bundle/help/bundle-destroy/output.txt | 2 +- .../help/bundle-generate-dashboard/output.txt | 2 +- .../help/bundle-generate-job/output.txt | 2 +- .../help/bundle-generate-pipeline/output.txt | 2 +- .../bundle/help/bundle-generate/output.txt | 2 +- acceptance/bundle/help/bundle-init/output.txt | 2 +- acceptance/bundle/help/bundle-open/output.txt | 2 +- acceptance/bundle/help/bundle-run/output.txt | 2 +- .../bundle/help/bundle-schema/output.txt | 2 +- .../bundle/help/bundle-summary/output.txt | 2 +- acceptance/bundle/help/bundle-sync/output.txt | 2 +- .../bundle/help/bundle-validate/output.txt | 2 +- acceptance/bundle/help/bundle/output.txt | 2 +- .../bundle/override/clusters/output.txt | 4 +- .../bundle/override/job_cluster/output.txt | 8 +-- .../override/job_cluster_var/output.txt | 20 +++---- .../job_tasks/out.development.stderr.txt | 2 +- .../bundle/override/job_tasks/output.txt | 8 +-- .../override/merge-string-map/output.txt | 8 +-- .../override/pipeline_cluster/output.txt | 8 +-- .../bundle/paths/fallback/output.job.json | 8 +-- .../paths/fallback/output.pipeline.json | 8 +-- acceptance/bundle/paths/fallback/output.txt | 8 +-- .../bundle/paths/nominal/output.job.json | 12 ++-- .../bundle/paths/nominal/output.pipeline.json | 8 +-- acceptance/bundle/paths/nominal/output.txt | 8 +-- .../relative_path_translation/output.txt | 4 +- acceptance/bundle/quality_monitor/output.txt | 6 +- acceptance/bundle/scripts/output.txt | 12 ++-- .../bundle/syncroot/dotdot-git/output.txt | 6 +- .../bundle/syncroot/dotdot-nogit/output.txt | 4 +- .../bundle/templates/dbt-sql/output.txt | 20 +++---- .../dbt-sql/output/my_dbt_sql/databricks.yml | 12 ++-- .../output/my_dbt_sql/profile_template.yml | 4 +- .../my_dbt_sql/resources/my_dbt_sql.job.yml | 2 +- .../templates/default-python/output.txt | 20 +++---- .../output/my_default_python/databricks.yml | 12 ++-- .../resources/my_default_python.job.yml | 2 +- .../output/my_default_python/setup.py | 2 +- .../bundle/templates/default-sql/output.txt | 20 +++---- .../output/my_default_sql/databricks.yml | 12 ++-- .../resources/my_default_sql_sql.job.yml | 2 +- .../experimental-jobs-as-code/output.txt | 18 +++--- .../output/my_jobs_as_code/databricks.yml | 12 ++-- .../resources/my_jobs_as_code_job.py | 2 +- .../bundle/templates/wrong-url/output.txt | 2 +- .../bundle/variables/arg-repeat/output.txt | 4 +- .../variables/complex-cycle-self/output.txt | 4 +- .../bundle/variables/complex-cycle/output.txt | 4 +- .../bundle/variables/complex/out.default.json | 2 +- .../bundle/variables/complex/out.dev.json | 2 +- .../bundle/variables/complex/output.txt | 4 +- .../complex_multiple_files/output.txt | 2 +- .../variables/double_underscore/output.txt | 2 +- acceptance/bundle/variables/empty/output.txt | 4 +- .../bundle/variables/env_overrides/output.txt | 16 ++--- .../bundle/variables/file-defaults/output.txt | 22 +++---- .../bundle/variables/file-defaults/test.toml | 8 +-- .../bundle/variables/git-branch/output.txt | 40 ++++++------- acceptance/bundle/variables/host/output.txt | 4 +- .../prepend-workspace-var/output.txt | 20 +++---- .../variables/resolve-nonstrings/output.txt | 2 +- .../bundle/variables/vanilla/output.txt | 8 +-- .../variable_overrides_in_target/output.txt | 16 ++--- acceptance/selftest/output.txt | 8 +-- acceptance/selftest/test.toml | 4 +- acceptance/terraform/output.txt | 8 +-- acceptance/workspace/jobs/create/output.txt | 2 +- .../bundle/testdata/apps/bundle_deploy.txt | 2 +- .../bundle/testdata/apps/bundle_validate.txt | 4 +- .../testdata/default_python/bundle_deploy.txt | 2 +- .../testdata/default_python/bundle_init.txt | 2 +- .../default_python/bundle_summary.txt | 58 +++++++++---------- .../default_python/bundle_validate.txt | 6 +- libs/testdiff/replacement.go | 49 ++++++++-------- 79 files changed, 324 insertions(+), 329 deletions(-) diff --git a/acceptance/acceptance_test.go b/acceptance/acceptance_test.go index ccdf74bcb..871b8bd62 100644 --- a/acceptance/acceptance_test.go +++ b/acceptance/acceptance_test.go @@ -102,13 +102,13 @@ func testAccept(t *testing.T, InprocessMode bool, singleTest string) int { } t.Setenv("CLI", execPath) - repls.SetPath(execPath, "$CLI") + repls.SetPath(execPath, "[CLI]") // Make helper scripts available t.Setenv("PATH", fmt.Sprintf("%s%c%s", filepath.Join(cwd, "bin"), os.PathListSeparator, os.Getenv("PATH"))) tempHomeDir := t.TempDir() - repls.SetPath(tempHomeDir, "$TMPHOME") + repls.SetPath(tempHomeDir, "[TMPHOME]") t.Logf("$TMPHOME=%v", tempHomeDir) // Make use of uv cache; since we set HomeEnvVar to temporary directory, it is not picked up automatically @@ -133,7 +133,7 @@ func testAccept(t *testing.T, InprocessMode bool, singleTest string) int { terraformrcPath := filepath.Join(buildDir, ".terraformrc") t.Setenv("TF_CLI_CONFIG_FILE", terraformrcPath) t.Setenv("DATABRICKS_TF_CLI_CONFIG_FILE", terraformrcPath) - repls.SetPath(terraformrcPath, "$DATABRICKS_TF_CLI_CONFIG_FILE") + repls.SetPath(terraformrcPath, "[DATABRICKS_TF_CLI_CONFIG_FILE]") terraformExecPath := filepath.Join(buildDir, "terraform") if runtime.GOOS == "windows" { @@ -141,10 +141,10 @@ func testAccept(t *testing.T, InprocessMode bool, singleTest string) int { } t.Setenv("DATABRICKS_TF_EXEC_PATH", terraformExecPath) t.Setenv("TERRAFORM", terraformExecPath) - repls.SetPath(terraformExecPath, "$TERRAFORM") + repls.SetPath(terraformExecPath, "[TERRAFORM]") // do it last so that full paths match first: - repls.SetPath(buildDir, "$BUILD_DIR") + repls.SetPath(buildDir, "[BUILD_DIR]") workspaceClient, err := databricks.NewWorkspaceClient() require.NoError(t, err) @@ -226,7 +226,7 @@ func runTest(t *testing.T, dir, coverDir string, repls testdiff.ReplacementsCont tmpDir = t.TempDir() } - repls.SetPathWithParents(tmpDir, "$TMPDIR") + repls.SetPathWithParents(tmpDir, "[TMPDIR]") repls.Repls = append(repls.Repls, config.Repls...) scriptContents := readMergedScriptContents(t, dir) diff --git a/acceptance/bundle/git-permerror/output.txt b/acceptance/bundle/git-permerror/output.txt index 60e77ca0e..03ab93442 100644 --- a/acceptance/bundle/git-permerror/output.txt +++ b/acceptance/bundle/git-permerror/output.txt @@ -2,20 +2,20 @@ >>> chmod 000 .git ->>> $CLI bundle validate +>>> [CLI] bundle validate Error: unable to load repository specific gitconfig: open config: permission denied Name: git-permerror Target: default Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/git-permerror/default + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/git-permerror/default Found 1 error Exit code: 1 ->>> $CLI bundle validate -o json +>>> [CLI] bundle validate -o json Error: unable to load repository specific gitconfig: open config: permission denied @@ -24,7 +24,7 @@ Exit code: 1 "bundle_root_path": "." } ->>> withdir subdir/a/b $CLI bundle validate -o json +>>> withdir subdir/a/b [CLI] bundle validate -o json Error: unable to load repository specific gitconfig: open config: permission denied @@ -38,12 +38,12 @@ Exit code: 1 >>> chmod 000 .git/HEAD ->>> $CLI bundle validate -o json +>>> [CLI] bundle validate -o json { "bundle_root_path": "." } ->>> withdir subdir/a/b $CLI bundle validate -o json +>>> withdir subdir/a/b [CLI] bundle validate -o json { "bundle_root_path": "." } @@ -53,7 +53,7 @@ Exit code: 1 >>> chmod 000 .git/config ->>> $CLI bundle validate -o json +>>> [CLI] bundle validate -o json Error: unable to load repository specific gitconfig: open config: permission denied @@ -62,7 +62,7 @@ Exit code: 1 "bundle_root_path": "." } ->>> withdir subdir/a/b $CLI bundle validate -o json +>>> withdir subdir/a/b [CLI] bundle validate -o json Error: unable to load repository specific gitconfig: open config: permission denied diff --git a/acceptance/bundle/help/bundle-deploy/output.txt b/acceptance/bundle/help/bundle-deploy/output.txt index 13c903f3e..84351e375 100644 --- a/acceptance/bundle/help/bundle-deploy/output.txt +++ b/acceptance/bundle/help/bundle-deploy/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle deploy --help +>>> [CLI] bundle deploy --help Deploy bundle Usage: diff --git a/acceptance/bundle/help/bundle-deployment/output.txt b/acceptance/bundle/help/bundle-deployment/output.txt index ddf5b3305..4199703b3 100644 --- a/acceptance/bundle/help/bundle-deployment/output.txt +++ b/acceptance/bundle/help/bundle-deployment/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle deployment --help +>>> [CLI] bundle deployment --help Deployment related commands Usage: diff --git a/acceptance/bundle/help/bundle-destroy/output.txt b/acceptance/bundle/help/bundle-destroy/output.txt index d70164301..5ed9c1b7b 100644 --- a/acceptance/bundle/help/bundle-destroy/output.txt +++ b/acceptance/bundle/help/bundle-destroy/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle destroy --help +>>> [CLI] bundle destroy --help Destroy deployed bundle resources Usage: diff --git a/acceptance/bundle/help/bundle-generate-dashboard/output.txt b/acceptance/bundle/help/bundle-generate-dashboard/output.txt index a63ce0ff8..683175940 100644 --- a/acceptance/bundle/help/bundle-generate-dashboard/output.txt +++ b/acceptance/bundle/help/bundle-generate-dashboard/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle generate dashboard --help +>>> [CLI] bundle generate dashboard --help Generate configuration for a dashboard Usage: diff --git a/acceptance/bundle/help/bundle-generate-job/output.txt b/acceptance/bundle/help/bundle-generate-job/output.txt index adc3f45ae..6a4274223 100644 --- a/acceptance/bundle/help/bundle-generate-job/output.txt +++ b/acceptance/bundle/help/bundle-generate-job/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle generate job --help +>>> [CLI] bundle generate job --help Generate bundle configuration for a job Usage: diff --git a/acceptance/bundle/help/bundle-generate-pipeline/output.txt b/acceptance/bundle/help/bundle-generate-pipeline/output.txt index cf5f70920..05c5573b8 100644 --- a/acceptance/bundle/help/bundle-generate-pipeline/output.txt +++ b/acceptance/bundle/help/bundle-generate-pipeline/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle generate pipeline --help +>>> [CLI] bundle generate pipeline --help Generate bundle configuration for a pipeline Usage: diff --git a/acceptance/bundle/help/bundle-generate/output.txt b/acceptance/bundle/help/bundle-generate/output.txt index 1d77dfdbd..725f19af0 100644 --- a/acceptance/bundle/help/bundle-generate/output.txt +++ b/acceptance/bundle/help/bundle-generate/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle generate --help +>>> [CLI] bundle generate --help Generate bundle configuration Usage: diff --git a/acceptance/bundle/help/bundle-init/output.txt b/acceptance/bundle/help/bundle-init/output.txt index bafe5a187..fbafedea2 100644 --- a/acceptance/bundle/help/bundle-init/output.txt +++ b/acceptance/bundle/help/bundle-init/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle init --help +>>> [CLI] bundle init --help Initialize using a bundle template. TEMPLATE_PATH optionally specifies which template to use. It can be one of the following: diff --git a/acceptance/bundle/help/bundle-open/output.txt b/acceptance/bundle/help/bundle-open/output.txt index 8b98aa850..b8f3f118b 100644 --- a/acceptance/bundle/help/bundle-open/output.txt +++ b/acceptance/bundle/help/bundle-open/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle open --help +>>> [CLI] bundle open --help Open a resource in the browser Usage: diff --git a/acceptance/bundle/help/bundle-run/output.txt b/acceptance/bundle/help/bundle-run/output.txt index 17763a295..4b9efbf2a 100644 --- a/acceptance/bundle/help/bundle-run/output.txt +++ b/acceptance/bundle/help/bundle-run/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle run --help +>>> [CLI] bundle run --help Run the job or pipeline identified by KEY. The KEY is the unique identifier of the resource to run. In addition to diff --git a/acceptance/bundle/help/bundle-schema/output.txt b/acceptance/bundle/help/bundle-schema/output.txt index 8f2983f5b..8b8a6b8e9 100644 --- a/acceptance/bundle/help/bundle-schema/output.txt +++ b/acceptance/bundle/help/bundle-schema/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle schema --help +>>> [CLI] bundle schema --help Generate JSON Schema for bundle configuration Usage: diff --git a/acceptance/bundle/help/bundle-summary/output.txt b/acceptance/bundle/help/bundle-summary/output.txt index 935c4bdc5..534bb8214 100644 --- a/acceptance/bundle/help/bundle-summary/output.txt +++ b/acceptance/bundle/help/bundle-summary/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle summary --help +>>> [CLI] bundle summary --help Summarize resources deployed by this bundle Usage: diff --git a/acceptance/bundle/help/bundle-sync/output.txt b/acceptance/bundle/help/bundle-sync/output.txt index 6588e6978..992138a20 100644 --- a/acceptance/bundle/help/bundle-sync/output.txt +++ b/acceptance/bundle/help/bundle-sync/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle sync --help +>>> [CLI] bundle sync --help Synchronize bundle tree to the workspace Usage: diff --git a/acceptance/bundle/help/bundle-validate/output.txt b/acceptance/bundle/help/bundle-validate/output.txt index a0c350faf..7fd1ae7ea 100644 --- a/acceptance/bundle/help/bundle-validate/output.txt +++ b/acceptance/bundle/help/bundle-validate/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle validate --help +>>> [CLI] bundle validate --help Validate configuration Usage: diff --git a/acceptance/bundle/help/bundle/output.txt b/acceptance/bundle/help/bundle/output.txt index e0e2ea47c..fc6dd623d 100644 --- a/acceptance/bundle/help/bundle/output.txt +++ b/acceptance/bundle/help/bundle/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle --help +>>> [CLI] bundle --help Databricks Asset Bundles let you express data/AI/analytics projects as code. Online documentation: https://docs.databricks.com/en/dev-tools/bundles/index.html diff --git a/acceptance/bundle/override/clusters/output.txt b/acceptance/bundle/override/clusters/output.txt index cff30b3af..a30a7bbff 100644 --- a/acceptance/bundle/override/clusters/output.txt +++ b/acceptance/bundle/override/clusters/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle validate -o json -t default +>>> [CLI] bundle validate -o json -t default { "autoscale": { "max_workers": 7, @@ -15,7 +15,7 @@ "spark_version": "13.3.x-scala2.12" } ->>> $CLI bundle validate -o json -t development +>>> [CLI] bundle validate -o json -t development { "autoscale": { "max_workers": 3, diff --git a/acceptance/bundle/override/job_cluster/output.txt b/acceptance/bundle/override/job_cluster/output.txt index ff6e8316e..e4120e1c3 100644 --- a/acceptance/bundle/override/job_cluster/output.txt +++ b/acceptance/bundle/override/job_cluster/output.txt @@ -1,10 +1,10 @@ ->>> $CLI bundle validate -o json -t development +>>> [CLI] bundle validate -o json -t development { "foo": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/override_job_cluster/development/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/override_job_cluster/development/state/metadata.json" }, "edit_mode": "UI_LOCKED", "format": "MULTI_TASK", @@ -27,12 +27,12 @@ } } ->>> $CLI bundle validate -o json -t staging +>>> [CLI] bundle validate -o json -t staging { "foo": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/override_job_cluster/staging/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/override_job_cluster/staging/state/metadata.json" }, "edit_mode": "UI_LOCKED", "format": "MULTI_TASK", diff --git a/acceptance/bundle/override/job_cluster_var/output.txt b/acceptance/bundle/override/job_cluster_var/output.txt index 0b19e5eb2..3545d6987 100644 --- a/acceptance/bundle/override/job_cluster_var/output.txt +++ b/acceptance/bundle/override/job_cluster_var/output.txt @@ -1,10 +1,10 @@ ->>> $CLI bundle validate -o json -t development +>>> [CLI] bundle validate -o json -t development { "foo": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/override_job_cluster/development/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/override_job_cluster/development/state/metadata.json" }, "edit_mode": "UI_LOCKED", "format": "MULTI_TASK", @@ -27,21 +27,21 @@ } } ->>> $CLI bundle validate -t development +>>> [CLI] bundle validate -t development Name: override_job_cluster Target: development Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/override_job_cluster/development + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/override_job_cluster/development Validation OK! ->>> $CLI bundle validate -o json -t staging +>>> [CLI] bundle validate -o json -t staging { "foo": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/override_job_cluster/staging/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/override_job_cluster/staging/state/metadata.json" }, "edit_mode": "UI_LOCKED", "format": "MULTI_TASK", @@ -64,11 +64,11 @@ Validation OK! } } ->>> $CLI bundle validate -t staging +>>> [CLI] bundle validate -t staging Name: override_job_cluster Target: staging Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/override_job_cluster/staging + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/override_job_cluster/staging Validation OK! diff --git a/acceptance/bundle/override/job_tasks/out.development.stderr.txt b/acceptance/bundle/override/job_tasks/out.development.stderr.txt index 7b6fef0cc..1873feb35 100644 --- a/acceptance/bundle/override/job_tasks/out.development.stderr.txt +++ b/acceptance/bundle/override/job_tasks/out.development.stderr.txt @@ -1,5 +1,5 @@ ->>> errcode $CLI bundle validate -o json -t development +>>> errcode [CLI] bundle validate -o json -t development Error: file ./test1.py not found diff --git a/acceptance/bundle/override/job_tasks/output.txt b/acceptance/bundle/override/job_tasks/output.txt index 915351d4e..1f7796217 100644 --- a/acceptance/bundle/override/job_tasks/output.txt +++ b/acceptance/bundle/override/job_tasks/output.txt @@ -28,7 +28,7 @@ ] } ->>> errcode $CLI bundle validate -o json -t staging +>>> errcode [CLI] bundle validate -o json -t staging Error: file ./test1.py not found @@ -63,14 +63,14 @@ Exit code: 1 ] } ->>> errcode $CLI bundle validate -t staging +>>> errcode [CLI] bundle validate -t staging Error: file ./test1.py not found Name: override_job_tasks Target: staging Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/override_job_tasks/staging + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/override_job_tasks/staging Found 1 error diff --git a/acceptance/bundle/override/merge-string-map/output.txt b/acceptance/bundle/override/merge-string-map/output.txt index b566aa07f..6e2aef87b 100644 --- a/acceptance/bundle/override/merge-string-map/output.txt +++ b/acceptance/bundle/override/merge-string-map/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle validate -o json -t dev +>>> [CLI] bundle validate -o json -t dev Warning: expected map, found string at resources.clusters.my_cluster in databricks.yml:6:17 @@ -13,7 +13,7 @@ Warning: expected map, found string } } ->>> $CLI bundle validate -t dev +>>> [CLI] bundle validate -t dev Warning: expected map, found string at resources.clusters.my_cluster in databricks.yml:6:17 @@ -21,7 +21,7 @@ Warning: expected map, found string Name: merge-string-map Target: dev Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/merge-string-map/dev + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/merge-string-map/dev Found 1 warning diff --git a/acceptance/bundle/override/pipeline_cluster/output.txt b/acceptance/bundle/override/pipeline_cluster/output.txt index 8babed0ec..d1a67f6b9 100644 --- a/acceptance/bundle/override/pipeline_cluster/output.txt +++ b/acceptance/bundle/override/pipeline_cluster/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle validate -o json -t development +>>> [CLI] bundle validate -o json -t development { "foo": { "clusters": [ @@ -14,14 +14,14 @@ ], "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/override_pipeline_cluster/development/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/override_pipeline_cluster/development/state/metadata.json" }, "name": "job", "permissions": [] } } ->>> $CLI bundle validate -o json -t staging +>>> [CLI] bundle validate -o json -t staging { "foo": { "clusters": [ @@ -36,7 +36,7 @@ ], "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/override_pipeline_cluster/staging/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/override_pipeline_cluster/staging/state/metadata.json" }, "name": "job", "permissions": [] diff --git a/acceptance/bundle/paths/fallback/output.job.json b/acceptance/bundle/paths/fallback/output.job.json index fe9e1cf3d..ac79e0cf6 100644 --- a/acceptance/bundle/paths/fallback/output.job.json +++ b/acceptance/bundle/paths/fallback/output.job.json @@ -2,14 +2,14 @@ { "job_cluster_key": "default", "notebook_task": { - "notebook_path": "/Workspace/Users/$USERNAME/.bundle/fallback/development/files/src/notebook" + "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/fallback/development/files/src/notebook" }, "task_key": "notebook_example" }, { "job_cluster_key": "default", "spark_python_task": { - "python_file": "/Workspace/Users/$USERNAME/.bundle/fallback/development/files/src/file.py" + "python_file": "/Workspace/Users/[USERNAME]/.bundle/fallback/development/files/src/file.py" }, "task_key": "spark_python_example" }, @@ -19,7 +19,7 @@ "dbt run", "dbt run" ], - "project_directory": "/Workspace/Users/$USERNAME/.bundle/fallback/development/files/src/dbt_project" + "project_directory": "/Workspace/Users/[USERNAME]/.bundle/fallback/development/files/src/dbt_project" }, "job_cluster_key": "default", "task_key": "dbt_example" @@ -28,7 +28,7 @@ "job_cluster_key": "default", "sql_task": { "file": { - "path": "/Workspace/Users/$USERNAME/.bundle/fallback/development/files/src/sql.sql" + "path": "/Workspace/Users/[USERNAME]/.bundle/fallback/development/files/src/sql.sql" }, "warehouse_id": "cafef00d" }, diff --git a/acceptance/bundle/paths/fallback/output.pipeline.json b/acceptance/bundle/paths/fallback/output.pipeline.json index 38521cb22..7ed4f74e6 100644 --- a/acceptance/bundle/paths/fallback/output.pipeline.json +++ b/acceptance/bundle/paths/fallback/output.pipeline.json @@ -1,22 +1,22 @@ [ { "file": { - "path": "/Workspace/Users/$USERNAME/.bundle/fallback/development/files/src/file1.py" + "path": "/Workspace/Users/[USERNAME]/.bundle/fallback/development/files/src/file1.py" } }, { "notebook": { - "path": "/Workspace/Users/$USERNAME/.bundle/fallback/development/files/src/notebook1" + "path": "/Workspace/Users/[USERNAME]/.bundle/fallback/development/files/src/notebook1" } }, { "file": { - "path": "/Workspace/Users/$USERNAME/.bundle/fallback/development/files/src/file2.py" + "path": "/Workspace/Users/[USERNAME]/.bundle/fallback/development/files/src/file2.py" } }, { "notebook": { - "path": "/Workspace/Users/$USERNAME/.bundle/fallback/development/files/src/notebook2" + "path": "/Workspace/Users/[USERNAME]/.bundle/fallback/development/files/src/notebook2" } } ] diff --git a/acceptance/bundle/paths/fallback/output.txt b/acceptance/bundle/paths/fallback/output.txt index 63121f3d7..85f185851 100644 --- a/acceptance/bundle/paths/fallback/output.txt +++ b/acceptance/bundle/paths/fallback/output.txt @@ -1,15 +1,15 @@ ->>> $CLI bundle validate -t development -o json +>>> [CLI] bundle validate -t development -o json ->>> $CLI bundle validate -t error +>>> [CLI] bundle validate -t error Error: notebook this value is overridden not found. Local notebook references are expected to contain one of the following file extensions: [.py, .r, .scala, .sql, .ipynb] Name: fallback Target: error Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/fallback/error + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/fallback/error Found 1 error diff --git a/acceptance/bundle/paths/nominal/output.job.json b/acceptance/bundle/paths/nominal/output.job.json index 9e1cb4d90..26d19d77c 100644 --- a/acceptance/bundle/paths/nominal/output.job.json +++ b/acceptance/bundle/paths/nominal/output.job.json @@ -2,14 +2,14 @@ { "job_cluster_key": "default", "notebook_task": { - "notebook_path": "/Workspace/Users/$USERNAME/.bundle/nominal/development/files/src/notebook" + "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/nominal/development/files/src/notebook" }, "task_key": "notebook_example" }, { "job_cluster_key": "default", "spark_python_task": { - "python_file": "/Workspace/Users/$USERNAME/.bundle/nominal/development/files/src/file.py" + "python_file": "/Workspace/Users/[USERNAME]/.bundle/nominal/development/files/src/file.py" }, "task_key": "spark_python_example" }, @@ -19,7 +19,7 @@ "dbt run", "dbt run" ], - "project_directory": "/Workspace/Users/$USERNAME/.bundle/nominal/development/files/src/dbt_project" + "project_directory": "/Workspace/Users/[USERNAME]/.bundle/nominal/development/files/src/dbt_project" }, "job_cluster_key": "default", "task_key": "dbt_example" @@ -28,7 +28,7 @@ "job_cluster_key": "default", "sql_task": { "file": { - "path": "/Workspace/Users/$USERNAME/.bundle/nominal/development/files/src/sql.sql" + "path": "/Workspace/Users/[USERNAME]/.bundle/nominal/development/files/src/sql.sql" }, "warehouse_id": "cafef00d" }, @@ -68,7 +68,7 @@ "for_each_task": { "task": { "notebook_task": { - "notebook_path": "/Workspace/Users/$USERNAME/.bundle/nominal/development/files/src/notebook" + "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/nominal/development/files/src/notebook" } } }, @@ -80,7 +80,7 @@ "task": { "job_cluster_key": "default", "spark_python_task": { - "python_file": "/Workspace/Users/$USERNAME/.bundle/nominal/development/files/src/file.py" + "python_file": "/Workspace/Users/[USERNAME]/.bundle/nominal/development/files/src/file.py" } } }, diff --git a/acceptance/bundle/paths/nominal/output.pipeline.json b/acceptance/bundle/paths/nominal/output.pipeline.json index 277b0c4a1..c6f2e0868 100644 --- a/acceptance/bundle/paths/nominal/output.pipeline.json +++ b/acceptance/bundle/paths/nominal/output.pipeline.json @@ -1,22 +1,22 @@ [ { "file": { - "path": "/Workspace/Users/$USERNAME/.bundle/nominal/development/files/src/file1.py" + "path": "/Workspace/Users/[USERNAME]/.bundle/nominal/development/files/src/file1.py" } }, { "notebook": { - "path": "/Workspace/Users/$USERNAME/.bundle/nominal/development/files/src/notebook1" + "path": "/Workspace/Users/[USERNAME]/.bundle/nominal/development/files/src/notebook1" } }, { "file": { - "path": "/Workspace/Users/$USERNAME/.bundle/nominal/development/files/src/file2.py" + "path": "/Workspace/Users/[USERNAME]/.bundle/nominal/development/files/src/file2.py" } }, { "notebook": { - "path": "/Workspace/Users/$USERNAME/.bundle/nominal/development/files/src/notebook2" + "path": "/Workspace/Users/[USERNAME]/.bundle/nominal/development/files/src/notebook2" } } ] diff --git a/acceptance/bundle/paths/nominal/output.txt b/acceptance/bundle/paths/nominal/output.txt index 1badcdec6..40670f4cb 100644 --- a/acceptance/bundle/paths/nominal/output.txt +++ b/acceptance/bundle/paths/nominal/output.txt @@ -1,15 +1,15 @@ ->>> $CLI bundle validate -t development -o json +>>> [CLI] bundle validate -t development -o json ->>> $CLI bundle validate -t error +>>> [CLI] bundle validate -t error Error: notebook this value is overridden not found. Local notebook references are expected to contain one of the following file extensions: [.py, .r, .scala, .sql, .ipynb] Name: nominal Target: error Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/nominal/error + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/nominal/error Found 1 error diff --git a/acceptance/bundle/paths/relative_path_translation/output.txt b/acceptance/bundle/paths/relative_path_translation/output.txt index 362f2ec7b..b13d612b6 100644 --- a/acceptance/bundle/paths/relative_path_translation/output.txt +++ b/acceptance/bundle/paths/relative_path_translation/output.txt @@ -1,4 +1,4 @@ ->>> $CLI bundle validate -t default -o json +>>> [CLI] bundle validate -t default -o json ->>> $CLI bundle validate -t override -o json +>>> [CLI] bundle validate -t override -o json diff --git a/acceptance/bundle/quality_monitor/output.txt b/acceptance/bundle/quality_monitor/output.txt index b3718c802..8a7f64ef2 100644 --- a/acceptance/bundle/quality_monitor/output.txt +++ b/acceptance/bundle/quality_monitor/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle validate -o json -t development +>>> [CLI] bundle validate -o json -t development { "mode": "development", "quality_monitors": { @@ -21,7 +21,7 @@ } } ->>> $CLI bundle validate -o json -t staging +>>> [CLI] bundle validate -o json -t staging { "mode": null, "quality_monitors": { @@ -46,7 +46,7 @@ } } ->>> $CLI bundle validate -o json -t production +>>> [CLI] bundle validate -o json -t production { "mode": null, "quality_monitors": { diff --git a/acceptance/bundle/scripts/output.txt b/acceptance/bundle/scripts/output.txt index ec5978380..2deedb0e7 100644 --- a/acceptance/bundle/scripts/output.txt +++ b/acceptance/bundle/scripts/output.txt @@ -1,5 +1,5 @@ ->>> EXITCODE=0 errcode $CLI bundle validate +>>> EXITCODE=0 errcode [CLI] bundle validate Executing 'preinit' script from myscript.py 0 preinit: hello stdout! from myscript.py 0 preinit: hello stderr! @@ -9,12 +9,12 @@ from myscript.py 0 postinit: hello stderr! Name: scripts Target: default Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/scripts/default + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/scripts/default Validation OK! ->>> EXITCODE=1 errcode $CLI bundle validate +>>> EXITCODE=1 errcode [CLI] bundle validate Executing 'preinit' script from myscript.py 1 preinit: hello stdout! from myscript.py 1 preinit: hello stderr! @@ -26,7 +26,7 @@ Found 1 error Exit code: 1 ->>> EXITCODE=0 errcode $CLI bundle deploy +>>> EXITCODE=0 errcode [CLI] bundle deploy Executing 'preinit' script from myscript.py 0 preinit: hello stdout! from myscript.py 0 preinit: hello stderr! @@ -42,7 +42,7 @@ from myscript.py 0 postbuild: hello stderr! Executing 'predeploy' script from myscript.py 0 predeploy: hello stdout! from myscript.py 0 predeploy: hello stderr! -Error: unable to deploy to /Workspace/Users/$USERNAME/.bundle/scripts/default/state as $USERNAME. +Error: unable to deploy to /Workspace/Users/[USERNAME]/.bundle/scripts/default/state as [USERNAME]. Please make sure the current user or one of their groups is listed under the permissions of this bundle. For assistance, contact the owners of this project. They may need to redeploy the bundle to apply the new permissions. diff --git a/acceptance/bundle/syncroot/dotdot-git/output.txt b/acceptance/bundle/syncroot/dotdot-git/output.txt index f1dc5fb01..dbfc8451f 100644 --- a/acceptance/bundle/syncroot/dotdot-git/output.txt +++ b/acceptance/bundle/syncroot/dotdot-git/output.txt @@ -1,10 +1,10 @@ -Error: path "$TMPDIR" is not within repository root "$TMPDIR/myrepo" +Error: path "[TMPDIR]" is not within repository root "[TMPDIR]/myrepo" Name: test-bundle Target: default Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/test-bundle/default + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/test-bundle/default Found 1 error diff --git a/acceptance/bundle/syncroot/dotdot-nogit/output.txt b/acceptance/bundle/syncroot/dotdot-nogit/output.txt index 46f617f35..4f189effd 100644 --- a/acceptance/bundle/syncroot/dotdot-nogit/output.txt +++ b/acceptance/bundle/syncroot/dotdot-nogit/output.txt @@ -1,7 +1,7 @@ Name: test-bundle Target: default Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/test-bundle/default + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/test-bundle/default Validation OK! diff --git a/acceptance/bundle/templates/dbt-sql/output.txt b/acceptance/bundle/templates/dbt-sql/output.txt index 972c7e152..2699ad554 100644 --- a/acceptance/bundle/templates/dbt-sql/output.txt +++ b/acceptance/bundle/templates/dbt-sql/output.txt @@ -1,32 +1,32 @@ ->>> $CLI bundle init dbt-sql --config-file ./input.json --output-dir output +>>> [CLI] bundle init dbt-sql --config-file ./input.json --output-dir output Welcome to the dbt template for Databricks Asset Bundles! A workspace was selected based on your current profile. For information about how to change this, see https://docs.databricks.com/dev-tools/cli/profiles.html. -workspace_host: $DATABRICKS_URL +workspace_host: [DATABRICKS_URL] 📊 Your new project has been created in the 'my_dbt_sql' directory! If you already have dbt installed, just type 'cd my_dbt_sql; dbt init' to get started. Refer to the README.md file for full "getting started" guide and production setup instructions. ->>> $CLI bundle validate -t dev +>>> [CLI] bundle validate -t dev Name: my_dbt_sql Target: dev Workspace: - Host: $DATABRICKS_URL - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/my_dbt_sql/dev + Host: [DATABRICKS_URL] + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/my_dbt_sql/dev Validation OK! ->>> $CLI bundle validate -t prod +>>> [CLI] bundle validate -t prod Name: my_dbt_sql Target: prod Workspace: - Host: $DATABRICKS_URL - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/my_dbt_sql/prod + Host: [DATABRICKS_URL] + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/my_dbt_sql/prod Validation OK! diff --git a/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/databricks.yml b/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/databricks.yml index cdf3704b9..3651ef12d 100644 --- a/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/databricks.yml +++ b/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/databricks.yml @@ -19,16 +19,16 @@ targets: # See also https://docs.databricks.com/dev-tools/bundles/deployment-modes.html. mode: development workspace: - host: $DATABRICKS_URL + host: [DATABRICKS_URL] prod: mode: production workspace: - host: $DATABRICKS_URL - # We explicitly specify /Workspace/Users/$USERNAME to make sure we only have a single copy. - root_path: /Workspace/Users/$USERNAME/.bundle/${bundle.name}/${bundle.target} + host: [DATABRICKS_URL] + # We explicitly specify /Workspace/Users/[USERNAME] to make sure we only have a single copy. + root_path: /Workspace/Users/[USERNAME]/.bundle/${bundle.name}/${bundle.target} permissions: - - user_name: $USERNAME + - user_name: [USERNAME] level: CAN_MANAGE run_as: - user_name: $USERNAME + user_name: [USERNAME] diff --git a/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/profile_template.yml b/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/profile_template.yml index 5e0f0fc29..bdb41ab20 100644 --- a/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/profile_template.yml +++ b/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/profile_template.yml @@ -5,7 +5,7 @@ fixed: type: databricks prompts: host: - default: $DATABRICKS_HOST + default: [DATABRICKS_HOST] token: hint: 'personal access token to use, dapiXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' hide_input: true @@ -16,7 +16,7 @@ prompts: hint: 'initial catalog' default: main schema: - hint: 'personal schema where dbt will build objects during development, example: $USERNAME' + hint: 'personal schema where dbt will build objects during development, example: [USERNAME]' threads: hint: 'threads to use during development, 1 or more' type: 'int' diff --git a/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/resources/my_dbt_sql.job.yml b/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/resources/my_dbt_sql.job.yml index d52f8ed50..b522931f9 100644 --- a/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/resources/my_dbt_sql.job.yml +++ b/acceptance/bundle/templates/dbt-sql/output/my_dbt_sql/resources/my_dbt_sql.job.yml @@ -11,7 +11,7 @@ resources: email_notifications: on_failure: - - $USERNAME + - [USERNAME] tasks: diff --git a/acceptance/bundle/templates/default-python/output.txt b/acceptance/bundle/templates/default-python/output.txt index 5493ac2cf..930e756de 100644 --- a/acceptance/bundle/templates/default-python/output.txt +++ b/acceptance/bundle/templates/default-python/output.txt @@ -1,30 +1,30 @@ ->>> $CLI bundle init default-python --config-file ./input.json --output-dir output +>>> [CLI] bundle init default-python --config-file ./input.json --output-dir output Welcome to the default Python template for Databricks Asset Bundles! -Workspace to use (auto-detected, edit in 'my_default_python/databricks.yml'): $DATABRICKS_URL +Workspace to use (auto-detected, edit in 'my_default_python/databricks.yml'): [DATABRICKS_URL] ✨ Your new project has been created in the 'my_default_python' directory! Please refer to the README.md file for "getting started" instructions. See also the documentation at https://docs.databricks.com/dev-tools/bundles/index.html. ->>> $CLI bundle validate -t dev +>>> [CLI] bundle validate -t dev Name: my_default_python Target: dev Workspace: - Host: $DATABRICKS_URL - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/my_default_python/dev + Host: [DATABRICKS_URL] + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/my_default_python/dev Validation OK! ->>> $CLI bundle validate -t prod +>>> [CLI] bundle validate -t prod Name: my_default_python Target: prod Workspace: - Host: $DATABRICKS_URL - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/my_default_python/prod + Host: [DATABRICKS_URL] + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/my_default_python/prod Validation OK! diff --git a/acceptance/bundle/templates/default-python/output/my_default_python/databricks.yml b/acceptance/bundle/templates/default-python/output/my_default_python/databricks.yml index 3fa777219..6df75c209 100644 --- a/acceptance/bundle/templates/default-python/output/my_default_python/databricks.yml +++ b/acceptance/bundle/templates/default-python/output/my_default_python/databricks.yml @@ -16,16 +16,16 @@ targets: mode: development default: true workspace: - host: $DATABRICKS_URL + host: [DATABRICKS_URL] prod: mode: production workspace: - host: $DATABRICKS_URL - # We explicitly specify /Workspace/Users/$USERNAME to make sure we only have a single copy. - root_path: /Workspace/Users/$USERNAME/.bundle/${bundle.name}/${bundle.target} + host: [DATABRICKS_URL] + # We explicitly specify /Workspace/Users/[USERNAME] to make sure we only have a single copy. + root_path: /Workspace/Users/[USERNAME]/.bundle/${bundle.name}/${bundle.target} permissions: - - user_name: $USERNAME + - user_name: [USERNAME] level: CAN_MANAGE run_as: - user_name: $USERNAME + user_name: [USERNAME] diff --git a/acceptance/bundle/templates/default-python/output/my_default_python/resources/my_default_python.job.yml b/acceptance/bundle/templates/default-python/output/my_default_python/resources/my_default_python.job.yml index e6148a4ad..d9e31691a 100644 --- a/acceptance/bundle/templates/default-python/output/my_default_python/resources/my_default_python.job.yml +++ b/acceptance/bundle/templates/default-python/output/my_default_python/resources/my_default_python.job.yml @@ -12,7 +12,7 @@ resources: email_notifications: on_failure: - - $USERNAME + - [USERNAME] tasks: - task_key: notebook_task diff --git a/acceptance/bundle/templates/default-python/output/my_default_python/setup.py b/acceptance/bundle/templates/default-python/output/my_default_python/setup.py index 84b24ecb8..548f1035e 100644 --- a/acceptance/bundle/templates/default-python/output/my_default_python/setup.py +++ b/acceptance/bundle/templates/default-python/output/my_default_python/setup.py @@ -23,7 +23,7 @@ setup( # to ensure that changes to wheel package are picked up when used on all-purpose clusters version=my_default_python.__version__ + "+" + local_version, url="https://databricks.com", - author="$USERNAME", + author="[USERNAME]", description="wheel file based on my_default_python/src", packages=find_packages(where="./src"), package_dir={"": "src"}, diff --git a/acceptance/bundle/templates/default-sql/output.txt b/acceptance/bundle/templates/default-sql/output.txt index fe0139093..06eff962b 100644 --- a/acceptance/bundle/templates/default-sql/output.txt +++ b/acceptance/bundle/templates/default-sql/output.txt @@ -1,32 +1,32 @@ ->>> $CLI bundle init default-sql --config-file ./input.json --output-dir output +>>> [CLI] bundle init default-sql --config-file ./input.json --output-dir output Welcome to the default SQL template for Databricks Asset Bundles! A workspace was selected based on your current profile. For information about how to change this, see https://docs.databricks.com/dev-tools/cli/profiles.html. -workspace_host: $DATABRICKS_URL +workspace_host: [DATABRICKS_URL] ✨ Your new project has been created in the 'my_default_sql' directory! Please refer to the README.md file for "getting started" instructions. See also the documentation at https://docs.databricks.com/dev-tools/bundles/index.html. ->>> $CLI bundle validate -t dev +>>> [CLI] bundle validate -t dev Name: my_default_sql Target: dev Workspace: - Host: $DATABRICKS_URL - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/my_default_sql/dev + Host: [DATABRICKS_URL] + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/my_default_sql/dev Validation OK! ->>> $CLI bundle validate -t prod +>>> [CLI] bundle validate -t prod Name: my_default_sql Target: prod Workspace: - Host: $DATABRICKS_URL - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/my_default_sql/prod + Host: [DATABRICKS_URL] + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/my_default_sql/prod Validation OK! diff --git a/acceptance/bundle/templates/default-sql/output/my_default_sql/databricks.yml b/acceptance/bundle/templates/default-sql/output/my_default_sql/databricks.yml index 16292bc84..6ef09cf3b 100644 --- a/acceptance/bundle/templates/default-sql/output/my_default_sql/databricks.yml +++ b/acceptance/bundle/templates/default-sql/output/my_default_sql/databricks.yml @@ -25,7 +25,7 @@ targets: mode: development default: true workspace: - host: $DATABRICKS_URL + host: [DATABRICKS_URL] variables: warehouse_id: f00dcafe catalog: main @@ -34,15 +34,15 @@ targets: prod: mode: production workspace: - host: $DATABRICKS_URL - # We explicitly specify /Workspace/Users/$USERNAME to make sure we only have a single copy. - root_path: /Workspace/Users/$USERNAME/.bundle/${bundle.name}/${bundle.target} + host: [DATABRICKS_URL] + # We explicitly specify /Workspace/Users/[USERNAME] to make sure we only have a single copy. + root_path: /Workspace/Users/[USERNAME]/.bundle/${bundle.name}/${bundle.target} variables: warehouse_id: f00dcafe catalog: main schema: default permissions: - - user_name: $USERNAME + - user_name: [USERNAME] level: CAN_MANAGE run_as: - user_name: $USERNAME + user_name: [USERNAME] diff --git a/acceptance/bundle/templates/default-sql/output/my_default_sql/resources/my_default_sql_sql.job.yml b/acceptance/bundle/templates/default-sql/output/my_default_sql/resources/my_default_sql_sql.job.yml index 86de0f9db..34d60e3d5 100644 --- a/acceptance/bundle/templates/default-sql/output/my_default_sql/resources/my_default_sql_sql.job.yml +++ b/acceptance/bundle/templates/default-sql/output/my_default_sql/resources/my_default_sql_sql.job.yml @@ -12,7 +12,7 @@ resources: email_notifications: on_failure: - - $USERNAME + - [USERNAME] parameters: - name: catalog diff --git a/acceptance/bundle/templates/experimental-jobs-as-code/output.txt b/acceptance/bundle/templates/experimental-jobs-as-code/output.txt index 10aca003e..984dad604 100644 --- a/acceptance/bundle/templates/experimental-jobs-as-code/output.txt +++ b/acceptance/bundle/templates/experimental-jobs-as-code/output.txt @@ -1,28 +1,28 @@ ->>> $CLI bundle init experimental-jobs-as-code --config-file ./input.json --output-dir output +>>> [CLI] bundle init experimental-jobs-as-code --config-file ./input.json --output-dir output Welcome to (EXPERIMENTAL) "Jobs as code" template for Databricks Asset Bundles! -Workspace to use (auto-detected, edit in 'my_jobs_as_code/databricks.yml'): $DATABRICKS_URL +Workspace to use (auto-detected, edit in 'my_jobs_as_code/databricks.yml'): [DATABRICKS_URL] ✨ Your new project has been created in the 'my_jobs_as_code' directory! Please refer to the README.md file for "getting started" instructions. See also the documentation at https://docs.databricks.com/dev-tools/bundles/index.html. ->>> $CLI bundle validate -t dev --output json -Warning: Ignoring Databricks CLI version constraint for development build. Required: >= 0.238.0, current: $DEV_VERSION +>>> [CLI] bundle validate -t dev --output json +Warning: Ignoring Databricks CLI version constraint for development build. Required: >= 0.238.0, current: [DEV_VERSION] { "jobs": { "my_jobs_as_code_job": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/my_jobs_as_code/dev/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/my_jobs_as_code/dev/state/metadata.json" }, "edit_mode": "UI_LOCKED", "email_notifications": { "on_failure": [ - "$USERNAME" + "[USERNAME]" ] }, "format": "MULTI_TASK", @@ -40,19 +40,19 @@ Warning: Ignoring Databricks CLI version constraint for development build. Requi } ], "max_concurrent_runs": 4, - "name": "[dev $USERNAME] my_jobs_as_code_job", + "name": "[dev [USERNAME]] my_jobs_as_code_job", "permissions": [], "queue": { "enabled": true }, "tags": { - "dev": "$USERNAME" + "dev": "[USERNAME]" }, "tasks": [ { "job_cluster_key": "job_cluster", "notebook_task": { - "notebook_path": "/Workspace/Users/$USERNAME/.bundle/my_jobs_as_code/dev/files/src/notebook" + "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/my_jobs_as_code/dev/files/src/notebook" }, "task_key": "notebook_task" }, diff --git a/acceptance/bundle/templates/experimental-jobs-as-code/output/my_jobs_as_code/databricks.yml b/acceptance/bundle/templates/experimental-jobs-as-code/output/my_jobs_as_code/databricks.yml index 54e69a256..9299c96e8 100644 --- a/acceptance/bundle/templates/experimental-jobs-as-code/output/my_jobs_as_code/databricks.yml +++ b/acceptance/bundle/templates/experimental-jobs-as-code/output/my_jobs_as_code/databricks.yml @@ -34,16 +34,16 @@ targets: mode: development default: true workspace: - host: $DATABRICKS_URL + host: [DATABRICKS_URL] prod: mode: production workspace: - host: $DATABRICKS_URL - # We explicitly specify /Workspace/Users/$USERNAME to make sure we only have a single copy. - root_path: /Workspace/Users/$USERNAME/.bundle/${bundle.name}/${bundle.target} + host: [DATABRICKS_URL] + # We explicitly specify /Workspace/Users/[USERNAME] to make sure we only have a single copy. + root_path: /Workspace/Users/[USERNAME]/.bundle/${bundle.name}/${bundle.target} permissions: - - user_name: $USERNAME + - user_name: [USERNAME] level: CAN_MANAGE run_as: - user_name: $USERNAME + user_name: [USERNAME] diff --git a/acceptance/bundle/templates/experimental-jobs-as-code/output/my_jobs_as_code/resources/my_jobs_as_code_job.py b/acceptance/bundle/templates/experimental-jobs-as-code/output/my_jobs_as_code/resources/my_jobs_as_code_job.py index 4854d656f..e8406fd7b 100644 --- a/acceptance/bundle/templates/experimental-jobs-as-code/output/my_jobs_as_code/resources/my_jobs_as_code_job.py +++ b/acceptance/bundle/templates/experimental-jobs-as-code/output/my_jobs_as_code/resources/my_jobs_as_code_job.py @@ -17,7 +17,7 @@ my_jobs_as_code_job = Job.from_dict( }, "email_notifications": { "on_failure": [ - "$USERNAME", + "[USERNAME]", ], }, "tasks": [ diff --git a/acceptance/bundle/templates/wrong-url/output.txt b/acceptance/bundle/templates/wrong-url/output.txt index b78cf4b68..6b4f9c459 100644 --- a/acceptance/bundle/templates/wrong-url/output.txt +++ b/acceptance/bundle/templates/wrong-url/output.txt @@ -1,4 +1,4 @@ -Error: git clone failed: git clone https://invalid-domain-123.databricks.com/hello/world $TMPDIR_GPARENT/world-123456 --no-tags --depth=1: exit status 128. Cloning into '$TMPDIR_GPARENT/world-123456'... +Error: git clone failed: git clone https://invalid-domain-123.databricks.com/hello/world [TMPDIR]_GPARENT/world-123456 --no-tags --depth=1: exit status 128. Cloning into '[TMPDIR]_GPARENT/world-123456'... fatal: unable to access 'https://invalid-domain-123.databricks.com/hello/world/': Could not resolve host: invalid-domain-123.databricks.com diff --git a/acceptance/bundle/variables/arg-repeat/output.txt b/acceptance/bundle/variables/arg-repeat/output.txt index 2f9de1a3c..4b97d70a1 100644 --- a/acceptance/bundle/variables/arg-repeat/output.txt +++ b/acceptance/bundle/variables/arg-repeat/output.txt @@ -1,5 +1,5 @@ ->>> errcode $CLI bundle validate --var a=one -o json +>>> errcode [CLI] bundle validate --var a=one -o json { "a": { "default": "hello", @@ -7,7 +7,7 @@ } } ->>> errcode $CLI bundle validate --var a=one --var a=two +>>> errcode [CLI] bundle validate --var a=one --var a=two Error: failed to assign two to a: variable has already been assigned value: one Name: arg-repeat diff --git a/acceptance/bundle/variables/complex-cycle-self/output.txt b/acceptance/bundle/variables/complex-cycle-self/output.txt index fa80154ca..7447de349 100644 --- a/acceptance/bundle/variables/complex-cycle-self/output.txt +++ b/acceptance/bundle/variables/complex-cycle-self/output.txt @@ -3,7 +3,7 @@ Warning: Detected unresolved variables after 11 resolution rounds Name: cycle Target: default Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/cycle/default + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/cycle/default Found 1 warning diff --git a/acceptance/bundle/variables/complex-cycle/output.txt b/acceptance/bundle/variables/complex-cycle/output.txt index fa80154ca..7447de349 100644 --- a/acceptance/bundle/variables/complex-cycle/output.txt +++ b/acceptance/bundle/variables/complex-cycle/output.txt @@ -3,7 +3,7 @@ Warning: Detected unresolved variables after 11 resolution rounds Name: cycle Target: default Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/cycle/default + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/cycle/default Found 1 warning diff --git a/acceptance/bundle/variables/complex/out.default.json b/acceptance/bundle/variables/complex/out.default.json index a1ccd52bc..0804ad588 100644 --- a/acceptance/bundle/variables/complex/out.default.json +++ b/acceptance/bundle/variables/complex/out.default.json @@ -4,7 +4,7 @@ "my_job": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/complex-variables/default/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/complex-variables/default/state/metadata.json" }, "edit_mode": "UI_LOCKED", "format": "MULTI_TASK", diff --git a/acceptance/bundle/variables/complex/out.dev.json b/acceptance/bundle/variables/complex/out.dev.json index bb939091b..e93c2c297 100644 --- a/acceptance/bundle/variables/complex/out.dev.json +++ b/acceptance/bundle/variables/complex/out.dev.json @@ -4,7 +4,7 @@ "my_job": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/complex-variables/dev/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/complex-variables/dev/state/metadata.json" }, "edit_mode": "UI_LOCKED", "format": "MULTI_TASK", diff --git a/acceptance/bundle/variables/complex/output.txt b/acceptance/bundle/variables/complex/output.txt index ce295421f..f1d4c04cc 100644 --- a/acceptance/bundle/variables/complex/output.txt +++ b/acceptance/bundle/variables/complex/output.txt @@ -1,10 +1,10 @@ ->>> $CLI bundle validate -o json +>>> [CLI] bundle validate -o json >>> jq .resources.jobs.my_job.tasks[0].task_key out.default.json "task with spark version 13.2.x-scala2.11 and jar /path/to/jar" ->>> $CLI bundle validate -o json -t dev +>>> [CLI] bundle validate -o json -t dev >>> jq .resources.jobs.my_job.tasks[0].task_key out.dev.json "task with spark version 14.2.x-scala2.11 and jar /newpath/to/jar" diff --git a/acceptance/bundle/variables/complex_multiple_files/output.txt b/acceptance/bundle/variables/complex_multiple_files/output.txt index ec2cad1ce..433e6da0c 100644 --- a/acceptance/bundle/variables/complex_multiple_files/output.txt +++ b/acceptance/bundle/variables/complex_multiple_files/output.txt @@ -4,7 +4,7 @@ "my_job": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/complex-variables-multiple-files/dev/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/complex-variables-multiple-files/dev/state/metadata.json" }, "edit_mode": "UI_LOCKED", "format": "MULTI_TASK", diff --git a/acceptance/bundle/variables/double_underscore/output.txt b/acceptance/bundle/variables/double_underscore/output.txt index 45529038d..0124f5442 100644 --- a/acceptance/bundle/variables/double_underscore/output.txt +++ b/acceptance/bundle/variables/double_underscore/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle validate -o json +>>> [CLI] bundle validate -o json [ { "task_key": "test default" diff --git a/acceptance/bundle/variables/empty/output.txt b/acceptance/bundle/variables/empty/output.txt index 8933443df..cbd0f1989 100644 --- a/acceptance/bundle/variables/empty/output.txt +++ b/acceptance/bundle/variables/empty/output.txt @@ -3,8 +3,8 @@ Error: no value assigned to required variable a. Assignment can be done using "- Name: empty${var.a} Target: default Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/empty${var.a}/default + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/empty${var.a}/default Found 1 error diff --git a/acceptance/bundle/variables/env_overrides/output.txt b/acceptance/bundle/variables/env_overrides/output.txt index 06e6e518b..93b3b6716 100644 --- a/acceptance/bundle/variables/env_overrides/output.txt +++ b/acceptance/bundle/variables/env_overrides/output.txt @@ -1,27 +1,27 @@ ->>> $CLI bundle validate -t env-with-single-variable-override -o json +>>> [CLI] bundle validate -t env-with-single-variable-override -o json "default-a dev-b" ->>> $CLI bundle validate -t env-with-two-variable-overrides -o json +>>> [CLI] bundle validate -t env-with-two-variable-overrides -o json "prod-a prod-b" ->>> BUNDLE_VAR_b=env-var-b $CLI bundle validate -t env-with-two-variable-overrides -o json +>>> BUNDLE_VAR_b=env-var-b [CLI] bundle validate -t env-with-two-variable-overrides -o json "prod-a env-var-b" ->>> errcode $CLI bundle validate -t env-missing-a-required-variable-assignment +>>> errcode [CLI] bundle validate -t env-missing-a-required-variable-assignment Error: no value assigned to required variable b. Assignment can be done using "--var", by setting the BUNDLE_VAR_b environment variable, or in .databricks/bundle//variable-overrides.json file Name: test bundle Target: env-missing-a-required-variable-assignment Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/test bundle/env-missing-a-required-variable-assignment + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/test bundle/env-missing-a-required-variable-assignment Found 1 error Exit code: 1 ->>> errcode $CLI bundle validate -t env-using-an-undefined-variable +>>> errcode [CLI] bundle validate -t env-using-an-undefined-variable Error: variable c is not defined but is assigned a value Name: test bundle @@ -30,7 +30,7 @@ Found 1 error Exit code: 1 ->>> $CLI bundle validate -t env-overrides-lookup -o json +>>> [CLI] bundle validate -t env-overrides-lookup -o json { "a": "default-a", "b": "prod-b", diff --git a/acceptance/bundle/variables/file-defaults/output.txt b/acceptance/bundle/variables/file-defaults/output.txt index 5b01a1b66..234ddcbbd 100644 --- a/acceptance/bundle/variables/file-defaults/output.txt +++ b/acceptance/bundle/variables/file-defaults/output.txt @@ -1,6 +1,6 @@ === variable file ->>> $CLI bundle validate -o json +>>> [CLI] bundle validate -o json { "job_cluster_key": "mlops_stacks-cluster", "new_cluster": { @@ -10,7 +10,7 @@ } === variable file and variable flag ->>> $CLI bundle validate -o json --var=cluster_key=mlops_stacks-cluster-overriden +>>> [CLI] bundle validate -o json --var=cluster_key=mlops_stacks-cluster-overriden { "job_cluster_key": "mlops_stacks-cluster-overriden", "new_cluster": { @@ -20,7 +20,7 @@ } === variable file and environment variable ->>> BUNDLE_VAR_cluster_key=mlops_stacks-cluster-overriden $CLI bundle validate -o json +>>> BUNDLE_VAR_cluster_key=mlops_stacks-cluster-overriden [CLI] bundle validate -o json { "job_cluster_key": "mlops_stacks-cluster-overriden", "new_cluster": { @@ -30,7 +30,7 @@ } === variable has value in config file ->>> $CLI bundle validate -o json --target with_value +>>> [CLI] bundle validate -o json --target with_value { "job_cluster_key": "mlops_stacks-cluster-from-file", "new_cluster": { @@ -40,8 +40,8 @@ } === file cannot be parsed ->>> errcode $CLI bundle validate -o json --target invalid_json -Error: failed to parse variables file $TMPDIR/.databricks/bundle/invalid_json/variable-overrides.json: error decoding JSON at :0:0: invalid character 'o' in literal false (expecting 'a') +>>> errcode [CLI] bundle validate -o json --target invalid_json +Error: failed to parse variables file [TMPDIR]/.databricks/bundle/invalid_json/variable-overrides.json: error decoding JSON at :0:0: invalid character 'o' in literal false (expecting 'a') Exit code: 1 @@ -54,8 +54,8 @@ Exit code: 1 } === file has wrong structure ->>> errcode $CLI bundle validate -o json --target wrong_file_structure -Error: failed to parse variables file $TMPDIR/.databricks/bundle/wrong_file_structure/variable-overrides.json: invalid format +>>> errcode [CLI] bundle validate -o json --target wrong_file_structure +Error: failed to parse variables file [TMPDIR]/.databricks/bundle/wrong_file_structure/variable-overrides.json: invalid format Variables file must be a JSON object with the following format: {"var1": "value1", "var2": "value2"} @@ -71,7 +71,7 @@ Exit code: 1 } === file has variable that is complex but default is string ->>> errcode $CLI bundle validate -o json --target complex_to_string +>>> errcode [CLI] bundle validate -o json --target complex_to_string Error: variable cluster_key is not of type complex, but the value in the variable file is a complex type @@ -85,7 +85,7 @@ Exit code: 1 } === file has variable that is string but default is complex ->>> errcode $CLI bundle validate -o json --target string_to_complex +>>> errcode [CLI] bundle validate -o json --target string_to_complex Error: variable cluster is of type complex, but the value in the variable file is not a complex type @@ -99,7 +99,7 @@ Exit code: 1 } === variable is required but it's not provided in the file ->>> errcode $CLI bundle validate -o json --target without_defaults +>>> errcode [CLI] bundle validate -o json --target without_defaults Error: no value assigned to required variable cluster. Assignment can be done using "--var", by setting the BUNDLE_VAR_cluster environment variable, or in .databricks/bundle//variable-overrides.json file diff --git a/acceptance/bundle/variables/file-defaults/test.toml b/acceptance/bundle/variables/file-defaults/test.toml index 08403b606..da8854775 100644 --- a/acceptance/bundle/variables/file-defaults/test.toml +++ b/acceptance/bundle/variables/file-defaults/test.toml @@ -1,8 +1,4 @@ # Fix for windows [[Repls]] -Old = '\$TMPDIR\\.databricks\\bundle\\wrong_file_structure\\variable-overrides.json' -New = '$$TMPDIR/.databricks/bundle/wrong_file_structure/variable-overrides.json' - -[[Repls]] -Old = '\$TMPDIR\\.databricks\\bundle\\invalid_json\\variable-overrides.json' -New = '$$TMPDIR/.databricks/bundle/invalid_json/variable-overrides.json' +Old = '\\' +New = '/' diff --git a/acceptance/bundle/variables/git-branch/output.txt b/acceptance/bundle/variables/git-branch/output.txt index 21ed9e7de..68f27a3f1 100644 --- a/acceptance/bundle/variables/git-branch/output.txt +++ b/acceptance/bundle/variables/git-branch/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle validate -o json +>>> [CLI] bundle validate -o json { "bundle": { "environment": "prod", @@ -11,7 +11,7 @@ "name": "git", "target": "prod", "terraform": { - "exec_path": "$TERRAFORM" + "exec_path": "[TERRAFORM]" } }, "sync": { @@ -27,24 +27,24 @@ } }, "workspace": { - "artifact_path": "/Workspace/Users/$USERNAME/.bundle/git/prod/artifacts", - "file_path": "/Workspace/Users/$USERNAME/.bundle/git/prod/files", - "resource_path": "/Workspace/Users/$USERNAME/.bundle/git/prod/resources", - "root_path": "/Workspace/Users/$USERNAME/.bundle/git/prod", - "state_path": "/Workspace/Users/$USERNAME/.bundle/git/prod/state" + "artifact_path": "/Workspace/Users/[USERNAME]/.bundle/git/prod/artifacts", + "file_path": "/Workspace/Users/[USERNAME]/.bundle/git/prod/files", + "resource_path": "/Workspace/Users/[USERNAME]/.bundle/git/prod/resources", + "root_path": "/Workspace/Users/[USERNAME]/.bundle/git/prod", + "state_path": "/Workspace/Users/[USERNAME]/.bundle/git/prod/state" } } ->>> $CLI bundle validate +>>> [CLI] bundle validate Name: git Target: prod Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/git/prod + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/git/prod Validation OK! ->>> $CLI bundle validate -o json -t dev +>>> [CLI] bundle validate -o json -t dev { "bundle": { "environment": "dev", @@ -56,7 +56,7 @@ Validation OK! "name": "git", "target": "dev", "terraform": { - "exec_path": "$TERRAFORM" + "exec_path": "[TERRAFORM]" } }, "sync": { @@ -72,19 +72,19 @@ Validation OK! } }, "workspace": { - "artifact_path": "/Workspace/Users/$USERNAME/.bundle/git/dev/artifacts", - "file_path": "/Workspace/Users/$USERNAME/.bundle/git/dev/files", - "resource_path": "/Workspace/Users/$USERNAME/.bundle/git/dev/resources", - "root_path": "/Workspace/Users/$USERNAME/.bundle/git/dev", - "state_path": "/Workspace/Users/$USERNAME/.bundle/git/dev/state" + "artifact_path": "/Workspace/Users/[USERNAME]/.bundle/git/dev/artifacts", + "file_path": "/Workspace/Users/[USERNAME]/.bundle/git/dev/files", + "resource_path": "/Workspace/Users/[USERNAME]/.bundle/git/dev/resources", + "root_path": "/Workspace/Users/[USERNAME]/.bundle/git/dev", + "state_path": "/Workspace/Users/[USERNAME]/.bundle/git/dev/state" } } ->>> $CLI bundle validate -t dev +>>> [CLI] bundle validate -t dev Name: git Target: dev Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/git/dev + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/git/dev Validation OK! diff --git a/acceptance/bundle/variables/host/output.txt b/acceptance/bundle/variables/host/output.txt index 89342908c..63c41426a 100644 --- a/acceptance/bundle/variables/host/output.txt +++ b/acceptance/bundle/variables/host/output.txt @@ -1,5 +1,5 @@ ->>> errcode $CLI bundle validate -o json +>>> errcode [CLI] bundle validate -o json Error: failed during request visitor: parse "https://${var.host}": invalid character "{" in host name { @@ -25,7 +25,7 @@ Error: failed during request visitor: parse "https://${var.host}": invalid chara } Exit code: 1 ->>> errcode $CLI bundle validate +>>> errcode [CLI] bundle validate Error: failed during request visitor: parse "https://${var.host}": invalid character "{" in host name Name: host diff --git a/acceptance/bundle/variables/prepend-workspace-var/output.txt b/acceptance/bundle/variables/prepend-workspace-var/output.txt index 93b652894..a48a58fba 100644 --- a/acceptance/bundle/variables/prepend-workspace-var/output.txt +++ b/acceptance/bundle/variables/prepend-workspace-var/output.txt @@ -7,7 +7,7 @@ }, "target": "dev", "terraform": { - "exec_path": "$TERRAFORM" + "exec_path": "[TERRAFORM]" } }, "resources": { @@ -15,7 +15,7 @@ "my_job": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Users/$USERNAME/path/to/root/state/metadata.json" + "metadata_file_path": "/Users/[USERNAME]/path/to/root/state/metadata.json" }, "edit_mode": "UI_LOCKED", "format": "MULTI_TASK", @@ -29,7 +29,7 @@ "existing_cluster_id": "500", "python_wheel_task": { "named_parameters": { - "conf-file": "/Users/$USERNAME/path/to/root/files/path/to/config.yaml" + "conf-file": "/Users/[USERNAME]/path/to/root/files/path/to/config.yaml" } }, "task_key": "" @@ -46,17 +46,17 @@ "targets": null, "variables": { "workspace_root": { - "default": "/Users/$USERNAME", + "default": "/Users/[USERNAME]", "description": "root directory in the Databricks workspace to store the asset bundle and associated artifacts", - "value": "/Users/$USERNAME" + "value": "/Users/[USERNAME]" } }, "workspace": { - "artifact_path": "/Users/$USERNAME/path/to/root/artifacts", - "file_path": "/Users/$USERNAME/path/to/root/files", + "artifact_path": "/Users/[USERNAME]/path/to/root/artifacts", + "file_path": "/Users/[USERNAME]/path/to/root/files", "profile": "profile_name", - "resource_path": "/Users/$USERNAME/path/to/root/resources", - "root_path": "/Users/$USERNAME/path/to/root", - "state_path": "/Users/$USERNAME/path/to/root/state" + "resource_path": "/Users/[USERNAME]/path/to/root/resources", + "root_path": "/Users/[USERNAME]/path/to/root", + "state_path": "/Users/[USERNAME]/path/to/root/state" } } diff --git a/acceptance/bundle/variables/resolve-nonstrings/output.txt b/acceptance/bundle/variables/resolve-nonstrings/output.txt index 3a1eb9c47..951ad7a0d 100644 --- a/acceptance/bundle/variables/resolve-nonstrings/output.txt +++ b/acceptance/bundle/variables/resolve-nonstrings/output.txt @@ -20,7 +20,7 @@ "job1": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/TestResolveVariableReferencesForPrimitiveNonStringFields/default/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/TestResolveVariableReferencesForPrimitiveNonStringFields/default/state/metadata.json" }, "edit_mode": "UI_LOCKED", "format": "MULTI_TASK", diff --git a/acceptance/bundle/variables/vanilla/output.txt b/acceptance/bundle/variables/vanilla/output.txt index e98882bb0..3958c39b9 100644 --- a/acceptance/bundle/variables/vanilla/output.txt +++ b/acceptance/bundle/variables/vanilla/output.txt @@ -1,15 +1,15 @@ ->>> BUNDLE_VAR_b=def $CLI bundle validate -o json +>>> BUNDLE_VAR_b=def [CLI] bundle validate -o json "abc def" ->>> errcode $CLI bundle validate +>>> errcode [CLI] bundle validate Error: no value assigned to required variable b. Assignment can be done using "--var", by setting the BUNDLE_VAR_b environment variable, or in .databricks/bundle//variable-overrides.json file Name: ${var.a} ${var.b} Target: default Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/${var.a} ${var.b}/default + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/${var.a} ${var.b}/default Found 1 error diff --git a/acceptance/bundle/variables/variable_overrides_in_target/output.txt b/acceptance/bundle/variables/variable_overrides_in_target/output.txt index 8998b691d..d112cf2de 100644 --- a/acceptance/bundle/variables/variable_overrides_in_target/output.txt +++ b/acceptance/bundle/variables/variable_overrides_in_target/output.txt @@ -1,5 +1,5 @@ ->>> $CLI bundle validate -o json -t use-default-variable-values +>>> [CLI] bundle validate -o json -t use-default-variable-values { "pipelines": { "my_pipeline": { @@ -12,7 +12,7 @@ "continuous": true, "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/foobar/use-default-variable-values/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/foobar/use-default-variable-values/state/metadata.json" }, "name": "a_string", "permissions": [] @@ -20,7 +20,7 @@ } } ->>> $CLI bundle validate -o json -t override-string-variable +>>> [CLI] bundle validate -o json -t override-string-variable { "pipelines": { "my_pipeline": { @@ -33,7 +33,7 @@ "continuous": true, "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/foobar/override-string-variable/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/foobar/override-string-variable/state/metadata.json" }, "name": "overridden_string", "permissions": [] @@ -41,7 +41,7 @@ } } ->>> $CLI bundle validate -o json -t override-int-variable +>>> [CLI] bundle validate -o json -t override-int-variable { "pipelines": { "my_pipeline": { @@ -54,7 +54,7 @@ "continuous": true, "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/foobar/override-int-variable/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/foobar/override-int-variable/state/metadata.json" }, "name": "a_string", "permissions": [] @@ -62,7 +62,7 @@ } } ->>> $CLI bundle validate -o json -t override-both-bool-and-string-variables +>>> [CLI] bundle validate -o json -t override-both-bool-and-string-variables { "pipelines": { "my_pipeline": { @@ -75,7 +75,7 @@ "continuous": false, "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/foobar/override-both-bool-and-string-variables/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/foobar/override-both-bool-and-string-variables/state/metadata.json" }, "name": "overridden_string", "permissions": [] diff --git a/acceptance/selftest/output.txt b/acceptance/selftest/output.txt index 91aa8c33e..cadbdebb5 100644 --- a/acceptance/selftest/output.txt +++ b/acceptance/selftest/output.txt @@ -18,13 +18,13 @@ Exit code: 7 === Capturing pwd >>> python3 -c import os; print(os.getcwd()) -$TMPDIR +[TMPDIR] === Capturing subdir >>> mkdir -p subdir/a/b/c >>> withdir subdir/a/b/c python3 -c import os; print(os.getcwd()) -$TMPDIR/subdir/a/b/c +[TMPDIR]/subdir/a/b/c === Custom output files - everything starting with out is captured and compared >>> echo HELLO @@ -35,5 +35,5 @@ CUSTOM_NUMBER_REGEX 123456 === Testing --version ->>> $CLI --version -Databricks CLI v$DEV_VERSION +>>> [CLI] --version +Databricks CLI v[DEV_VERSION] diff --git a/acceptance/selftest/test.toml b/acceptance/selftest/test.toml index 9607ec5df..762e28ceb 100644 --- a/acceptance/selftest/test.toml +++ b/acceptance/selftest/test.toml @@ -16,5 +16,5 @@ New = "CUSTOM_NUMBER_REGEX" [[Repls]] # Fix path with reverse slashes in the output for Windows. -Old = '\$TMPDIR\\subdir\\a\\b\\c' -New = '$$TMPDIR/subdir/a/b/c' +Old = 'TMPDIR]\\subdir\\a\\b\\c' +New = 'TMPDIR]/subdir/a/b/c' diff --git a/acceptance/terraform/output.txt b/acceptance/terraform/output.txt index 32589ddab..6bdc809f6 100644 --- a/acceptance/terraform/output.txt +++ b/acceptance/terraform/output.txt @@ -1,5 +1,5 @@ ->>> $TERRAFORM init -no-color -get=false +>>> [TERRAFORM] init -no-color -get=false Initializing the backend... @@ -35,12 +35,12 @@ If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. ->>> $TERRAFORM plan -no-color +>>> [TERRAFORM] plan -no-color data.databricks_current_user.me: Reading... -data.databricks_current_user.me: Read complete after (redacted) [id=$USER.Id] +data.databricks_current_user.me: Read complete after (redacted) [id=[USERID]] Changes to Outputs: - + username = "$USERNAME" + + username = "[USERNAME]" You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure. diff --git a/acceptance/workspace/jobs/create/output.txt b/acceptance/workspace/jobs/create/output.txt index a9487fe5b..50b823aa0 100644 --- a/acceptance/workspace/jobs/create/output.txt +++ b/acceptance/workspace/jobs/create/output.txt @@ -1,5 +1,5 @@ ->>> $CLI jobs create --json {"name":"abc"} +>>> [CLI] jobs create --json {"name":"abc"} { "job_id":1111 } diff --git a/integration/bundle/testdata/apps/bundle_deploy.txt b/integration/bundle/testdata/apps/bundle_deploy.txt index 211164174..437a55596 100644 --- a/integration/bundle/testdata/apps/bundle_deploy.txt +++ b/integration/bundle/testdata/apps/bundle_deploy.txt @@ -1,4 +1,4 @@ -Uploading bundle files to /Workspace/Users/$USERNAME/.bundle/$UNIQUE_PRJ/files... +Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/$UNIQUE_PRJ/files... Deploying resources... Updating deployment state... Deployment complete! diff --git a/integration/bundle/testdata/apps/bundle_validate.txt b/integration/bundle/testdata/apps/bundle_validate.txt index dc9016a0f..567fafd24 100644 --- a/integration/bundle/testdata/apps/bundle_validate.txt +++ b/integration/bundle/testdata/apps/bundle_validate.txt @@ -1,7 +1,7 @@ Name: basic Target: default Workspace: - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/$UNIQUE_PRJ + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/$UNIQUE_PRJ Validation OK! diff --git a/integration/bundle/testdata/default_python/bundle_deploy.txt b/integration/bundle/testdata/default_python/bundle_deploy.txt index d7b8cede9..076e7618f 100644 --- a/integration/bundle/testdata/default_python/bundle_deploy.txt +++ b/integration/bundle/testdata/default_python/bundle_deploy.txt @@ -1,6 +1,6 @@ Building project_name_$UNIQUE_PRJ... Uploading project_name_$UNIQUE_PRJ-0.0.1+[NUMID].[NUMID]-py3-none-any.whl... -Uploading bundle files to /Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev/files... +Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev/files... Deploying resources... Updating deployment state... Deployment complete! diff --git a/integration/bundle/testdata/default_python/bundle_init.txt b/integration/bundle/testdata/default_python/bundle_init.txt index c2917ea4e..6ea0801ad 100644 --- a/integration/bundle/testdata/default_python/bundle_init.txt +++ b/integration/bundle/testdata/default_python/bundle_init.txt @@ -1,6 +1,6 @@ Welcome to the default Python template for Databricks Asset Bundles! -Workspace to use (auto-detected, edit in 'project_name_$UNIQUE_PRJ/databricks.yml'): $DATABRICKS_URL +Workspace to use (auto-detected, edit in 'project_name_$UNIQUE_PRJ/databricks.yml'): [DATABRICKS_URL] ✨ Your new project has been created in the 'project_name_$UNIQUE_PRJ' directory! diff --git a/integration/bundle/testdata/default_python/bundle_summary.txt b/integration/bundle/testdata/default_python/bundle_summary.txt index 0b4c15764..450f01c46 100644 --- a/integration/bundle/testdata/default_python/bundle_summary.txt +++ b/integration/bundle/testdata/default_python/bundle_summary.txt @@ -22,54 +22,54 @@ "resources/project_name_$UNIQUE_PRJ.pipeline.yml" ], "workspace": { - "host": "$DATABRICKS_URL", + "host": "[DATABRICKS_URL]", "current_user": { "active": true, - "displayName": "$USERNAME", + "displayName": "[USERNAME]", "emails": [ { "primary": true, "type": "work", - "value": "$USERNAME" + "value": "[USERNAME]" } ], "groups": [ { - "$ref": "Groups/$USER.Groups[0]", + "$ref": "Groups/[USERGROUP]", "display": "team.engineering", "type": "direct", - "value": "$USER.Groups[0]" + "value": "[USERGROUP]" } ], - "id": "$USER.Id", + "id": "[USERID]", "name": { - "familyName": "$USERNAME", - "givenName": "$USERNAME" + "familyName": "[USERNAME]", + "givenName": "[USERNAME]" }, "schemas": [ "urn:ietf:params:scim:schemas:core:2.0:User", "urn:ietf:params:scim:schemas:extension:workspace:2.0:User" ], - "short_name": "$USERNAME", - "userName": "$USERNAME" + "short_name": "[USERNAME]", + "userName": "[USERNAME]" }, - "root_path": "/Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev", - "file_path": "/Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev/files", - "resource_path": "/Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev/resources", - "artifact_path": "/Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev/artifacts", - "state_path": "/Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev/state" + "root_path": "/Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev", + "file_path": "/Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev/files", + "resource_path": "/Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev/resources", + "artifact_path": "/Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev/artifacts", + "state_path": "/Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev/state" }, "resources": { "jobs": { "project_name_$UNIQUE_PRJ_job": { "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev/state/metadata.json" }, "edit_mode": "UI_LOCKED", "email_notifications": { "on_failure": [ - "$USERNAME" + "[USERNAME]" ] }, "format": "MULTI_TASK", @@ -88,18 +88,18 @@ } ], "max_concurrent_runs": 4, - "name": "[dev $USERNAME] project_name_$UNIQUE_PRJ_job", + "name": "[dev [USERNAME]] project_name_$UNIQUE_PRJ_job", "queue": { "enabled": true }, "tags": { - "dev": "$USERNAME" + "dev": "[USERNAME]" }, "tasks": [ { "job_cluster_key": "job_cluster", "notebook_task": { - "notebook_path": "/Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev/files/src/notebook" + "notebook_path": "/Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev/files/src/notebook" }, "task_key": "notebook_task" }, @@ -140,31 +140,31 @@ "unit": "DAYS" } }, - "url": "$DATABRICKS_URL/jobs/[NUMID]?o=[NUMID]" + "url": "[DATABRICKS_URL]/jobs/[NUMID]?o=[NUMID]" } }, "pipelines": { "project_name_$UNIQUE_PRJ_pipeline": { "catalog": "main", "configuration": { - "bundle.sourcePath": "/Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev/files/src" + "bundle.sourcePath": "/Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev/files/src" }, "deployment": { "kind": "BUNDLE", - "metadata_file_path": "/Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev/state/metadata.json" + "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev/state/metadata.json" }, "development": true, "id": "[UUID]", "libraries": [ { "notebook": { - "path": "/Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev/files/src/dlt_pipeline" + "path": "/Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev/files/src/dlt_pipeline" } } ], - "name": "[dev $USERNAME] project_name_$UNIQUE_PRJ_pipeline", + "name": "[dev [USERNAME]] project_name_$UNIQUE_PRJ_pipeline", "target": "project_name_$UNIQUE_PRJ_dev", - "url": "$DATABRICKS_URL/pipelines/[UUID]?o=[NUMID]" + "url": "[DATABRICKS_URL]/pipelines/[UUID]?o=[NUMID]" } } }, @@ -174,12 +174,12 @@ ] }, "presets": { - "name_prefix": "[dev $USERNAME] ", + "name_prefix": "[dev [USERNAME]] ", "pipelines_development": true, "trigger_pause_status": "PAUSED", "jobs_max_concurrent_runs": 4, "tags": { - "dev": "$USERNAME" + "dev": "[USERNAME]" } } -} +} \ No newline at end of file diff --git a/integration/bundle/testdata/default_python/bundle_validate.txt b/integration/bundle/testdata/default_python/bundle_validate.txt index 578fd6494..c5c62b521 100644 --- a/integration/bundle/testdata/default_python/bundle_validate.txt +++ b/integration/bundle/testdata/default_python/bundle_validate.txt @@ -1,8 +1,8 @@ Name: project_name_$UNIQUE_PRJ Target: dev Workspace: - Host: $DATABRICKS_URL - User: $USERNAME - Path: /Workspace/Users/$USERNAME/.bundle/project_name_$UNIQUE_PRJ/dev + Host: [DATABRICKS_URL] + User: [USERNAME] + Path: /Workspace/Users/[USERNAME]/.bundle/project_name_$UNIQUE_PRJ/dev Validation OK! diff --git a/libs/testdiff/replacement.go b/libs/testdiff/replacement.go index 7077e611b..5bbba1be1 100644 --- a/libs/testdiff/replacement.go +++ b/libs/testdiff/replacement.go @@ -2,7 +2,6 @@ package testdiff import ( "encoding/json" - "fmt" "path/filepath" "regexp" "runtime" @@ -16,7 +15,7 @@ import ( ) const ( - testerName = "$USERNAME" + testerName = "[USERNAME]" ) var ( @@ -140,25 +139,25 @@ func PrepareReplacementsWorkspaceClient(t testutil.TestingT, r *ReplacementsCont t.Helper() // in some clouds (gcp) w.Config.Host includes "https://" prefix in others it's really just a host (azure) host := strings.TrimPrefix(strings.TrimPrefix(w.Config.Host, "http://"), "https://") - r.Set("https://"+host, "$DATABRICKS_URL") - r.Set("http://"+host, "$DATABRICKS_URL") - r.Set(host, "$DATABRICKS_HOST") - r.Set(w.Config.ClusterID, "$DATABRICKS_CLUSTER_ID") - r.Set(w.Config.WarehouseID, "$DATABRICKS_WAREHOUSE_ID") - r.Set(w.Config.ServerlessComputeID, "$DATABRICKS_SERVERLESS_COMPUTE_ID") - r.Set(w.Config.AccountID, "$DATABRICKS_ACCOUNT_ID") - r.Set(w.Config.Username, "$DATABRICKS_USERNAME") - r.SetPath(w.Config.Profile, "$DATABRICKS_CONFIG_PROFILE") - r.Set(w.Config.ConfigFile, "$DATABRICKS_CONFIG_FILE") - r.Set(w.Config.GoogleServiceAccount, "$DATABRICKS_GOOGLE_SERVICE_ACCOUNT") - r.Set(w.Config.AzureResourceID, "$DATABRICKS_AZURE_RESOURCE_ID") + r.Set("https://"+host, "[DATABRICKS_URL]") + r.Set("http://"+host, "[DATABRICKS_URL]") + r.Set(host, "[DATABRICKS_HOST]") + r.Set(w.Config.ClusterID, "[DATABRICKS_CLUSTER_ID]") + r.Set(w.Config.WarehouseID, "[DATABRICKS_WAREHOUSE_ID]") + r.Set(w.Config.ServerlessComputeID, "[DATABRICKS_SERVERLESS_COMPUTE_ID]") + r.Set(w.Config.AccountID, "[DATABRICKS_ACCOUNT_ID]") + r.Set(w.Config.Username, "[DATABRICKS_USERNAME]") + r.SetPath(w.Config.Profile, "[DATABRICKS_CONFIG_PROFILE]") + r.Set(w.Config.ConfigFile, "[DATABRICKS_CONFIG_FILE]") + r.Set(w.Config.GoogleServiceAccount, "[DATABRICKS_GOOGLE_SERVICE_ACCOUNT]") + r.Set(w.Config.AzureResourceID, "[DATABRICKS_AZURE_RESOURCE_ID]") r.Set(w.Config.AzureClientID, testerName) - r.Set(w.Config.AzureTenantID, "$ARM_TENANT_ID") - r.Set(w.Config.AzureEnvironment, "$ARM_ENVIRONMENT") - r.Set(w.Config.ClientID, "$DATABRICKS_CLIENT_ID") - r.SetPath(w.Config.DatabricksCliPath, "$DATABRICKS_CLI_PATH") + r.Set(w.Config.AzureTenantID, "[ARM_TENANT_ID]") + r.Set(w.Config.AzureEnvironment, "[ARM_ENVIRONMENT]") + r.Set(w.Config.ClientID, "[DATABRICKS_CLIENT_ID]") + r.SetPath(w.Config.DatabricksCliPath, "[DATABRICKS_CLI_PATH]") // This is set to words like "path" that happen too frequently - // r.Set(w.Config.AuthType, "$DATABRICKS_AUTH_TYPE") + // r.Set(w.Config.AuthType, "[DATABRICKS_AUTH_TYPE]") } func PrepareReplacementsUser(t testutil.TestingT, r *ReplacementsContext, u iam.User) { @@ -179,14 +178,14 @@ func PrepareReplacementsUser(t testutil.TestingT, r *ReplacementsContext, u iam. r.Set(iamutil.GetShortUserName(&u), testerName) - for ind, val := range u.Groups { - r.Set(val.Value, fmt.Sprintf("$USER.Groups[%d]", ind)) + for _, val := range u.Groups { + r.Set(val.Value, "[USERGROUP]") } - r.Set(u.Id, "$USER.Id") + r.Set(u.Id, "[USERID]") - for ind, val := range u.Roles { - r.Set(val.Value, fmt.Sprintf("$USER.Roles[%d]", ind)) + for _, val := range u.Roles { + r.Set(val.Value, "[USERROLE]") } } @@ -207,5 +206,5 @@ func PrepareReplacementsTemporaryDirectory(t testutil.TestingT, r *ReplacementsC func PrepareReplacementsDevVersion(t testutil.TestingT, r *ReplacementsContext) { t.Helper() - r.append(devVersionRegex, "$$DEV_VERSION") + r.append(devVersionRegex, "[DEV_VERSION]") } From 2eb9abb5ee5db2984de5d791f1d347f7c91e5e21 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 3 Feb 2025 17:00:52 +0100 Subject: [PATCH 07/16] Bump github.com/spf13/pflag from 1.0.5 to 1.0.6 (#2281) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [github.com/spf13/pflag](https://github.com/spf13/pflag) from 1.0.5 to 1.0.6.
Release notes

Sourced from github.com/spf13/pflag's releases.

v1.0.6

What's Changed

New Contributors

Full Changelog: https://github.com/spf13/pflag/compare/v1.0.5...v1.0.6

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/spf13/pflag&package-manager=go_modules&previous-version=1.0.5&new-version=1.0.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go.mod | 2 +- go.sum | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/go.mod b/go.mod index 151133944..b3f11e918 100644 --- a/go.mod +++ b/go.mod @@ -22,7 +22,7 @@ require ( github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // BSD-2-Clause github.com/sabhiram/go-gitignore v0.0.0-20210923224102-525f6e181f06 // MIT github.com/spf13/cobra v1.8.1 // Apache 2.0 - github.com/spf13/pflag v1.0.5 // BSD-3-Clause + github.com/spf13/pflag v1.0.6 // BSD-3-Clause github.com/stretchr/testify v1.10.0 // MIT github.com/wI2L/jsondiff v0.6.1 // MIT golang.org/x/exp v0.0.0-20240222234643-814bf88cf225 diff --git a/go.sum b/go.sum index 3c7f20937..4e295a82d 100644 --- a/go.sum +++ b/go.sum @@ -147,8 +147,9 @@ github.com/skeema/knownhosts v1.3.0 h1:AM+y0rI04VksttfwjkSTNQorvGqmwATnvnAHpSgc0 github.com/skeema/knownhosts v1.3.0/go.mod h1:sPINvnADmT/qYH1kfv+ePMmOBTH6Tbl7b5LvTDjFK7M= github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM= github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y= -github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o= +github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= From 07efe830231d0b2c43079faeb1daee30781eb6cc Mon Sep 17 00:00:00 2001 From: Denis Bilenko Date: Tue, 4 Feb 2025 17:08:01 +0100 Subject: [PATCH 08/16] Use go-version-file instead of go-version in github actions (#2290) This minimizes number of places where we hard-code go version. Note, since we have go version specified without patch version ("1.23") in go.mod, it will use most recent in 1.23.x line. I think this is fine. https://github.com/actions/setup-go?tab=readme-ov-file#getting-go-version-from-the-gomod-file --- .github/workflows/push.yml | 6 +++--- .github/workflows/release-snapshot.yml | 2 +- .github/workflows/release.yml | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml index f27459baa..c41afc18c 100644 --- a/.github/workflows/push.yml +++ b/.github/workflows/push.yml @@ -50,7 +50,7 @@ jobs: - name: Setup Go uses: actions/setup-go@3041bf56c941b39c61721a86cd11f3bb1338122a # v5.2.0 with: - go-version: 1.23.4 + go-version-file: go.mod - name: Setup Python uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0 @@ -82,7 +82,7 @@ jobs: - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 - uses: actions/setup-go@3041bf56c941b39c61721a86cd11f3bb1338122a # v5.2.0 with: - go-version: 1.23.4 + go-version-file: go.mod # Use different schema from regular job, to avoid overwriting the same key cache-dependency-path: | go.sum @@ -116,7 +116,7 @@ jobs: - name: Setup Go uses: actions/setup-go@3041bf56c941b39c61721a86cd11f3bb1338122a # v5.2.0 with: - go-version: 1.23.4 + go-version-file: go.mod # Use different schema from regular job, to avoid overwriting the same key cache-dependency-path: | go.sum diff --git a/.github/workflows/release-snapshot.yml b/.github/workflows/release-snapshot.yml index 548d93e90..8b4684eab 100644 --- a/.github/workflows/release-snapshot.yml +++ b/.github/workflows/release-snapshot.yml @@ -34,7 +34,7 @@ jobs: - name: Setup Go uses: actions/setup-go@3041bf56c941b39c61721a86cd11f3bb1338122a # v5.2.0 with: - go-version: 1.23.4 + go-version-file: go.mod # The default cache key for this action considers only the `go.sum` file. # We include .goreleaser.yaml here to differentiate from the cache used by the push action diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 5d5811b19..fe5b4170b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -26,7 +26,7 @@ jobs: - name: Setup Go uses: actions/setup-go@3041bf56c941b39c61721a86cd11f3bb1338122a # v5.2.0 with: - go-version: 1.23.4 + go-version-file: go.mod # The default cache key for this action considers only the `go.sum` file. # We include .goreleaser.yaml here to differentiate from the cache used by the push action From d86ad9189902d629cc85e0d79250a7301bb29aeb Mon Sep 17 00:00:00 2001 From: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com> Date: Tue, 4 Feb 2025 22:08:11 +0530 Subject: [PATCH 09/16] Allow test servers to return errors responses (#2291) ## Changes The APIs at Databricks when returning a non `200` status code will return a response body of the format: ``` { "error_code": "Error code", "message": "Human-readable error message." } ``` This PR adds the ability to stub non-200 status codes in the test server, allowing us to mock API errors from Databricks. ## Tests New test --- acceptance/acceptance_test.go | 8 +++-- acceptance/cmd_server_test.go | 4 +-- acceptance/config_test.go | 3 +- acceptance/server_test.go | 32 +++++++++---------- .../jobs/create-error/out.requests.txt | 1 + .../workspace/jobs/create-error/output.txt | 5 +++ acceptance/workspace/jobs/create-error/script | 1 + .../workspace/jobs/create-error/test.toml | 12 +++++++ libs/testserver/server.go | 11 +++---- 9 files changed, 49 insertions(+), 28 deletions(-) create mode 100644 acceptance/workspace/jobs/create-error/out.requests.txt create mode 100644 acceptance/workspace/jobs/create-error/output.txt create mode 100644 acceptance/workspace/jobs/create-error/script create mode 100644 acceptance/workspace/jobs/create-error/test.toml diff --git a/acceptance/acceptance_test.go b/acceptance/acceptance_test.go index 871b8bd62..6e302fa96 100644 --- a/acceptance/acceptance_test.go +++ b/acceptance/acceptance_test.go @@ -261,8 +261,12 @@ func runTest(t *testing.T, dir, coverDir string, repls testdiff.ReplacementsCont for _, stub := range config.Server { require.NotEmpty(t, stub.Pattern) - server.Handle(stub.Pattern, func(req *http.Request) (resp any, err error) { - return stub.Response.Body, nil + server.Handle(stub.Pattern, func(req *http.Request) (any, int) { + statusCode := http.StatusOK + if stub.Response.StatusCode != 0 { + statusCode = stub.Response.StatusCode + } + return stub.Response.Body, statusCode }) } cmd.Env = append(cmd.Env, "DATABRICKS_HOST="+server.URL) diff --git a/acceptance/cmd_server_test.go b/acceptance/cmd_server_test.go index 9af63d0db..04d56c7d4 100644 --- a/acceptance/cmd_server_test.go +++ b/acceptance/cmd_server_test.go @@ -15,7 +15,7 @@ import ( func StartCmdServer(t *testing.T) *testserver.Server { server := testserver.New(t) - server.Handle("/", func(r *http.Request) (any, error) { + server.Handle("/", func(r *http.Request) (any, int) { q := r.URL.Query() args := strings.Split(q.Get("args"), " ") @@ -40,7 +40,7 @@ func StartCmdServer(t *testing.T) *testserver.Server { exitcode = 1 } result["exitcode"] = exitcode - return result, nil + return result, http.StatusOK }) return server } diff --git a/acceptance/config_test.go b/acceptance/config_test.go index c7be223de..97be457e2 100644 --- a/acceptance/config_test.go +++ b/acceptance/config_test.go @@ -57,7 +57,8 @@ type ServerStub struct { // The response body to return. Response struct { - Body string + Body string + StatusCode int } } diff --git a/acceptance/server_test.go b/acceptance/server_test.go index 4957a7668..a7695b21e 100644 --- a/acceptance/server_test.go +++ b/acceptance/server_test.go @@ -11,7 +11,7 @@ import ( ) func AddHandlers(server *testserver.Server) { - server.Handle("GET /api/2.0/policies/clusters/list", func(r *http.Request) (any, error) { + server.Handle("GET /api/2.0/policies/clusters/list", func(r *http.Request) (any, int) { return compute.ListPoliciesResponse{ Policies: []compute.Policy{ { @@ -23,10 +23,10 @@ func AddHandlers(server *testserver.Server) { Name: "some-test-cluster-policy", }, }, - }, nil + }, http.StatusOK }) - server.Handle("GET /api/2.0/instance-pools/list", func(r *http.Request) (any, error) { + server.Handle("GET /api/2.0/instance-pools/list", func(r *http.Request) (any, int) { return compute.ListInstancePools{ InstancePools: []compute.InstancePoolAndStats{ { @@ -34,10 +34,10 @@ func AddHandlers(server *testserver.Server) { InstancePoolId: "1234", }, }, - }, nil + }, http.StatusOK }) - server.Handle("GET /api/2.1/clusters/list", func(r *http.Request) (any, error) { + server.Handle("GET /api/2.1/clusters/list", func(r *http.Request) (any, int) { return compute.ListClustersResponse{ Clusters: []compute.ClusterDetails{ { @@ -49,32 +49,32 @@ func AddHandlers(server *testserver.Server) { ClusterId: "9876", }, }, - }, nil + }, http.StatusOK }) - server.Handle("GET /api/2.0/preview/scim/v2/Me", func(r *http.Request) (any, error) { + server.Handle("GET /api/2.0/preview/scim/v2/Me", func(r *http.Request) (any, int) { return iam.User{ Id: "1000012345", UserName: "tester@databricks.com", - }, nil + }, http.StatusOK }) - server.Handle("GET /api/2.0/workspace/get-status", func(r *http.Request) (any, error) { + server.Handle("GET /api/2.0/workspace/get-status", func(r *http.Request) (any, int) { return workspace.ObjectInfo{ ObjectId: 1001, ObjectType: "DIRECTORY", Path: "", ResourceId: "1001", - }, nil + }, http.StatusOK }) - server.Handle("GET /api/2.1/unity-catalog/current-metastore-assignment", func(r *http.Request) (any, error) { + server.Handle("GET /api/2.1/unity-catalog/current-metastore-assignment", func(r *http.Request) (any, int) { return catalog.MetastoreAssignment{ DefaultCatalogName: "main", - }, nil + }, http.StatusOK }) - server.Handle("GET /api/2.0/permissions/directories/1001", func(r *http.Request) (any, error) { + server.Handle("GET /api/2.0/permissions/directories/1001", func(r *http.Request) (any, int) { return workspace.WorkspaceObjectPermissions{ ObjectId: "1001", ObjectType: "DIRECTORY", @@ -88,10 +88,10 @@ func AddHandlers(server *testserver.Server) { }, }, }, - }, nil + }, http.StatusOK }) - server.Handle("POST /api/2.0/workspace/mkdirs", func(r *http.Request) (any, error) { - return "{}", nil + server.Handle("POST /api/2.0/workspace/mkdirs", func(r *http.Request) (any, int) { + return "{}", http.StatusOK }) } diff --git a/acceptance/workspace/jobs/create-error/out.requests.txt b/acceptance/workspace/jobs/create-error/out.requests.txt new file mode 100644 index 000000000..b22876b70 --- /dev/null +++ b/acceptance/workspace/jobs/create-error/out.requests.txt @@ -0,0 +1 @@ +{"method":"POST","path":"/api/2.1/jobs/create","body":{"name":"abc"}} diff --git a/acceptance/workspace/jobs/create-error/output.txt b/acceptance/workspace/jobs/create-error/output.txt new file mode 100644 index 000000000..0e69eeb4b --- /dev/null +++ b/acceptance/workspace/jobs/create-error/output.txt @@ -0,0 +1,5 @@ + +>>> [CLI] jobs create --json {"name":"abc"} +Error: Invalid access token. + +Exit code: 1 diff --git a/acceptance/workspace/jobs/create-error/script b/acceptance/workspace/jobs/create-error/script new file mode 100644 index 000000000..9ff7b5b87 --- /dev/null +++ b/acceptance/workspace/jobs/create-error/script @@ -0,0 +1 @@ +trace $CLI jobs create --json '{"name":"abc"}' diff --git a/acceptance/workspace/jobs/create-error/test.toml b/acceptance/workspace/jobs/create-error/test.toml new file mode 100644 index 000000000..b45bf77e5 --- /dev/null +++ b/acceptance/workspace/jobs/create-error/test.toml @@ -0,0 +1,12 @@ +LocalOnly = true # request recording currently does not work with cloud environment +RecordRequests = true + +[[Server]] +Pattern = "POST /api/2.1/jobs/create" +Response.Body = ''' +{ + "error_code": "PERMISSION_DENIED", + "message": "Invalid access token." +} +''' +Response.StatusCode = 403 diff --git a/libs/testserver/server.go b/libs/testserver/server.go index 2e8dbdfda..a751531ed 100644 --- a/libs/testserver/server.go +++ b/libs/testserver/server.go @@ -40,15 +40,11 @@ func New(t testutil.TestingT) *Server { } } -type HandlerFunc func(req *http.Request) (resp any, err error) +type HandlerFunc func(req *http.Request) (resp any, statusCode int) func (s *Server) Handle(pattern string, handler HandlerFunc) { s.Mux.HandleFunc(pattern, func(w http.ResponseWriter, r *http.Request) { - resp, err := handler(r) - if err != nil { - http.Error(w, err.Error(), http.StatusInternalServerError) - return - } + resp, statusCode := handler(r) if s.RecordRequests { body, err := io.ReadAll(r.Body) @@ -63,9 +59,10 @@ func (s *Server) Handle(pattern string, handler HandlerFunc) { } w.Header().Set("Content-Type", "application/json") + w.WriteHeader(statusCode) var respBytes []byte - + var err error respString, ok := resp.(string) if ok { respBytes = []byte(respString) From 84b694f2a158186fa6d641e55e51af29761fa419 Mon Sep 17 00:00:00 2001 From: Simon Poltier Date: Tue, 4 Feb 2025 19:28:19 +0100 Subject: [PATCH 10/16] accept JSON includes (#2265) #2201 disabled using JSON as part of a bundle definition. I believe this was not intended. ## Changes Accept json files as includes, just as YAML files. ## Tests Covered by the tests in #2201 --- acceptance/bundle/includes/non_yaml_in_include/output.txt | 4 ++-- bundle/config/loader/process_root_includes.go | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/acceptance/bundle/includes/non_yaml_in_include/output.txt b/acceptance/bundle/includes/non_yaml_in_include/output.txt index 6006ca14e..f5211cc4b 100644 --- a/acceptance/bundle/includes/non_yaml_in_include/output.txt +++ b/acceptance/bundle/includes/non_yaml_in_include/output.txt @@ -1,7 +1,7 @@ -Error: Files in the 'include' configuration section must be YAML files. +Error: Files in the 'include' configuration section must be YAML or JSON files. in databricks.yml:5:4 -The file test.py in the 'include' configuration section is not a YAML file, and only YAML files are supported. To include files to sync, specify them in the 'sync.include' configuration section instead. +The file test.py in the 'include' configuration section is not a YAML or JSON file, and only such files are supported. To include files to sync, specify them in the 'sync.include' configuration section instead. Name: non_yaml_in_includes diff --git a/bundle/config/loader/process_root_includes.go b/bundle/config/loader/process_root_includes.go index 198095742..69e6dd4e4 100644 --- a/bundle/config/loader/process_root_includes.go +++ b/bundle/config/loader/process_root_includes.go @@ -71,11 +71,11 @@ func (m *processRootIncludes) Apply(ctx context.Context, b *bundle.Bundle) diag. continue } seen[rel] = true - if filepath.Ext(rel) != ".yaml" && filepath.Ext(rel) != ".yml" { + if filepath.Ext(rel) != ".yaml" && filepath.Ext(rel) != ".yml" && filepath.Ext(rel) != ".json" { diags = diags.Append(diag.Diagnostic{ Severity: diag.Error, - Summary: "Files in the 'include' configuration section must be YAML files.", - Detail: fmt.Sprintf("The file %s in the 'include' configuration section is not a YAML file, and only YAML files are supported. To include files to sync, specify them in the 'sync.include' configuration section instead.", rel), + Summary: "Files in the 'include' configuration section must be YAML or JSON files.", + Detail: fmt.Sprintf("The file %s in the 'include' configuration section is not a YAML or JSON file, and only such files are supported. To include files to sync, specify them in the 'sync.include' configuration section instead.", rel), Locations: b.Config.GetLocations(fmt.Sprintf("include[%d]", i)), }) continue From dcc61cd7636b4cdc647256bbd07aac8d4651da9e Mon Sep 17 00:00:00 2001 From: rikjansen-hu Date: Tue, 4 Feb 2025 19:30:02 +0100 Subject: [PATCH 11/16] Fix env variable for AzureCli local config (#2248) ## Changes Solves #1722 (current solution passes wrong variable) ## Tests None, this is a simple find-and-replace on a previous PR. Proof that this is the correct [variable](https://learn.microsoft.com/en-us/cli/azure/azure-cli-configuration#cli-configuration-file). This just passes the variable along to the Terraform environment, which [should](https://github.com/hashicorp/terraform/issues/25416) be picked up by Terraform. Co-authored-by: Rik Jansen --- bundle/deploy/terraform/init.go | 4 ++-- bundle/deploy/terraform/init_test.go | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/bundle/deploy/terraform/init.go b/bundle/deploy/terraform/init.go index 5957611a4..a204222d0 100644 --- a/bundle/deploy/terraform/init.go +++ b/bundle/deploy/terraform/init.go @@ -101,9 +101,9 @@ var envCopy = []string{ // same auxiliary programs (e.g. `az`, or `gcloud`) as the CLI. "PATH", - // Include $AZURE_CONFIG_FILE in set of environment variables to pass along. + // Include $AZURE_CONFIG_DIR in set of environment variables to pass along. // This is set in Azure DevOps by the AzureCLI@2 task. - "AZURE_CONFIG_FILE", + "AZURE_CONFIG_DIR", // Include $TF_CLI_CONFIG_FILE to override terraform provider in development. // See: https://developer.hashicorp.com/terraform/cli/config/config-file#explicit-installation-method-configuration diff --git a/bundle/deploy/terraform/init_test.go b/bundle/deploy/terraform/init_test.go index c7a4ffe4a..4645ed007 100644 --- a/bundle/deploy/terraform/init_test.go +++ b/bundle/deploy/terraform/init_test.go @@ -292,7 +292,7 @@ func TestInheritEnvVars(t *testing.T) { t.Setenv("HOME", "/home/testuser") t.Setenv("PATH", "/foo:/bar") t.Setenv("TF_CLI_CONFIG_FILE", "/tmp/config.tfrc") - t.Setenv("AZURE_CONFIG_FILE", "/tmp/foo/bar") + t.Setenv("AZURE_CONFIG_DIR", "/tmp/foo/bar") ctx := context.Background() env := map[string]string{} @@ -301,7 +301,7 @@ func TestInheritEnvVars(t *testing.T) { assert.Equal(t, "/home/testuser", env["HOME"]) assert.Equal(t, "/foo:/bar", env["PATH"]) assert.Equal(t, "/tmp/config.tfrc", env["TF_CLI_CONFIG_FILE"]) - assert.Equal(t, "/tmp/foo/bar", env["AZURE_CONFIG_FILE"]) + assert.Equal(t, "/tmp/foo/bar", env["AZURE_CONFIG_DIR"]) } } From 2e1455841cced1337ad8ff8b456f57f217a9a74d Mon Sep 17 00:00:00 2001 From: Pieter Noordhuis Date: Tue, 4 Feb 2025 22:20:02 +0100 Subject: [PATCH 12/16] Update CODEOWNERS for cmd/labs (#2295) ## Changes The `CODEOWNERS` file must live in one of the directories specified in the [docs][docs], so the existing file under `cmd/labs` didn't work. This change moves the contents to the top-level file and includes @alexott as owner. [docs]: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners#codeowners-file-location --- .github/CODEOWNERS | 1 + cmd/labs/CODEOWNERS | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) delete mode 100644 cmd/labs/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 76835de7d..3c3895bc1 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1 +1,2 @@ * @pietern @andrewnester @shreyas-goenka @denik +cmd/labs @alexott @nfx diff --git a/cmd/labs/CODEOWNERS b/cmd/labs/CODEOWNERS deleted file mode 100644 index cc93a75e6..000000000 --- a/cmd/labs/CODEOWNERS +++ /dev/null @@ -1 +0,0 @@ -* @nfx From 1678503cb04abfed7fdd1d595d93ed722c75330e Mon Sep 17 00:00:00 2001 From: Ilya Kuznetsov Date: Wed, 5 Feb 2025 10:01:51 +0100 Subject: [PATCH 13/16] Fix docs template (#2283) ## Changes Comment breaks markdown front-matter and description cannot be read ## Tests --- bundle/docsgen/output/reference.md | 3 ++- bundle/docsgen/output/resources.md | 3 ++- bundle/docsgen/templates/reference.md | 3 ++- bundle/docsgen/templates/resources.md | 3 ++- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/bundle/docsgen/output/reference.md b/bundle/docsgen/output/reference.md index a2241d017..8a89d354b 100644 --- a/bundle/docsgen/output/reference.md +++ b/bundle/docsgen/output/reference.md @@ -1,8 +1,9 @@ - --- description: Configuration reference for databricks.yml --- + + # Configuration reference This article provides reference for keys supported by configuration (YAML). See [_](/dev-tools/bundles/index.md). diff --git a/bundle/docsgen/output/resources.md b/bundle/docsgen/output/resources.md index ff80ee635..df7578c73 100644 --- a/bundle/docsgen/output/resources.md +++ b/bundle/docsgen/output/resources.md @@ -1,8 +1,9 @@ - --- description: Learn about resources supported by Databricks Asset Bundles and how to configure them. --- + + # resources allows you to specify information about the resources used by the bundle in the `resources` mapping in the bundle configuration. See [resources mapping](/dev-tools/bundles/settings.md#resources) and [resources key reference](/dev-tools/bundles/reference.md#resources). diff --git a/bundle/docsgen/templates/reference.md b/bundle/docsgen/templates/reference.md index a17d53315..345afc509 100644 --- a/bundle/docsgen/templates/reference.md +++ b/bundle/docsgen/templates/reference.md @@ -1,8 +1,9 @@ - --- description: Configuration reference for databricks.yml --- + + # Configuration reference This article provides reference for keys supported by configuration (YAML). See [_](/dev-tools/bundles/index.md). diff --git a/bundle/docsgen/templates/resources.md b/bundle/docsgen/templates/resources.md index fccfac47d..e9a6c8c5b 100644 --- a/bundle/docsgen/templates/resources.md +++ b/bundle/docsgen/templates/resources.md @@ -1,8 +1,9 @@ - --- description: Learn about resources supported by Databricks Asset Bundles and how to configure them. --- + + # resources allows you to specify information about the resources used by the bundle in the `resources` mapping in the bundle configuration. See [resources mapping](/dev-tools/bundles/settings.md#resources) and [resources key reference](/dev-tools/bundles/reference.md#resources). From 57b8d336e03cb2b9aa0c353d9cce7d85d5c72c7f Mon Sep 17 00:00:00 2001 From: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com> Date: Wed, 5 Feb 2025 15:02:15 +0530 Subject: [PATCH 14/16] Add ability to record headers in acceptance tests (#2296) ## Changes HTTP headers like the User-Agent are an important part of our internal ETL pipelines. This PR adds the ability to validate the headers used in an HTTP request as part of our acceptance tests. ## Tests Modifying existing test. --- acceptance/acceptance_test.go | 3 +++ acceptance/config_test.go | 3 +++ .../workspace/jobs/create/out.requests.txt | 2 +- acceptance/workspace/jobs/create/test.toml | 17 ++++++++++++ libs/testdiff/replacement.go | 18 +++++++++++++ libs/testserver/server.go | 26 ++++++++++++++----- 6 files changed, 61 insertions(+), 8 deletions(-) diff --git a/acceptance/acceptance_test.go b/acceptance/acceptance_test.go index 6e302fa96..f205217ff 100644 --- a/acceptance/acceptance_test.go +++ b/acceptance/acceptance_test.go @@ -156,6 +156,8 @@ func testAccept(t *testing.T, InprocessMode bool, singleTest string) int { testdiff.PrepareReplacementsWorkspaceClient(t, &repls, workspaceClient) testdiff.PrepareReplacementsUUID(t, &repls) testdiff.PrepareReplacementsDevVersion(t, &repls) + testdiff.PrepareReplacementSdkVersion(t, &repls) + testdiff.PrepareReplacementsGoVersion(t, &repls) testDirs := getTests(t) require.NotEmpty(t, testDirs) @@ -253,6 +255,7 @@ func runTest(t *testing.T, dir, coverDir string, repls testdiff.ReplacementsCont if len(config.Server) > 0 || config.RecordRequests { server = testserver.New(t) server.RecordRequests = config.RecordRequests + server.IncludeRequestHeaders = config.IncludeRequestHeaders // If no custom server stubs are defined, add the default handlers. if len(config.Server) == 0 { diff --git a/acceptance/config_test.go b/acceptance/config_test.go index 97be457e2..e24a683e7 100644 --- a/acceptance/config_test.go +++ b/acceptance/config_test.go @@ -47,6 +47,9 @@ type TestConfig struct { // Record the requests made to the server and write them as output to // out.requests.txt RecordRequests bool + + // List of request headers to include when recording requests. + IncludeRequestHeaders []string } type ServerStub struct { diff --git a/acceptance/workspace/jobs/create/out.requests.txt b/acceptance/workspace/jobs/create/out.requests.txt index b22876b70..4a85c4c43 100644 --- a/acceptance/workspace/jobs/create/out.requests.txt +++ b/acceptance/workspace/jobs/create/out.requests.txt @@ -1 +1 @@ -{"method":"POST","path":"/api/2.1/jobs/create","body":{"name":"abc"}} +{"headers":{"Authorization":"Bearer dapi1234","User-Agent":"cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/jobs_create cmd-exec-id/[UUID] auth/pat"},"method":"POST","path":"/api/2.1/jobs/create","body":{"name":"abc"}} diff --git a/acceptance/workspace/jobs/create/test.toml b/acceptance/workspace/jobs/create/test.toml index e69569c18..1fd9b3cec 100644 --- a/acceptance/workspace/jobs/create/test.toml +++ b/acceptance/workspace/jobs/create/test.toml @@ -1,5 +1,6 @@ LocalOnly = true # request recording currently does not work with cloud environment RecordRequests = true +IncludeRequestHeaders = ["Authorization", "User-Agent"] [[Server]] Pattern = "POST /api/2.1/jobs/create" @@ -8,3 +9,19 @@ Response.Body = ''' "job_id": 1111 } ''' + +[[Repls]] +Old = "(linux|darwin|windows)" +New = "[OS]" + +[[Repls]] +Old = " upstream/[A-Za-z0-9.-]+" +New = "" + +[[Repls]] +Old = " upstream-version/[A-Za-z0-9.-]+" +New = "" + +[[Repls]] +Old = " cicd/[A-Za-z0-9.-]+" +New = "" diff --git a/libs/testdiff/replacement.go b/libs/testdiff/replacement.go index 5bbba1be1..d4d5eb27b 100644 --- a/libs/testdiff/replacement.go +++ b/libs/testdiff/replacement.go @@ -12,6 +12,7 @@ import ( "github.com/databricks/cli/libs/iamutil" "github.com/databricks/databricks-sdk-go" "github.com/databricks/databricks-sdk-go/service/iam" + "golang.org/x/mod/semver" ) const ( @@ -208,3 +209,20 @@ func PrepareReplacementsDevVersion(t testutil.TestingT, r *ReplacementsContext) t.Helper() r.append(devVersionRegex, "[DEV_VERSION]") } + +func PrepareReplacementSdkVersion(t testutil.TestingT, r *ReplacementsContext) { + t.Helper() + r.Set(databricks.Version(), "[SDK_VERSION]") +} + +func goVersion() string { + gv := runtime.Version() + ssv := strings.ReplaceAll(gv, "go", "v") + sv := semver.Canonical(ssv) + return strings.TrimPrefix(sv, "v") +} + +func PrepareReplacementsGoVersion(t testutil.TestingT, r *ReplacementsContext) { + t.Helper() + r.Set(goVersion(), "[GO_VERSION]") +} diff --git a/libs/testserver/server.go b/libs/testserver/server.go index a751531ed..5e3efe1c5 100644 --- a/libs/testserver/server.go +++ b/libs/testserver/server.go @@ -5,6 +5,7 @@ import ( "io" "net/http" "net/http/httptest" + "slices" "github.com/stretchr/testify/assert" @@ -17,15 +18,17 @@ type Server struct { t testutil.TestingT - RecordRequests bool + RecordRequests bool + IncludeRequestHeaders []string Requests []Request } type Request struct { - Method string `json:"method"` - Path string `json:"path"` - Body any `json:"body"` + Headers map[string]string `json:"headers,omitempty"` + Method string `json:"method"` + Path string `json:"path"` + Body any `json:"body"` } func New(t testutil.TestingT) *Server { @@ -50,10 +53,19 @@ func (s *Server) Handle(pattern string, handler HandlerFunc) { body, err := io.ReadAll(r.Body) assert.NoError(s.t, err) + headers := make(map[string]string) + for k, v := range r.Header { + if len(v) == 0 || !slices.Contains(s.IncludeRequestHeaders, k) { + continue + } + headers[k] = v[0] + } + s.Requests = append(s.Requests, Request{ - Method: r.Method, - Path: r.URL.Path, - Body: json.RawMessage(body), + Headers: headers, + Method: r.Method, + Path: r.URL.Path, + Body: json.RawMessage(body), }) } From 5c90752797d4cd2e49bdda5a05fb35e9a59d5edc Mon Sep 17 00:00:00 2001 From: Andrew Nester Date: Wed, 5 Feb 2025 11:53:36 +0000 Subject: [PATCH 15/16] acc: Added acceptance test for CLI commands inside bundle with and without profile flag (#2270) ## Changes This encodes existing behaviour in CLI as reported here: #1358 --- .../auth/bundle_and_profile/.databrickscfg | 5 +++ .../auth/bundle_and_profile/databricks.yml | 14 ++++++++ acceptance/auth/bundle_and_profile/output.txt | 32 +++++++++++++++++++ acceptance/auth/bundle_and_profile/script | 30 +++++++++++++++++ acceptance/auth/bundle_and_profile/test.toml | 8 +++++ 5 files changed, 89 insertions(+) create mode 100644 acceptance/auth/bundle_and_profile/.databrickscfg create mode 100644 acceptance/auth/bundle_and_profile/databricks.yml create mode 100644 acceptance/auth/bundle_and_profile/output.txt create mode 100644 acceptance/auth/bundle_and_profile/script create mode 100644 acceptance/auth/bundle_and_profile/test.toml diff --git a/acceptance/auth/bundle_and_profile/.databrickscfg b/acceptance/auth/bundle_and_profile/.databrickscfg new file mode 100644 index 000000000..628505286 --- /dev/null +++ b/acceptance/auth/bundle_and_profile/.databrickscfg @@ -0,0 +1,5 @@ +[DEFAULT] +host = $DATABRICKS_HOST + +[profile_name] +host = https://test@non-existing-subdomain.databricks.com diff --git a/acceptance/auth/bundle_and_profile/databricks.yml b/acceptance/auth/bundle_and_profile/databricks.yml new file mode 100644 index 000000000..975661395 --- /dev/null +++ b/acceptance/auth/bundle_and_profile/databricks.yml @@ -0,0 +1,14 @@ +bundle: + name: test-auth + +workspace: + host: $DATABRICKS_HOST + +targets: + dev: + default: true + workspace: + host: $DATABRICKS_HOST + prod: + workspace: + host: https://bar.com diff --git a/acceptance/auth/bundle_and_profile/output.txt b/acceptance/auth/bundle_and_profile/output.txt new file mode 100644 index 000000000..022b3148d --- /dev/null +++ b/acceptance/auth/bundle_and_profile/output.txt @@ -0,0 +1,32 @@ + +=== Inside the bundle, no flags +>>> errcode [CLI] current-user me +"[USERNAME]" + +=== Inside the bundle, target flags +>>> errcode [CLI] current-user me -t dev +"[USERNAME]" + +=== Inside the bundle, target and matching profile +>>> errcode [CLI] current-user me -t dev -p DEFAULT +"[USERNAME]" + +=== Inside the bundle, profile flag not matching bundle host. Badness: should use profile from flag instead and not fail +>>> errcode [CLI] current-user me -p profile_name +Error: cannot resolve bundle auth configuration: config host mismatch: profile uses host https://non-existing-subdomain.databricks.com, but CLI configured to use [DATABRICKS_URL] + +Exit code: 1 + +=== Inside the bundle, target and not matching profile +>>> errcode [CLI] current-user me -t dev -p profile_name +Error: cannot resolve bundle auth configuration: config host mismatch: profile uses host https://non-existing-subdomain.databricks.com, but CLI configured to use [DATABRICKS_URL] + +Exit code: 1 + +=== Outside the bundle, no flags +>>> errcode [CLI] current-user me +"[USERNAME]" + +=== Outside the bundle, profile flag +>>> errcode [CLI] current-user me -p profile_name +"[USERNAME]" diff --git a/acceptance/auth/bundle_and_profile/script b/acceptance/auth/bundle_and_profile/script new file mode 100644 index 000000000..b37f5e01d --- /dev/null +++ b/acceptance/auth/bundle_and_profile/script @@ -0,0 +1,30 @@ +# Replace placeholder with an actual host URL +envsubst < databricks.yml > out.yml && mv out.yml databricks.yml +envsubst < .databrickscfg > out && mv out .databrickscfg +export DATABRICKS_CONFIG_FILE=.databrickscfg + +host=$DATABRICKS_HOST +unset DATABRICKS_HOST + +title "Inside the bundle, no flags" +trace errcode $CLI current-user me | jq .userName + +title "Inside the bundle, target flags" +trace errcode $CLI current-user me -t dev | jq .userName + +title "Inside the bundle, target and matching profile" +trace errcode $CLI current-user me -t dev -p DEFAULT | jq .userName + +title "Inside the bundle, profile flag not matching bundle host. Badness: should use profile from flag instead and not fail" +trace errcode $CLI current-user me -p profile_name | jq .userName + +title "Inside the bundle, target and not matching profile" +trace errcode $CLI current-user me -t dev -p profile_name + +cd .. +export DATABRICKS_HOST=$host +title "Outside the bundle, no flags" +trace errcode $CLI current-user me | jq .userName + +title "Outside the bundle, profile flag" +trace errcode $CLI current-user me -p profile_name | jq .userName diff --git a/acceptance/auth/bundle_and_profile/test.toml b/acceptance/auth/bundle_and_profile/test.toml new file mode 100644 index 000000000..b20190ca5 --- /dev/null +++ b/acceptance/auth/bundle_and_profile/test.toml @@ -0,0 +1,8 @@ +Badness = "When -p flag is used inside the bundle folder for any CLI commands, CLI use bundle host anyway instead of profile one" + +# Some of the clouds have DATABRICKS_HOST variable setup without https:// prefix +# In the result, output is replaced with DATABRICKS_URL variable instead of DATABRICKS_HOST +# This is a workaround to replace DATABRICKS_URL with DATABRICKS_HOST +[[Repls]] +Old='DATABRICKS_HOST' +New='DATABRICKS_URL' From 27caf413f2d601de8d81d6cc3fb2614ff1bf354c Mon Sep 17 00:00:00 2001 From: Marcin Wojtyczka Date: Wed, 5 Feb 2025 14:24:15 +0100 Subject: [PATCH 16/16] Add support for extras to the labs CLI (#2288) ## Changes Added support for extras / optional Python dependencies in the labs CLI. Added new `extras` field under install. Example: ```yaml install: script: install.py extras: cli ``` Resolves: #2257 ## Tests Manual test --- cmd/labs/project/installer.go | 5 +++++ cmd/labs/project/schema.json | 5 +++++ .../.databricks/labs/blueprint/lib/labs.yml | 1 + 3 files changed, 11 insertions(+) diff --git a/cmd/labs/project/installer.go b/cmd/labs/project/installer.go index 05f7d68aa..2e42ce43d 100644 --- a/cmd/labs/project/installer.go +++ b/cmd/labs/project/installer.go @@ -32,6 +32,7 @@ type hook struct { RequireDatabricksConnect bool `yaml:"require_databricks_connect,omitempty"` MinRuntimeVersion string `yaml:"min_runtime_version,omitempty"` WarehouseTypes whTypes `yaml:"warehouse_types,omitempty"` + Extras string `yaml:"extras,omitempty"` } func (h *hook) RequireRunningCluster() bool { @@ -258,6 +259,10 @@ func (i *installer) setupPythonVirtualEnvironment(ctx context.Context, w *databr } } feedback <- "Installing Python library dependencies" + if i.Installer.Extras != "" { + // install main and optional dependencies + return i.installPythonDependencies(ctx, fmt.Sprintf(".[%s]", i.Installer.Extras)) + } return i.installPythonDependencies(ctx, ".") } diff --git a/cmd/labs/project/schema.json b/cmd/labs/project/schema.json index a779b15e4..7aa65813c 100644 --- a/cmd/labs/project/schema.json +++ b/cmd/labs/project/schema.json @@ -42,6 +42,11 @@ }, "warehouse_types": { "enum": [ "PRO", "CLASSIC", "TYPE_UNSPECIFIED" ] + }, + "extras": { + "type": "string", + "pattern": "^([^,]+)(,([^,]+))*$", + "default": "" } } }, diff --git a/cmd/labs/project/testdata/installed-in-home/.databricks/labs/blueprint/lib/labs.yml b/cmd/labs/project/testdata/installed-in-home/.databricks/labs/blueprint/lib/labs.yml index 0ac4bf826..b8a0e695e 100644 --- a/cmd/labs/project/testdata/installed-in-home/.databricks/labs/blueprint/lib/labs.yml +++ b/cmd/labs/project/testdata/installed-in-home/.databricks/labs/blueprint/lib/labs.yml @@ -8,6 +8,7 @@ install: warehouse_types: - PRO script: install.py + extras: "" entrypoint: main.py min_python: 3.9 commands: