## Changes
This PR:
1. Incrementally improves the error messages shown to the user when the
volume they are referring to in `workspace.artifact_path` does not
exist.
2. Performs this validation in both `bundle validate` and `bundle
deploy` compared to before on just deployments.
3. It runs "fast" validations on `bundle deploy`, which earlier were
only run on `bundle validate`.
## Tests
Unit tests and manually. Also, existing integration tests provide
coverage (`TestUploadArtifactToVolumeNotYetDeployed`,
`TestUploadArtifactFileToVolumeThatDoesNotExist`)
Examples:
```
.venv➜ bundle-playground git:(master) ✗ cli bundle validate
Error: cannot access volume capital.whatever.my_volume: User does not have READ VOLUME on Volume 'capital.whatever.my_volume'.
at workspace.artifact_path
in databricks.yml:7:18
```
and
```
.venv➜ bundle-playground git:(master) ✗ cli bundle validate
Error: volume capital.whatever.foobar does not exist
at workspace.artifact_path
resources.volumes.foo
in databricks.yml:7:18
databricks.yml:12:7
You are using a volume in your artifact_path that is managed by
this bundle but which has not been deployed yet. Please first deploy
the volume using 'bundle deploy' and then switch over to using it in
the artifact_path.
```
## Changes
https://github.com/databricks/mlops-stacks/pull/187 broke mlops-stacks
deployments for non-UC projects. Snoozing the test until upstream is
fixed.
## Tests
The test is skipped on my local machine. CI run will verify it's also
skipped on Github Actions runners.
By default it stops after 3 issues of a given type, which gives false
impression and also unhelpful if you fixing it with aider.
1000 is almost like unlimited but not unlimited in case there is a bug
in a linter.
## Changes
This PR adds back debugging functionality that was lost during migration
to `internal.Main` as an entry point for integration tests.
The PR that caused the regression:
https://github.com/databricks/cli/pull/2009. Specifically the addition
of internal.Main as the entrypoint for all integration tests.
## Tests
Manually, by trying to debug a test.
This is useful on CI and locally for debugging and being able to
copy-paste command to tweak the options.
Removed redundant and imprecise messages like "✓ Running tests ...".
On main branch: ‘make test’ takes about 33s
On this branch: ‘make test’ takes about 2.7s
(all measurements are for hot cache)
What’s done (from highest impact to lowest):
- Remove -coverprofile= option - this option was disabling "go test"'s
built-in cache and also it took extra time to calculate the coverage
(extra 21s).
- Exclude ./integration/ folder, there are no unit tests there, but
having it included adds significant time. "go test"'s caching also does
not work there for me, due to TestMain() presence (extra 7.2s).
- Remove dependency on "make lint" - nice to have, but slow to re-check
the whole repo and should already be done by IDE (extra 2.5s).
- Remove dependency on "make vendor" — rarely needed; on CI it is
already executed separately (extra 1.1s).
The coverage option is still available under "make cover". Use "make
showcover" to show it.
I’ve also removed separate "make testonly". If you only want tests, run
"make test". If you want lint+test run "make lint test" etc.
I've also modified the test command, removed unnecessary -short, -v,
--raw-command.
## Changes
- Enable new linter: testifylint.
- Apply fixes with --fix.
- Fix remaining issues (mostly with aider).
There were 2 cases we --fix did the wrong thing - this seems to a be a
bug in linter: https://github.com/Antonboom/testifylint/issues/210
Nonetheless, I kept that check enabled, it seems useful, just need to be
fixed manually after autofix.
## Tests
Existing tests
## Changes
I noticed a diff in the schema in #2052.
This check should be performed automatically.
## Tests
This PR includes a commit that changes the schema to check that the
workflow actually fails.
## Changes
- Add t.Helper() in testcli-related helpers, this ensures that output is
attributed correctly to test case and not to the helper.
- Modify testlcli.Run() to run process in foreground. This is needed for
t.Helper to work.
- Extend a few assertions with message to help attribute it to proper
helper where needed.
## Tests
Manually reviewed test output.
Before:
```
+ go test --timeout 3h -v -run TestDefaultPython/3.9 ./integration/bundle/
=== RUN TestDefaultPython
=== RUN TestDefaultPython/3.9
workspace.go:26: aws
golden.go:14: run args: [bundle, init, default-python, --config-file, config.json]
runner.go:206: [databricks stderr]:
runner.go:206: [databricks stderr]: Welcome to the default Python template for Databricks Asset Bundles!
...
testdiff.go:23:
Error Trace: /Users/denis.bilenko/work/cli/libs/testdiff/testdiff.go:23
/Users/denis.bilenko/work/cli/libs/testdiff/golden.go:43
/Users/denis.bilenko/work/cli/internal/testcli/golden.go:23
/Users/denis.bilenko/work/cli/integration/bundle/init_default_python_test.go:92
/Users/denis.bilenko/work/cli/integration/bundle/init_default_python_test.go:45
...
```
After:
```
+ go test --timeout 3h -v -run TestDefaultPython/3.9 ./integration/bundle/
=== RUN TestDefaultPython
=== RUN TestDefaultPython/3.9
init_default_python_test.go:51: CLOUD_ENV=aws
init_default_python_test.go:92: args: bundle, init, default-python, --config-file, config.json
init_default_python_test.go:92: stderr:
init_default_python_test.go:92: stderr: Welcome to the default Python template for Databricks Asset Bundles!
...
init_default_python_test.go:92:
Error Trace: /Users/denis.bilenko/work/cli/libs/testdiff/testdiff.go:24
/Users/denis.bilenko/work/cli/libs/testdiff/golden.go:46
/Users/denis.bilenko/work/cli/internal/testcli/golden.go:23
/Users/denis.bilenko/work/cli/integration/bundle/init_default_python_test.go:92
/Users/denis.bilenko/work/cli/integration/bundle/init_default_python_test.go:45
...
```
## Changes
Fix cases where accumulated diagnostics are lost instead of being
propagated further. In some cases it's not possible, add a comment
there.
## Tests
Existing tests
## Changes
- Detach "golden files" assertions from testcli runner. Now any output
can be compared, no matter how it is obtained.
- Move those assertion to libs/testdiff package.
This allows using "golden files" in non-integration tests.
## Tests
Existing tests
First stage is to run goimports and formatter, second is full suite.
This ensures that imports and formatting are fixed even in presence of
other issues. Otherwise golanci-lint refuses to fix anything
https://github.com/golangci/golangci-lint/issues/5257
This helpful when running aider with config like this - aider will use
that to autofix what it can after every update:
```
% cat .aider.conf.yml
lint-cmd:
- "go: ./lint.sh"
```
## Changes
Relax the checks of `lib/template/builtin_test` so they don't fail for a
local development copy that has uncommitted draft templates. Right now
these tests fail because I have some git-ignored uncommitted templates
in my local dev copy.
## Changes
1. Removes default yaml-fields during schema generation, caused by [this
PR](https://github.com/databricks/cli/pull/2032) (current yaml package
can't read `json` annotations in struct fields)
2. Addresses missing annotations for fields from OpenAPI spec, which are
named differently in go SDK
3. Adds filtering for annotations.yaml to include only CLI package
fields
4. Implements alphabetical sort for yaml keys to avoid unnecessary diff
in PRs
## Tests
Manually tested
## Changes
Add new type of test helpers that run the command and compare full
output (golden files approach).
In case of JSON, there is also an option to ignore certain paths.
Add test for different versions of Python to go through bundle init
default-python / validate / deploy / summary.
## Tests
New integration tests.
Shreyas Goenka
Andrew Nester
shreyas-goenka
Denis Bilenko
Pieter Noordhuis
Lennart Kats (databricks)
Ilya Kuznetsov