databricks-cli/acceptance
Denis Bilenko 5d392acbef
acc: Allow mixing custom stubs with default server impl (#2334)
## Changes
- Currently if you define [[Server]] block, you disable the default
server implementation. With this change [[Server]] block takes
precedence over default server but default server remains.
- Switched mux implementation to
[gorilla/mux](https://github.com/gorilla/mux) -- unlike built-in it does
not panic if you set two handlers on the same part (instead the earliest
one wins). It also does not have any dependencies.
- Move acceptance/selftest into acceptance/selftest/basic and added
acceptance/selftest/server that demoes server override.
- Rewrite server set up to ensure that env vars and replacements are set
up correctly. Previously replacements for DATABRICKS_HOST referred to
default server, not to the custom server.
- Avoid calling CurrentUser.Me() in the local case. This allows
overriding /api/2.0/preview/scim/v2/Me, which we use in some tests (e.g.
bundle/templates-machinery/helpers-error). Previously the test passed
because CurrentUser.Me() was calling default server which is incorrect
but it happened to make the tests pass.
- The default server is now available on DATABRICKS_DEFAULT_HOST env
var.
- Rewrite "not found" handler in local test to handle error better (do
not raise http500 when header is already written).

## Tests
New acceptance test selftest/server specifically tests that both custom
and default handlers are available in a single test.
2025-02-11 15:03:41 +00:00
..
auth/bundle_and_profile acc: Allow mixing custom stubs with default server impl (#2334) 2025-02-11 15:03:41 +00:00
bin Always print warnings and errors; clean up format (#2213) 2025-02-07 11:29:40 +00:00
bundle Append newline to "-o json" for validate/summary/run (#2326) 2025-02-10 14:00:49 +01:00
help Fix duplicate "apps" entry in help output (#2191) 2025-01-20 16:02:29 +00:00
selftest acc: Allow mixing custom stubs with default server impl (#2334) 2025-02-11 15:03:41 +00:00
terraform Upgrade to TF provider 1.65.1 (#2328) 2025-02-10 14:06:02 +00:00
workspace/jobs Serialize recorded requests with indentation in acceptance tests (#2329) 2025-02-10 19:03:27 +05:30
.gitignore Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
README.md Add acceptance/selftest, showcasing basic features (#2229) 2025-01-27 09:17:22 +01:00
acceptance_test.go acc: Allow mixing custom stubs with default server impl (#2334) 2025-02-11 15:03:41 +00:00
cmd_server_test.go acc: Allow mixing custom stubs with default server impl (#2334) 2025-02-11 15:03:41 +00:00
config_test.go acc: Implement config merge (#2294) 2025-02-07 17:38:27 +01:00
install_terraform.py Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
script.cleanup Add acceptance tests (#2081) 2025-01-08 12:41:08 +00:00
script.prepare acc: Disable git hooks (#2249) 2025-01-28 14:00:41 +00:00
server_test.go acc: Allow mixing custom stubs with default server impl (#2334) 2025-02-11 15:03:41 +00:00

README.md

Acceptance tests are blackbox tests that are run against compiled binary.

Currently these tests are run against "fake" HTTP server pretending to be Databricks API. However, they will be extended to run against real environment as regular integration tests.

To author a test,

  • Add a new directory under acceptance. Any level of nesting is supported.
  • Add databricks.yml there.
  • Add script with commands to run, e.g. $CLI bundle validate. The test case is recognized by presence of script.

The test runner will run script and capture output and compare it with output.txt file in the same directory.

In order to write output.txt for the first time or overwrite it with the current output pass -update flag to go test.

The scripts are run with bash -e so any errors will be propagated. They are captured in output.txt by appending Exit code: N line at the end.

For more complex tests one can also use:

  • errcode helper: if the command fails with non-zero code, it appends Exit code: N to the output but returns success to caller (bash), allowing continuation of script.
  • trace helper: prints the arguments before executing the command.
  • custom output files: redirect output to custom file (it must start with out), e.g. $CLI bundle validate > out.txt 2> out.error.txt.

See selftest for a toy test.