databricks-cli/acceptance
Denis Bilenko e5730bf57e
Use real terraform in acceptance tests (#2267)
## Changes
- Add a script install_terraform.py that downloads terraform and
provider and generates a config to use, inspired by
https://gist.github.com/pietern/1cb6b6f3e0a452328e13cdc75031105e
- Make acceptance tests run this script once before running the tests
and set the required env vars to make cli use this terraform
installation.
- Use OS-specific directory for things that are build by acceptance test
runner (CLI and terraform).

This enables acceptance tests against cloud #2242 and local test for
bundle deploy #2254.

## Tests
- Add an acceptance test for standalone terraform. This is useful to
debug terraform with TF_LOG=DEBUG to see that it uses local provider.
- Other acceptance tests are updated with regard to terraform exec path.
- The overall time for tests locally is unchanged (if terraform is
already fetched).
2025-01-31 13:53:13 +00:00
..
bin Add -inprocess mode for acceptance tests (#2184) 2025-01-21 21:21:12 +00:00
bundle Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
help Fix duplicate "apps" entry in help output (#2191) 2025-01-20 16:02:29 +00:00
selftest Add default regex for DEV_VERSION (#2241) 2025-01-27 15:34:53 +01:00
terraform Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
workspace/jobs/create Add request body assertions to acceptance tests (#2263) 2025-01-31 13:31:23 +00:00
.gitignore Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
README.md Add acceptance/selftest, showcasing basic features (#2229) 2025-01-27 09:17:22 +01:00
acceptance_test.go Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
cmd_server_test.go Always close test HTTP server during cleanup (#2261) 2025-01-29 15:54:33 +00:00
config_test.go Add request body assertions to acceptance tests (#2263) 2025-01-31 13:31:23 +00:00
install_terraform.py Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
script.cleanup Add acceptance tests (#2081) 2025-01-08 12:41:08 +00:00
script.prepare acc: Disable git hooks (#2249) 2025-01-28 14:00:41 +00:00
server_test.go acc: Include "id" into /api/2.0/preview/scim/v2/Me response (#2266) 2025-01-29 17:35:03 +00:00
test.toml acc: Support per-test configuration; GOOS option to disable OS (#2227) 2025-01-24 14:28:23 +00:00

README.md

Acceptance tests are blackbox tests that are run against compiled binary.

Currently these tests are run against "fake" HTTP server pretending to be Databricks API. However, they will be extended to run against real environment as regular integration tests.

To author a test,

  • Add a new directory under acceptance. Any level of nesting is supported.
  • Add databricks.yml there.
  • Add script with commands to run, e.g. $CLI bundle validate. The test case is recognized by presence of script.

The test runner will run script and capture output and compare it with output.txt file in the same directory.

In order to write output.txt for the first time or overwrite it with the current output pass -update flag to go test.

The scripts are run with bash -e so any errors will be propagated. They are captured in output.txt by appending Exit code: N line at the end.

For more complex tests one can also use:

  • errcode helper: if the command fails with non-zero code, it appends Exit code: N to the output but returns success to caller (bash), allowing continuation of script.
  • trace helper: prints the arguments before executing the command.
  • custom output files: redirect output to custom file (it must start with out), e.g. $CLI bundle validate > out.txt 2> out.error.txt.

See selftest for a toy test.