databricks-cli/acceptance
Denis Bilenko 54e16d5f62
Always print warnings and errors; clean up format (#2213)
## Changes
- Print warnings and errors by default.
- Fix ErrAlreadyPrinted not to be logged at Error level.
- Format log messages as "Warn: message" instead of "WARN" to make it
more readable and in-line with the rest of the output.
- Only print attributes (pid, mutator, etc) and time when the overall
level is debug (so --debug output has not changed much).

## Tests
- Existing acceptance tests show how warning messages appear in various
test case.
- Added new test for `--debug` output.
- Add sort_lines.py helper to avoid dependency on 'sort' which is
locale-sensitive.
2025-02-07 11:29:40 +00:00
..
auth/bundle_and_profile acc: Added acceptance test for CLI commands inside bundle with and without profile flag (#2270) 2025-02-05 11:53:36 +00:00
bin Always print warnings and errors; clean up format (#2213) 2025-02-07 11:29:40 +00:00
bundle Always print warnings and errors; clean up format (#2213) 2025-02-07 11:29:40 +00:00
help Fix duplicate "apps" entry in help output (#2191) 2025-01-20 16:02:29 +00:00
selftest acc: Use [VARNAME] instead of $VARNAME (#2282) 2025-02-03 14:10:19 +00:00
terraform acc: Use [VARNAME] instead of $VARNAME (#2282) 2025-02-03 14:10:19 +00:00
workspace/jobs Extend testserver for deployment (#2299) 2025-02-07 10:26:20 +00:00
.gitignore Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
README.md Add acceptance/selftest, showcasing basic features (#2229) 2025-01-27 09:17:22 +01:00
acceptance_test.go Extend testserver for deployment (#2299) 2025-02-07 10:26:20 +00:00
cmd_server_test.go Extend testserver for deployment (#2299) 2025-02-07 10:26:20 +00:00
config_test.go Add ability to record headers in acceptance tests (#2296) 2025-02-05 09:32:15 +00:00
install_terraform.py Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
script.cleanup Add acceptance tests (#2081) 2025-01-08 12:41:08 +00:00
script.prepare acc: Disable git hooks (#2249) 2025-01-28 14:00:41 +00:00
server_test.go Extend testserver for deployment (#2299) 2025-02-07 10:26:20 +00:00
test.toml acc: Support per-test configuration; GOOS option to disable OS (#2227) 2025-01-24 14:28:23 +00:00

README.md

Acceptance tests are blackbox tests that are run against compiled binary.

Currently these tests are run against "fake" HTTP server pretending to be Databricks API. However, they will be extended to run against real environment as regular integration tests.

To author a test,

  • Add a new directory under acceptance. Any level of nesting is supported.
  • Add databricks.yml there.
  • Add script with commands to run, e.g. $CLI bundle validate. The test case is recognized by presence of script.

The test runner will run script and capture output and compare it with output.txt file in the same directory.

In order to write output.txt for the first time or overwrite it with the current output pass -update flag to go test.

The scripts are run with bash -e so any errors will be propagated. They are captured in output.txt by appending Exit code: N line at the end.

For more complex tests one can also use:

  • errcode helper: if the command fails with non-zero code, it appends Exit code: N to the output but returns success to caller (bash), allowing continuation of script.
  • trace helper: prints the arguments before executing the command.
  • custom output files: redirect output to custom file (it must start with out), e.g. $CLI bundle validate > out.txt 2> out.error.txt.

See selftest for a toy test.