databricks-cli/acceptance
Denis Bilenko e2db0cd0e2
Remove bundle.{Seq,If,Defer,newPhase,logString}, switch to regular functions (#2390)
## Changes
- Instead of constructing chains of mutators and then executing them,
execute them directly.
- Remove functionality related to chain-building: Seq, If, Defer,
newPhase, logString.
- Phases become functions that apply the changes directly rather than
construct mutator chains that will be called later.
- Add a helper ApplySeq to call multiple mutators, use it where
Apply+Seq were used before.

This is intended to be a refactoring without functional changes, but
there are a few behaviour changes:
- Since defer() is used to call unlock instead of bundle.Defer()
unlocking will now happen even in case of panics.
- In --debug, the phase names are are still logged once before start of
the phase but each entry no longer has 'seq' or phase name in it.
- The message "Deployment complete!" was printed even if
terraform.Apply() mutator had an error. It no longer does that.

## Motivation

The use of the chains was necessary when mutators were returning a list
of other mutators instead of calling them directly. But that has since
been removed, so now the chain machinery have no purpose anymore.

Use of direct functions simplifies the logic and makes bugs more
apparent and easy to fix.

Other improvements that this unlocks:
- Simpler stacktraces/debugging (breakpoints).
- Use of functions with narrowly scoped API: instead of mutators that
receive full bundle config, we can use focused functions that only deal
with sections they care about prepareGitSettings(currentGitSection) ->
updatedGitSection. This makes the data flow more apparent.
- Parallel computations across mutators (within phase): launch
goroutines fetching data from APIs at the beggining, process them once
they are ready.

## Tests
Existing tests.
2025-02-27 11:41:58 +00:00
..
auth acc: replace LocalOnly option with Local & Cloud (#2387) 2025-02-26 16:01:49 +00:00
bin acc: Support multiline patterns in diff.py (#2384) 2025-02-26 14:25:39 +01:00
bundle Remove bundle.{Seq,If,Defer,newPhase,logString}, switch to regular functions (#2390) 2025-02-27 11:41:58 +00:00
cmd/workspace/apps acc: replace LocalOnly option with Local & Cloud (#2387) 2025-02-26 16:01:49 +00:00
help Fix duplicate "apps" entry in help output (#2191) 2025-01-20 16:02:29 +00:00
selftest acc: replace LocalOnly option with Local & Cloud (#2387) 2025-02-26 16:01:49 +00:00
terraform acc: replace LocalOnly option with Local & Cloud (#2387) 2025-02-26 16:01:49 +00:00
workspace/jobs acc: replace LocalOnly option with Local & Cloud (#2387) 2025-02-26 16:01:49 +00:00
.gitignore Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
README.md Add acceptance/selftest, showcasing basic features (#2229) 2025-01-27 09:17:22 +01:00
acceptance_test.go acc: replace LocalOnly option with Local & Cloud (#2387) 2025-02-26 16:01:49 +00:00
cmd_server_test.go acc: Simplify writing handlers; support headers in responses (#2338) 2025-02-12 13:00:57 +00:00
config_test.go acc: replace LocalOnly option with Local & Cloud (#2387) 2025-02-26 16:01:49 +00:00
install_terraform.py Use real terraform in acceptance tests (#2267) 2025-01-31 13:53:13 +00:00
script.cleanup Add acceptance tests (#2081) 2025-01-08 12:41:08 +00:00
script.prepare acc: Disable git hooks (#2249) 2025-01-28 14:00:41 +00:00
server_test.go Support serverless mode in default-python template (explicit prompt) (#2377) 2025-02-26 14:07:30 +01:00
test.toml acc: replace LocalOnly option with Local & Cloud (#2387) 2025-02-26 16:01:49 +00:00

README.md

Acceptance tests are blackbox tests that are run against compiled binary.

Currently these tests are run against "fake" HTTP server pretending to be Databricks API. However, they will be extended to run against real environment as regular integration tests.

To author a test,

  • Add a new directory under acceptance. Any level of nesting is supported.
  • Add databricks.yml there.
  • Add script with commands to run, e.g. $CLI bundle validate. The test case is recognized by presence of script.

The test runner will run script and capture output and compare it with output.txt file in the same directory.

In order to write output.txt for the first time or overwrite it with the current output pass -update flag to go test.

The scripts are run with bash -e so any errors will be propagated. They are captured in output.txt by appending Exit code: N line at the end.

For more complex tests one can also use:

  • errcode helper: if the command fails with non-zero code, it appends Exit code: N to the output but returns success to caller (bash), allowing continuation of script.
  • trace helper: prints the arguments before executing the command.
  • custom output files: redirect output to custom file (it must start with out), e.g. $CLI bundle validate > out.txt 2> out.error.txt.

See selftest for a toy test.