Commit Graph

84 Commits

Author SHA1 Message Date
Shreyas Goenka c2eaf1ae49
- 2025-01-29 16:32:29 +01:00
Shreyas Goenka b93fa07917
Revert "-"
This reverts commit 76ca212641.
2025-01-29 16:15:50 +01:00
Shreyas Goenka 76ca212641
- 2025-01-29 16:15:28 +01:00
Shreyas Goenka bb87563d23
inlint append 2025-01-29 16:12:35 +01:00
Shreyas Goenka 8da6c57b86
address comments 2025-01-29 16:07:49 +01:00
Shreyas Goenka 8505abd683
- 2025-01-29 15:37:03 +01:00
Shreyas Goenka ca17487b8e
- 2025-01-29 15:25:22 +01:00
Shreyas Goenka 3e7acdb373
- 2025-01-29 14:58:48 +01:00
Shreyas Goenka 005589a415
- 2025-01-29 14:55:34 +01:00
Shreyas Goenka badfb9c133
- 2025-01-29 14:54:24 +01:00
Shreyas Goenka 1f1d705de9
move config to toml 2025-01-29 14:51:36 +01:00
Shreyas Goenka 3549c22b40
- 2025-01-29 12:57:33 +01:00
Shreyas Goenka 39477d5650
- 2025-01-29 12:55:58 +01:00
Shreyas Goenka 7c5d65feb1
allow override test to run in parallel 2025-01-29 12:08:33 +01:00
Shreyas Goenka af6ae9c14a
Merge remote-tracking branch 'origin' into validate-response-body 2025-01-28 22:27:05 +01:00
Denis Bilenko 4ba222ab36
Fix env_overrides not to use variables in workspace.profile (#2251)
This does not work when this test is run against cloud.

Needed for https://github.com/databricks/cli/pull/2242
2025-01-28 15:22:56 +01:00
Denis Bilenko 5971bd5c1a
acc: Disable git hooks (#2249)
Otherwise hooks from universe and custom hooks run in tests.
2025-01-28 14:00:41 +00:00
shreyas-goenka 65e4f79dfe
Switch to using `[` from `<` in text replacements (#2224)
## Changes
Noticed this when working on
https://github.com/databricks/cli/pull/2221. `<` is a special HTML
character that is encoded during text replacement when using
`AssertEqualTexts`.


## Tests
N/A
2025-01-28 10:54:23 +00:00
Denis Bilenko 3ffac80007
acc: Use real terraform when CLOUD_ENV is set (#2245)
## Changes
- If CLOUD_ENV is set to do not override with dummy value. This allows
running acceptance tests as integration tests.
- Needed for https://github.com/databricks/cli/pull/2242

## Tests
Manually run the test suite against dogfood. `CLOUD_ENV=aws go test
./acceptance`
2025-01-28 10:23:44 +00:00
Denis Bilenko 11436faafe
acc: Avoid reading and applying replacements on large files; validate utf8 (#2244)
## Changes
- Do not start replacement / comparison if file is too large or not
valid utf-8.
- This helps to prevent replacements if there is accidentally a large
binary (e.g. terraform).

## Tests
Found this problem when working on
https://github.com/databricks/cli/pull/2242 -- the tests tried to
applied replacements on terraform binary and crashed. With this change,
an error is reported instead.
2025-01-28 10:22:29 +00:00
Denis Bilenko 60709e3d48
acc: Restore unexpected output error (#2243)
## Changes
Restore original behaviour of acceptance tests: any unaccounted for
files trigger an error (not just those that start with "out"). This got
changed in
https://github.com/databricks/cli/pull/2146/files#diff-2bb968d823f4afb825e1dcea2879bdbdedf2b7c15d4e77f47905691b14246a04L196
which started only checking files starting with "out*" and skipping
everything else.

## Tests
Existing tests.
2025-01-28 10:15:32 +00:00
Denis Bilenko be908ee1a1
Add acceptance test for 'experimental.scripts' (#2240) 2025-01-27 15:28:33 +00:00
Denis Bilenko 67d1413db5
Add default regex for DEV_VERSION (#2241)
## Changes

- Replace development version with $DEV_VERSION
- Update experimental-jobs-as-code to make use of it.

## Tests
- Existing tests.
- Using this in https://github.com/databricks/cli/pull/2213
2025-01-27 15:34:53 +01:00
Denis Bilenko 52bf7e388a
acc: Propagate user's UV_CACHE_DIR to tests (#2239)
There is a speed up in 0.5s but it is still 4.4s, so something else is
slow there.

Benchmarking bundle/templates/experimental-jobs-as-code:

```
# Without UV_CACHE_DIR
~/work/cli/acceptance/bundle/templates/experimental-jobs-as-code % hyperfine --warmup 2 'testme -count=1'
Benchmark 1: testme -count=1
  Time (mean ± σ):      4.950 s ±  0.079 s    [User: 2.730 s, System: 8.524 s]
  Range (min … max):    4.838 s …  5.076 s    10 runs

# With UV_CACHE_DIR
~/work/cli/acceptance/bundle/templates/experimental-jobs-as-code % hyperfine --warmup 2 'testme -count=1'
Benchmark 1: testme -count=1
  Time (mean ± σ):      4.410 s ±  0.049 s    [User: 2.669 s, System: 8.710 s]
  Range (min … max):    4.324 s …  4.467 s    10 runs
```
2025-01-27 15:25:56 +01:00
Denis Bilenko b7dd70b8b3
acc: Add a couple of error tests for 'bundle init' (#2233)
This captures how we log errors related to subprocess run and what does
the output look like.
2025-01-27 12:22:40 +00:00
Denis Bilenko 1cb32eca90
acc: Support custom replacements (#2231)
## Changes
- Ability to extend a list of replacements via test.toml
- Modify selftest to both demo this feature and to get rid of sed on
Windows.

## Tests
Acceptance tests. I'm also using it
https://github.com/databricks/cli/pull/2213 for things like pid.
2025-01-27 09:11:06 +00:00
Denis Bilenko 82b0dd36d6
Add acceptance/selftest, showcasing basic features (#2229)
Also make TestInprocessMode use this test.
2025-01-27 09:17:22 +01:00
Denis Bilenko b3d98fe666
acc: Print replacements on error and rm duplicates (#2230)
## Changes
- File comparison files in acceptance test, print the contents of all
applied replacements. Do it once per test.
- Remove duplicate entries in replacement list.

## Tests
Manually, change out files of existing test, you'll get this printed
once, after first assertion:

```
        acceptance_test.go:307: Available replacements:
            REPL /Users/denis\.bilenko/work/cli/acceptance/build/databricks => $$CLI
            REPL /private/var/folders/5y/9kkdnjw91p11vsqwk0cvmk200000gp/T/TestAccept598522733/001 => $$TMPHOME
            ...
```
2025-01-27 07:45:09 +00:00
Denis Bilenko 468660dc45
Add an acc test covering failures when reading .git (#2223)
## Changes
- New test covering failures in reading .git. One case results in error,
some result in warning (not shown).
- New helper withdir runs commands in a subdirectory.

## Tests
New acceptance test.
2025-01-24 15:53:06 +00:00
Denis Bilenko 959e43e556
acc: Support per-test configuration; GOOS option to disable OS (#2227)
## Changes
- Acceptance tests load test.toml to configure test behaviour.
- If file is not found in the test directory, parents are searched,
until the test root.
- Currently there is one option: runtime.GOOS to switch off tests per
OS.

## Tests
Using it in https://github.com/databricks/cli/pull/2223 to disable test
on Windows that cannot be run there.
2025-01-24 14:28:23 +00:00
Shreyas Goenka 24be18a2c7
merge 2025-01-24 14:25:53 +01:00
Shreyas Goenka fffc89c6d5
- 2025-01-24 14:22:11 +01:00
Shreyas Goenka accec7118c
remove request bits 2025-01-24 14:17:56 +01:00
Shreyas Goenka 0d14526098
move server to libs 2025-01-24 14:05:37 +01:00
shreyas-goenka a47a058506
Limit test server to only accept GET on read endpoints (#2225)
## Changes
Now the test server will only match GET queries for these endpoints

## Tests
Existing tests.
2025-01-24 11:05:00 +00:00
Shreyas Goenka 09791fc16a
decouple changes 2025-01-24 12:03:11 +01:00
Denis Bilenko b4ed235104
Include EvalSymlinks in SetPath and use SetPath on all paths (#2219)
## Changes
When adding path, a few things should take care of:
- symlink expansion
- forward/backward slashes, so that tests could do sed 's/\\\\/\//g' to
make it pass on Windows (see
acceptance/bundle/syncroot/dotdot-git/script)

SetPath() function takes care of both.

This PR uses SetPath() on all paths consistently.

## Tests
Existing tests.
2025-01-24 10:18:44 +00:00
Denis Bilenko d6d9b994d4
acc: only print non-zero exit codes in errcode function (#2222)
Reduce noise in the output and matches how "Exit code" is handled for
the whole script.
2025-01-24 10:47:12 +01:00
Shreyas Goenka de326bf933
- 2025-01-23 20:13:16 +01:00
Shreyas Goenka 07c25584d3
Add support for mocking servers in acceptance tests 2025-01-23 19:56:58 +01:00
Ilya Kuznetsov 0487e816cc
Reading variables from file (#2171)
## Changes

New source of default values for variables - variable file
`.databricks/bundle/<target>/variable-overrides.json`

CLI tries to stat and read that file every time during variable
initialisation phase

<!-- Summary of your changes that are easy to understand -->

## Tests

Acceptance tests
2025-01-23 14:35:33 +00:00
Andrew Nester 8af9efaa62
Show an error when non-yaml files used in include section (#2201)
## Changes
`include` section is used only to include other bundle configuration
YAML files. If any other file type is used, raise an error and guide
users to use `sync.include` instead

## Tests
Added acceptance test

---------

Co-authored-by: Julia Crawford (Databricks) <julia.crawford@databricks.com>
2025-01-23 13:58:18 +00:00
Denis Bilenko 1f63aa0912
tests: Improve reporting in case of FS errors (#2216)
## Changes
If there are unreadable files in a directory, raise an error but
continue with further diagnostics, because the answer is in the script
output.

## Tests
Manually - I'm working on some tests that create unreadable files, the
report is much better with this change.
2025-01-23 11:46:22 +00:00
Denis Bilenko ba3a400327
Remove test-specific logic from generic test runner (#2215)
Revert changes to acceptance_test.go added in #2177 and add
test-specific fix.
2025-01-23 11:59:01 +01:00
Denis Bilenko 20c1902a45
Fix passing SingleTest to TestAccept (#2210) 2025-01-22 16:26:16 +00:00
Denis Bilenko 876526a19a
Use local git config in tests (#2205)
I've seen this error: could not lock config file
$TMPDIR_GPARENT/TestAccept3968313522/002/.gitconfig: File exists

This is likely the cause.
2025-01-22 12:20:49 +01:00
Denis Bilenko e9902036b8
Set WorktreeRoot to sync root outside git repo (#2197)
## Changes
If git is not detected, set default worktree root to sync root.
Otherwise NewFileSet/View raise an error about worktree root being
outside view root in acceptance/bundle/sync-paths-dotdot.

This behavior is introduced in
https://github.com/databricks/cli/pull/1945

Stacked on https://github.com/databricks/cli/pull/2202

## Tests
Existing tests.
2025-01-22 10:50:13 +00:00
Denis Bilenko fde30ff1ab
Add a test for sync root outside of git root (#2202)
- Move acceptance/bundle/sync-paths-dotdot test to
acceptance/bundle/syncroot/dotdot-notgit
- Add new test acceptance/bundle/syncroot/dotdot-git

Fix replacer to work with this test and on Windows:
- Make PATH work on Windows by using EvalSymlinks.
- Make concatenated path match within JSON but stripping quotes.
2025-01-22 10:17:45 +00:00
Denis Bilenko 3a32c63919
Add -inprocess mode for acceptance tests (#2184)
## Changes
- If you pass -inprocess flag to acceptance tests, they will run in the
same process as test itself. This enables debugging.
- If you set singleTest variable on top of acceptance_test.go, you'll
only run that test and with inprocess mode. This is intended for
debugging in VSCode.
- (minor) Converted KeepTmp to flag -keeptmp from env var KEEP_TMP for
consistency with other flags.

## Tests
- I verified that acceptance tests pass with -inprocess mode: `go test
-inprocess < /dev/null | cat`
- I verified that debugging in VSCode works: set a test name in
singleTest variable, set breakpoints inside CLI and click "debug test"
in VSCode.
2025-01-21 21:21:12 +00:00
Denis Bilenko 34a37cf4a8
Clone ReplacementContext before passing into test (#2198)
## Changes
- Add a new method Clone() on ReplacementContext
- Use it when passing common replacements to test cases.

## Tests
Manually. I have a different branch where this bug manifested and this
change helped.
2025-01-21 12:47:34 +00:00