Commit Graph

1647 Commits

Author SHA1 Message Date
Shreyas Goenka 39ff2909db
pass test 2025-02-04 15:52:39 +01:00
Shreyas Goenka 918af62827
remove eventually files 2025-02-04 15:44:03 +01:00
Shreyas Goenka 981dbf787d
add bash script for waiting 2025-02-04 15:34:25 +01:00
Shreyas Goenka 0423b09733
add filtering for auth 2025-02-03 20:31:25 +01:00
Shreyas Goenka 403f61228d
address comments 2025-02-03 13:41:21 +01:00
Shreyas Goenka f3e7594f39
fx test 2025-02-03 13:14:38 +01:00
Shreyas Goenka 2cbc39fdc9
- 2025-02-03 13:10:42 +01:00
Shreyas Goenka 2cd25e388e
- 2025-02-03 12:24:32 +01:00
Shreyas Goenka dc0ab300dd
- 2025-02-03 12:22:22 +01:00
Shreyas Goenka 5c2205a6f7
- 2025-02-03 12:17:25 +01:00
Shreyas Goenka a8b366ee79
- 2025-02-03 12:16:49 +01:00
Shreyas Goenka 4f979007af
- 2025-02-03 12:14:23 +01:00
Shreyas Goenka c1a322555a
fx test 2025-02-03 12:09:54 +01:00
Shreyas Goenka 5385faf7d8
fix test 2025-02-03 11:54:01 +01:00
Denis Bilenko f267318bb9
Include acceptance tests in integration tests (#2242)
## Changes
- Include acceptance directory in integration tests. Acceptance tests
will not start local server if CLOUD_ENV is set, so they become
integration tests.
- Add dependency for vendor to integration, so that CLI can be build
there.
- Implement LocalOnly option in test.toml to opt out of running
acceptance tests as integration tests. Use it in certain tests that are
difficult or not necessary to fix when run as integration tests.
- Update terraform test to redact timings out.
- Clean up .workspace.current_user from outputs of the tests.

## Tests
Existing tests.
2025-02-03 10:43:25 +00:00
Shreyas Goenka 88015876ad
pass test 2025-02-03 11:42:17 +01:00
Denis Bilenko fcedfe4c78
acc: Consistent & detailed output for file issues (#2279)
## Changes
- Include compact relPath in the error message title. Include full paths
in separate lines below.
- Previously sometimes full paths were printed, sometime only rel path.

## Tests
Manually trigger the errors.
2025-02-03 10:29:13 +00:00
Denis Bilenko 2f798c4ded
acc: Remove initial '$CLI --version' call (#2280)
It is proven to be not necessary.

```
~/work/cli/acceptance % hyperfine -w 2 'go test'  # with change:
Benchmark 1: go test
  Time (mean ± σ):      4.983 s ±  0.209 s    [User: 6.073 s, System: 9.869 s]
  Range (min … max):    4.792 s …  5.483 s    10 runs

~/work/cli/acceptance % git stash  # without change:
~/work/cli/acceptance % hyperfine -w 2 'go test'
Benchmark 1: go test
  Time (mean ± σ):      5.018 s ±  0.100 s    [User: 6.142 s, System: 10.234 s]
  Range (min … max):    4.899 s …  5.182 s    10 runs
```
2025-02-03 10:03:18 +00:00
Shreyas Goenka da0cf951b9
- 2025-02-03 06:28:37 +01:00
Shreyas Goenka c412eb7666
- 2025-02-03 06:28:16 +01:00
Shreyas Goenka 382efe41f8
- 2025-02-03 06:27:34 +01:00
Shreyas Goenka d7bf1dc87e
add test for upload 2025-02-03 05:19:26 +01:00
Shreyas Goenka e4a1f42737
- 2025-02-03 04:40:01 +01:00
Shreyas Goenka 259a21a120
- 2025-02-03 04:33:21 +01:00
Shreyas Goenka 90148d8a50
replace os 2025-02-03 04:26:58 +01:00
Shreyas Goenka f09a780887
fix panic 2025-02-03 04:24:09 +01:00
Shreyas Goenka b83e57621e
- 2025-02-03 04:20:24 +01:00
Shreyas Goenka 427c755ea7
major cleanup 2025-02-03 04:18:38 +01:00
Shreyas Goenka 5d75c3f098
Merge remote-tracking branch 'origin' into implement-async-logger 2025-02-03 00:21:02 +01:00
Denis Bilenko e5730bf57e
Use real terraform in acceptance tests (#2267)
## Changes
- Add a script install_terraform.py that downloads terraform and
provider and generates a config to use, inspired by
https://gist.github.com/pietern/1cb6b6f3e0a452328e13cdc75031105e
- Make acceptance tests run this script once before running the tests
and set the required env vars to make cli use this terraform
installation.
- Use OS-specific directory for things that are build by acceptance test
runner (CLI and terraform).

This enables acceptance tests against cloud #2242 and local test for
bundle deploy #2254.

## Tests
- Add an acceptance test for standalone terraform. This is useful to
debug terraform with TF_LOG=DEBUG to see that it uses local provider.
- Other acceptance tests are updated with regard to terraform exec path.
- The overall time for tests locally is unchanged (if terraform is
already fetched).
2025-01-31 13:53:13 +00:00
shreyas-goenka 787dbe9099
Add request body assertions to acceptance tests (#2263)
## Changes
With this PR, any acceptance tests that define custom server stubs in
`test.toml` will automatically record all HTTP requests made and assert
on them.

Builds on top of https://github.com/databricks/cli/pull/2226

## Tests
Modifying existing acceptance test.
2025-01-31 13:31:23 +00:00
shreyas-goenka 3c6eacb05b
Add feature to mock server APIs in acceptance tests (#2226)
## Changes
This PR allows us to define custom server stubs in a `test.toml` file. 

Note: A followup PR will add functionality to do assertions on the API
request itself.

## Tests
New acceptance test.
2025-01-30 10:43:07 +00:00
Denis Bilenko f1efbd7d9f
acc: add -norepl flag that disables replacements (for debugging) (#2269) 2025-01-30 10:38:54 +00:00
Denis Bilenko a03ea73011
Add ruff.toml with increased line-length (#2268)
The default is 88 which reformats too much.

This has no effect on templates but affects Python script in this PR
https://github.com/databricks/cli/pull/2267

For context, we do not set any line length for golang and have 177 .go
files with max line length 150 or more.
2025-01-30 09:52:41 +00:00
Denis Bilenko 58ef34f320
acc: Include "id" into /api/2.0/preview/scim/v2/Me response (#2266)
This is something terraform provider expects.

Related to https://github.com/databricks/cli/pull/2242
2025-01-29 17:35:03 +00:00
shreyas-goenka 55c03cc119
Always close test HTTP server during cleanup (#2261)
## Changes
This PR registers the `server.Close()` function to be run during test
cleanup in the server initialization function. This ensures that all
test servers are closed as soon as the test they are scoped to finish.

Motivated by https://github.com/databricks/cli/pull/2255/files where a
regression was introduced where we did not close the test server.

## Tests
N/A
2025-01-29 15:54:33 +00:00
Andrew Nester ce965b22b2
[Release] Release v0.240.0 (#2264)
Bundles:
* Added support for double underscore variable references
([#2203](https://github.com/databricks/cli/pull/2203)).
* Do not wait for app compute to start on `bundle deploy`
([#2144](https://github.com/databricks/cli/pull/2144)).
* Remove bundle.git.inferred
([#2258](https://github.com/databricks/cli/pull/2258)).
* libs/python: Remove DetectInterpreters
([#2234](https://github.com/databricks/cli/pull/2234)).

API Changes:
 * Added `databricks access-control` command group.
 * Added `databricks serving-endpoints http-request` command.
* Changed `databricks serving-endpoints create` command with new
required argument order.
* Changed `databricks serving-endpoints get-open-api` command return
type to become non-empty.
* Changed `databricks recipients update` command return type to become
non-empty.

OpenAPI commit 0be1b914249781b5e903b7676fd02255755bc851 (2025-01-22)
Dependency updates:
* Bump github.com/databricks/databricks-sdk-go from 0.55.0 to 0.56.1
([#2238](https://github.com/databricks/cli/pull/2238)).
* Upgrade TF provider to 1.64.1
([#2247](https://github.com/databricks/cli/pull/2247)).
2025-01-29 16:55:53 +01:00
Shreyas Goenka f092e21594
add back worker input 2025-01-29 16:45:21 +01:00
Shreyas Goenka 8f8463f665
- 2025-01-29 16:05:25 +01:00
Shreyas Goenka ee3568cf64
-' 2025-01-29 16:02:41 +01:00
Shreyas Goenka acd64fa296
merge 2025-01-29 16:02:09 +01:00
Shreyas Goenka 155fe7b83d
- 2025-01-29 16:00:23 +01:00
Denis Bilenko 38efedcd73
Remove bundle.git.inferred (#2258)
The only use case for it was to emit a warning and based on the
discussion here
https://github.com/databricks/cli/pull/2213/files#r1933558087 the
warning it not useful and logging that with reduced severity is also not
useful.
2025-01-29 14:15:52 +00:00
shreyas-goenka c3a6e11627
Add integration test for the /telemetry-ext endpoint (#2259)
## Changes
Followup from
https://github.com/databricks/cli/pull/2209#pullrequestreview-2580308075.

This PR adds an integration test to validate that the API type bindings
work against the telemetry endpoint.

## Tests
N/A

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2025-01-29 14:05:58 +00:00
Gleb Kanterov 13596eb605
PythonMutator: Fix relative path error (#2253)
## Changes

Fix relative path errors in the Python mutator that was failing during
deployment since v0.239.1.

Before that:

```
% databricks bundle deploy  
Deploying resources...
Updating deployment state...
Error: failed to compute relative path for job jobs_as_code_project_job: Rel: can't make resources/jobs_as_code_project_job.py relative to /Users/$USER/jobs_as_code_project
```

As a result, the bundle was deployed, but the deployment state wasn't
updated.

## Tests

Unit tests, adding acceptance tests in
https://github.com/databricks/cli/pull/2254
2025-01-29 13:56:57 +00:00
Andrew Nester ec7808da34
Added support for double underscore variable references (#2203)
## Changes
Added support for double underscore variable references.

Previously we made this restriction stronger with no particular reason,
TF provider supports multiple underscores and thus DABs should do as
well.

Fixes #1753

## Tests
Added acceptance and integration tests
2025-01-29 13:38:28 +00:00
Ilya Kuznetsov 59d6fbfee9
Restore variable file tests (#2220)
## Changes

Uncomment flaky tests, they work properly with latest changes from main 

## Tests
<!-- How is this tested? -->
2025-01-29 13:34:26 +00:00
Ilya Kuznetsov 708c4fbb7a
Autogenerated documentation for bundle config (#2033)
## Changes

Documentation autogeneration tool. This tool uses same annotations_*.yml
files as in json-schema

Result will go
[there](https://docs.databricks.com/en/dev-tools/bundles/reference.html)
and
[there](https://docs.databricks.com/en/dev-tools/bundles/resources.html#cluster)

## Tests
Manually
2025-01-29 12:14:21 +00:00
shreyas-goenka 30f57d3b49
Add protos for bundle telemetry (#2209)
## Changes
These types correspond to the telemetry protobufs defined in universe.

## Tests
No tests are needed since this PR only adds the type bindings.

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2025-01-29 11:49:53 +00:00
shreyas-goenka 884b5f26ed
Set bundle auth configuration in command context (#2195)
## Changes

This change is required to enable tracking execution time telemetry for
bundle commands. In order to track execution time for the command
generally, we need to have the databricks auth configuration available
at this section of the code:


41bbd89257/cmd/root/root.go (L99)

In order to do this we can rely on the `configUsed` context key.   

Most commands rely on the `root.MustWorkspaceClient` function which
automatically sets the client config in the `configUsed` context key.
Bundle commands, however, do not do so. They instead store their
workspace clients in the `&bundle.Bundle{}` object.

With this PR, the `configUsed` context key will be set for all `bundle`
commands. Functionally nothing changes.

## Tests
Existing tests. Also manually verified that either
`root.MustConfigureBundle` or `utils.ConfigureBundleWithVariables` is
called for all bundle commands (except `bundle init`) thus ensuring this
context key would be set for all bundle commands.

refs for the functions:
1. `root.MustConfigureBundle`:
41bbd89257/cmd/root/bundle.go (L88)
2. `utils.ConfigureBundleWithVariables`:
41bbd89257/cmd/bundle/utils/utils.go (L19)

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2025-01-29 11:02:08 +00:00