Commit Graph

384 Commits

Author SHA1 Message Date
Shreyas Goenka dd9442ef1a
Merge remote-tracking branch 'origin' into fs-rm 2023-06-06 01:17:07 +02:00
shreyas-goenka ae10419eb8
Add fs cat command for dbfs files (#430)
## Changes
TSIA

## Tests
Manually and integration tests
2023-06-06 01:16:23 +02:00
Shreyas Goenka 188ed170db
Merge remote-tracking branch 'origin' into fs-rm 2023-06-06 01:05:00 +02:00
Shreyas Goenka da652c9957
- 2023-06-06 01:03:20 +02:00
Andrew Nester df3f5863c7
Do not generate prompts for certain commands (#438)
## Changes
Some of the commands do not support prompts, for example `workspace
get-status` but we were wrongly suggesting customers some option.

Quick fix for this is not to provide prompts for these known commands.

Note: it uses a method from this PR in Go SDK
https://github.com/databricks/databricks-sdk-go/pull/416

## Tests
Running `workspace get-status`

Before
```
andrew.nester@HFW9Y94129 multiples-tasks % ../../cli/cli workspace get-status
Error: Path () doesn't start with '/'
```

After
```
andrew.nester@HFW9Y94129 multiples-tasks % ../../cli/cli workspace get-status
Error: accepts 1 arg(s), received 0

```
2023-06-05 19:38:45 +02:00
Shreyas Goenka 5619d0fed7
- 2023-06-05 17:48:55 +02:00
Shreyas Goenka b64b7ab1a7
- 2023-06-05 17:48:21 +02:00
Shreyas Goenka 8b4f22ce95
Merge remote-tracking branch 'origin' into fs-rm 2023-06-05 17:42:09 +02:00
shreyas-goenka 6ff00122ad
Add fs ls command for dbfs (#429)
## Changes
1. Adds fs ls command
2. Adds ability to define multiple templates

## Tests
Manually and integration tests
2023-06-05 17:41:30 +02:00
Andrew Nester 1f130f3722
Do not use FgWhite and FgBlack for terminal output (#435)
## Changes
Using white / black color for terminal output will lead to poorly
displayed content in either light or dark terminal backgrounds. Some
other CLIs experienced same issues
(https://github.com/qri-io/qri/pull/774)

Instead, let's just use color to highlight some of the output so it's
more compatible with different background styles

## Tests
<img width="772" alt="Screenshot 2023-06-05 at 16 05 09"
src="https://github.com/databricks/cli/assets/2969996/01790239-6a33-4059-86a8-d5117ea0b75f">

---

<img width="757" alt="Screenshot 2023-06-05 at 16 05 20"
src="https://github.com/databricks/cli/assets/2969996/ea3b9fdc-3782-4f4f-a9df-19e66af0c04f">
2023-06-05 17:30:40 +02:00
Andrew Nester 3dbf7a575a
Better error message if can not load prompts (#437)
## Changes
Better error message if can not load prompts

## Tests
Setup 2 jobs with the same name and ran `cli job get`

```
andrew.nester@HFW9Y94129 multiples-tasks % ../../cli/cli jobs get
Error: failed to load names for Jobs drop-down. Please manually specify required arguments. Original error: duplicate .Settings.Name: duplicatejob
```
2023-06-05 16:32:03 +02:00
Shreyas Goenka 25aa5b667c
- 2023-06-05 15:42:56 +02:00
Shreyas Goenka 4b8f5746ef
Merge branch 'fs-ls' into fs-rm 2023-06-05 15:42:09 +02:00
Shreyas Goenka 9160a6e9ed
initialize dbfs client at root 2023-06-05 15:15:21 +02:00
Shreyas Goenka 08826094ce
added preallocation of size 2023-06-05 15:09:09 +02:00
Shreyas Goenka 265e517126
address comments 2 2023-06-05 15:07:08 +02:00
Shreyas Goenka b048c8675d
address comments 2023-06-05 14:01:06 +02:00
Shreyas Goenka fc7fbbec6f
- 2023-06-05 13:55:29 +02:00
Shreyas Goenka 8050dc524f
move sort outside 2023-06-05 13:53:38 +02:00
Shreyas Goenka 19a20512e9
comments 2023-06-05 13:49:33 +02:00
Shreyas Goenka 54a6602aae
Add fs rm command for dbfs 2023-06-05 03:07:19 +02:00
Shreyas Goenka 3dc47709d0
manually check for dbfs prefix 2023-06-05 02:13:28 +02:00
Shreyas Goenka 492382caa1
added resolvedbfs path func 2023-06-05 01:34:28 +02:00
Shreyas Goenka db953acb2f
Merge remote-tracking branch 'origin' into fs-ls 2023-06-05 01:21:39 +02:00
Shreyas Goenka 6b667b5e01
added test for ls on flie 2023-06-05 01:20:54 +02:00
Shreyas Goenka 2706bebb38
lint 2023-06-05 01:10:15 +02:00
Shreyas Goenka a4879c0d91
added integration tests 2023-06-05 01:06:42 +02:00
Shreyas Goenka b5f5f1103f
Add fs ls command for dbfs 2023-06-02 17:24:04 +02:00
Pieter Noordhuis 28d36577a2
Speed up sync integration tests (#428)
## Changes

This change implements:
* Channels for line-by-line output from stdout/stderr
* A function to wait for a sync step to complete (using above)
* Ensure all tests are prefixed `TestAccSync`
* Use temporary paths in WSFS instead of cloning a repo

## Tests

The same integration tests now pass in ~90 seconds (was ~250s).
2023-06-02 14:02:18 +00:00
Pieter Noordhuis 1c0d67f66c
Add fs.FS adapter for the filer interface (#422)
## Changes

This enables the use of `io/fs` functions `fs.Glob` and `fs.WalkDir`
with filers.

We can't use `fs.FS` as the standard interface instead of `filer.Filer` because:
1. It was made for reading from filesystems only, not writing
2. It doesn't take a context for the core functions

Therefore a wrapper will do.

## Tests

* Added unit tests to cover the adapter through a fake filer.
* Manually ran `fs.WalkDir` against both WSFS and DBFS filers.
2023-06-02 12:49:59 +00:00
Andrew Nester f8255f356b
Added spinner when loading command prompts (#420)
## Changes
Added spinner when loading command prompts

## Tests
```
andrew.nester@HFW9Y94129 multiples-tasks % cli jobs get-run
⡿ No RUN_ID argument specified. Loading names for Jobs drop-down.```
2023-06-02 14:13:18 +02:00
Serge Smertin a6c9533c1c
Add profile on `databricks auth login` (#423)
## Changes
- added saving profile to `~/.databrickscfg` whenever we do `databricks
auth login`.
- we either match profile by account id / canonical host or introduce
the new one from deployment name.
- fail on multiple profiles with matching accounts or workspace hosts.
- overriding `~/.databrickscfg` keeps the (valid) comments, but
reformats the file.

## Tests
<!-- How is this tested? -->
- `make test`
- `go run main.go auth login --account-id XXX --host
https://accounts.cloud.databricks.com/`
- `go run main.go auth token --account-id XXX --host
https://accounts.cloud.databricks.com/`
- `go run main.go auth login --host https://XXX.cloud.databricks.com/`
2023-06-02 13:49:39 +02:00
shreyas-goenka 91097856b5
Add check for path is a directory in filer.ReadDir (#426)
## Tests
Integration tests
2023-06-02 12:28:35 +02:00
Pieter Noordhuis 2b56af6016
Add Stat function to filer.Filer interface (#421)
## Changes

TSIA

## Tests

New integration test passes.
2023-06-01 20:23:22 +02:00
Pieter Noordhuis 7a4ca786d8
Use cmdio in version command for `--output` flag (#419)
## Changes

Use cmdio in the version command such that it accepts the `--output` flag.

This removes the existing `--detail` flag which previously made the
command print JSON output.

## Tests

New integration test passes.
2023-06-01 12:03:22 +02:00
Serge Smertin 24ebfdf31e
Add readable console logger (#370)
## Changes

Add a readable colored console logger that is active only for TTYs:

<img width="764" alt="image"
src="https://user-images.githubusercontent.com/259697/235221427-ca482b32-9f88-4adb-ada3-8c4f35f50f06.png">

## Tests

Run `go run main.go clusters list --log-level debug --profile demo`
2023-06-01 11:37:33 +02:00
dependabot[bot] d44d8ff2dc
Bump github.com/mattn/go-isatty from 0.0.18 to 0.0.19 (#412) 2023-06-01 09:45:54 +02:00
Pieter Noordhuis 9ae86e3ae3
Fix locker integration test (#417)
## Changes

The failure was caused by swapping out the error types returned by the filer in #139.

## Tests

Integration tests pass again.
2023-06-01 09:38:03 +02:00
Pieter Noordhuis 349e2aff40
Allow equivalence checking of filer errors to fs errors (#416)
## Changes

The pattern `errors.Is(err, fs.ErrNotExist)` is common to check for an
error type.

Errors can implement `Is(error) bool` with a custom equivalence checker.

## Tests

New asserts all pass in the integration test.
2023-05-31 20:47:00 +02:00
Pieter Noordhuis 42cd8daee0
Make filer.Filer return fs.DirEntry from ReadDir (#415)
## Changes

This allows for compatibility with the stdlib functions in io/fs.

## Tests

Integration tests pass.
2023-05-31 14:22:26 +02:00
Pieter Noordhuis 27df4e765c
Implement DBFS filer (#139)
Adds a DBFS implementation of the `filer.Filer` interface.

The integration tests are reused between the workspace filesystem and
DBFS implementations to ensure identical behavior.
2023-05-31 13:24:20 +02:00
Pieter Noordhuis 92cb52041d
Add Mkdir and ReadDir functions to filer.Filer interface (#414)
## Changes

This cherry-picks the filer changes from #408.

## Tests

Manually ran integration tests.
2023-05-31 11:11:17 +02:00
Andrew Nester 05eaf7ff50
Added secrets input prompt for secrets put-secret command (#413)
## Changes
Added secrets input prompt for secrets put-secrets command

## Tests

<img width="623" alt="Screenshot 2023-05-30 at 12 06 24"
src="https://github.com/databricks/cli/assets/2969996/9338e6ba-c504-48cc-ac97-cac97dde7a3a">
2023-05-31 10:18:29 +02:00
Andrew Nester aed6450baf
Do not prompt for List methods (#411)
## Changes
Do not prompt for List methods

## Tests
Running
```
cli workspace list
```

Before

```
cli workspace list
Error: Path () doesn't start with '/'
```

After

```
cli workspace list
Error: accepts 1 arg(s), received 0

```
2023-05-26 16:02:53 +02:00
Andrew Nester 3c4d6f637f
Changed service template to correctly handle required positional arguments (#405) 2023-05-26 14:46:08 +02:00
Pieter Noordhuis e366faf4a6
Release v0.100.2 (#410)
## Changes

CLI:
* Reduce parallellism in locker integration test
([#407](https://github.com/databricks/bricks/pull/407)).

Bundles:
* Don't pass synthesized TMPDIR if not already set
([#409](https://github.com/databricks/bricks/pull/409)).
* Added support for bundle.Seq, simplified Mutator.Apply interface
([#403](https://github.com/databricks/bricks/pull/403)).
* Regenerated internal schema structs based on Terraform provider
schemas ([#401](https://github.com/databricks/bricks/pull/401)).
2023-05-26 13:28:38 +02:00
Pieter Noordhuis e4ab455ea1
Don't pass synthesized TMPDIR if not already set (#409)
## Changes

On Unix systems, the default of `/tmp` always works. No need to
synthesize a path for it.

The custom TMPDIR was causing issues when used from GitHub Actions
runners.

## Tests

Confirmed manually this fixes the issue on GitHub Actions runners.
2023-05-26 13:05:30 +02:00
Pieter Noordhuis 854444077f
Reduce parallellism in locker integration test (#407)
## Changes

This test is hitting request throttling. A lower number of parallel
routines acquiring it should be sufficient.

## Tests

n/a
2023-05-25 15:22:51 +02:00
Andrew Nester 6141476ca2
Added support for bundle.Seq, simplified Mutator.Apply interface (#403)
## Changes
Added support for `bundle.Seq`, simplified `Mutator.Apply` interface by
removing list of mutators from return values/

## Tests
1. Ran `cli bundle deploy` and interrupted it with Cmd + C mid execution
so lock is not released
2. Ran `cli bundle deploy` top make sure that CLI is not trying to
release lock when it fail to acquire it
```
andrew.nester@HFW9Y94129 multiples-tasks % cli bundle deploy
Starting upload of bundle files
Uploaded bundle files at /Users/andrew.nester@databricks.com/.bundle/simple-task/development/files!

^C
andrew.nester@HFW9Y94129 multiples-tasks % cli bundle deploy
Error: deploy lock acquired by andrew.nester@databricks.com at 2023-05-24 12:10:23.050343 +0200 CEST. Use --force to override
```
2023-05-24 14:45:19 +02:00
Andrew Nester 273271bc59
Regenerated internal schema structs based on Terraform provider schemas (#401)
## Changes
Regenerated internal schema structs based on Terraform provider schemas

Allows to use `serverless` flag in bundle config.

## Tests
Ran `cli bundle deploy` with bundle which contains pipeline with
serverless key true
2023-05-23 19:33:24 +02:00