steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
Databricks CLI
281 Commits 85 Branches 104 Tags 56 MiB
Go 99.6%
Python 0.2%
Shell 0.2%
Go to file
shreyas-goenka df0293510e
Fixes for pipeline progress logging (#330) 
## Changes
1. Events are now printed in chronological order
2. Simplify events rendering by removing update/flow name. This makes it
more consistent with the web UI too
3. Switch to server side filtering on update_id

## Tests
Manually

Happy run:
```
shreyas.goenka@THW32HFW6T pipeline-progress % bricks bundle run foo
2023-04-12T20:00:22.879Z update_progress INFO "Update e1becc is INITIALIZING."
2023-04-12T20:00:22.906Z update_progress INFO "Update e1becc is SETTING_UP_TABLES."
2023-04-12T20:00:24.496Z update_progress INFO "Update e1becc is RUNNING."
2023-04-12T20:00:24.497Z flow_progress   INFO "Flow 'sales_orders_raw' is QUEUED."
2023-04-12T20:00:24.586Z flow_progress   INFO "Flow 'sales_orders_raw' is STARTING."
2023-04-12T20:00:24.748Z flow_progress   INFO "Flow 'sales_orders_raw' is RUNNING."
2023-04-12T20:00:26.672Z flow_progress   INFO "Flow 'sales_orders_raw' has COMPLETED."
2023-04-12T20:00:27.753Z update_progress INFO "Update e1becc is COMPLETED."
```

Sad run:
```
shreyas.goenka@THW32HFW6T pipeline-progress % bricks bundle run foo
2023-04-12T20:02:07.764Z update_progress INFO "Update 04b80e is INITIALIZING."
2023-04-12T20:02:07.870Z update_progress ERROR "Update 04b80e is FAILED."
Error: update failed
```
 		2023-04-14 12:21:44 +02:00
.github Publish snapshot binaries to snapshot release (#329) 2023-04-12 22:16:30 +02:00
.vscode Let vscode trim whitespace on save (#172) 2023-01-23 09:52:50 +01:00
bundle Fixes for pipeline progress logging (#330) 2023-04-14 12:21:44 +02:00
cmd Log os.Args, bricks version, and exit status (#324) 2023-04-12 22:12:36 +02:00
docs Add structured logging infrastructure (#246) 2023-03-16 14:46:53 +01:00
folders Don't depend on working directory in folders.FindDirWithLeaf (#54) 2022-09-14 15:08:55 +02:00
internal Acquire lock prior to deploy (#270) 2023-03-22 16:37:26 +01:00
libs Do not fail snapshot destroy if snapshot does not exist (#328) 2023-04-12 21:37:01 +02:00
python Remove package project (#321) 2023-04-11 16:59:27 +02:00
sandbox updated dependencies 2022-05-14 19:56:09 +02:00
.gitignore Implement Terraform state synchronization and deploy (#98) 2022-12-06 00:40:45 +01:00
.goreleaser.yaml Publish snapshot binaries to snapshot release (#329) 2023-04-12 22:16:30 +02:00
Makefile updated dependencies 2022-05-14 19:56:09 +02:00
README.md Update README.md 2022-05-16 13:31:47 +02:00
go.mod Remove unused retries package (#326) 2023-04-12 13:51:33 +02:00
go.sum Remove unused retries package (#326) 2023-04-12 13:51:33 +02:00
main.go Remove package project (#321) 2023-04-11 16:59:27 +02:00
main_test.go Configure user agent in root command (#195) 2023-02-03 16:47:33 +01:00

README.md

Bricks CLI 🧱 build

Where's "data"? Secured by the unity catalog. Projects build lifecycle is secured by bricks 🧱

This is an early PoC at this stage. make build (or download the latest from releases page).

Reuses authentication from Databricks CLI. And terraform provider. See details here: https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs#environment-variables

Supports:

  • Databricks CLI
  • Databricks CLI Profiles
  • Azure CLI Auth
  • Azure MSI Auth
  • Azure SPN Auth
  • Google OIDC Auth
  • Direct DATABRICKS_HOST, DATABRICKS_TOKEN or DATABRICKS_USERNAME + DATABRICKS_PASSWORD variables.

What works:

  • ./bricks fs ls /
  • ./bricks test
  • ./bricks launch test.py

What doesn't work:

  • Everything else.

This project reuses some code from Databricks Terraform Provider