steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
Databricks CLI
236 Commits 100 Branches 106 Tags 54 MiB
Go 99.6%
Python 0.2%
Shell 0.1%
Go to file
shreyas-goenka 75d516939b
Error out if notebook file does not exist locally (#261) 
Adds check for whether file exists locally

case 1: local (relative) file does not exist
```
    foo:
      name: "[job-output] test-job by shreyas"

      tasks:
        - task_key: my_notebook_task
          existing_cluster_id: ***
          notebook_task:
            notebook_path: "./doesnotexist"
```
output:
```
shreyas.goenka@THW32HFW6T job-output % bricks bundle deploy
Error: notebook ./doesnotexist not found. Error: open /Users/shreyas.goenka/projects/job-output/doesnotexist: no such file or directory
```


case 2: remote (absolute) file does not exist
```
    foo:
      name: "[job-output] test-job by shreyas"

      tasks:
        - task_key: my_notebook_task
          existing_cluster_id: ***
          notebook_task:
            notebook_path: "/Users/shreyas.goenka@databricks.com/doesnotexist"
```

output:
```
shreyas.goenka@THW32HFW6T job-output % bricks bundle deploy
shreyas.goenka@THW32HFW6T job-output % bricks bundle run foo
Error: failed to reach TERMINATED or SKIPPED, got INTERNAL_ERROR: Task my_notebook_task failed with message: Notebook not found: /Users/shreyas.goenka@databricks.com/doesnotexist. This caused all downstream tasks to get skipped.
```

case 3: remote exists
Successful deploy and run
 		2023-03-21 18:13:16 +01:00
.github Bump Go version to 1.19 (#197) 2023-02-03 15:13:29 +01:00
.vscode Let vscode trim whitespace on save (#172) 2023-01-23 09:52:50 +01:00
bundle Error out if notebook file does not exist locally (#261) 2023-03-21 18:13:16 +01:00
cmd Allow logger defaults to be configured through environment variables (#266) 2023-03-21 17:05:04 +01:00
docs Add structured logging infrastructure (#246) 2023-03-16 14:46:53 +01:00
experimental/github Make tests pass (#40) 2022-09-07 20:08:42 +02:00
folders Don't depend on working directory in folders.FindDirWithLeaf (#54) 2022-09-14 15:08:55 +02:00
internal Path escape filepaths in the URL (#250) 2023-03-15 17:25:57 +01:00
libs Error out if notebook file does not exist locally (#261) 2023-03-21 18:13:16 +01:00
project Move git package to libs/git (#189) 2023-01-31 19:19:16 +01:00
python Use new logger throughout codebase (#256) 2023-03-17 15:17:31 +01:00
retries Use new logger throughout codebase (#256) 2023-03-17 15:17:31 +01:00
sandbox updated dependencies 2022-05-14 19:56:09 +02:00
terraform Use new logger throughout codebase (#256) 2023-03-17 15:17:31 +01:00
.gitignore Implement Terraform state synchronization and deploy (#98) 2022-12-06 00:40:45 +01:00
.goreleaser.yaml Include commit hash in snapshot version (#193) 2023-02-20 15:46:57 +01:00
Makefile updated dependencies 2022-05-14 19:56:09 +02:00
README.md Update README.md 2022-05-16 13:31:47 +02:00
go.mod Add structured logging infrastructure (#246) 2023-03-16 14:46:53 +01:00
go.sum Add structured logging infrastructure (#246) 2023-03-16 14:46:53 +01:00
main.go Move and hide launch and test commands (#222) 2023-03-09 10:26:56 +01:00
main_test.go Configure user agent in root command (#195) 2023-02-03 16:47:33 +01:00

README.md

Bricks CLI 🧱 build

Where's "data"? Secured by the unity catalog. Projects build lifecycle is secured by bricks 🧱

This is an early PoC at this stage. make build (or download the latest from releases page).

Reuses authentication from Databricks CLI. And terraform provider. See details here: https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs#environment-variables

Supports:

  • Databricks CLI
  • Databricks CLI Profiles
  • Azure CLI Auth
  • Azure MSI Auth
  • Azure SPN Auth
  • Google OIDC Auth
  • Direct DATABRICKS_HOST, DATABRICKS_TOKEN or DATABRICKS_USERNAME + DATABRICKS_PASSWORD variables.

What works:

  • ./bricks fs ls /
  • ./bricks test
  • ./bricks launch test.py

What doesn't work:

  • Everything else.

This project reuses some code from Databricks Terraform Provider