Databricks CLI
Go to file
Renaud Hartert 235973e7b1
[Fix] Do not buffer files in memory when downloading (#1599)
## Changes

This PR fixes a performance bug that led downloaded files (e.g. with
`databricks fs cp dbfs:/Volumes/.../somefile .`) to be buffered in
memory before being written.

Results from profiling the download of a ~100MB file:

Before:
```
Type: alloc_space
Showing nodes accounting for 374.02MB, 98.50% of 379.74MB total
```

After:
```
Type: alloc_space
Showing nodes accounting for 3748.67kB, 100% of 3748.67kB total
```

Note that this fix is temporary. A longer term solution should be to use
the API provided by the Go SDK rather than making an HTTP request
directly from the CLI.

fix #1575 

## Tests

Verified that the CLI properly download the file when doing the
profiling.
2024-07-17 07:14:02 +00:00
.codegen Bump github.com/databricks/databricks-sdk-go from 0.42.0 to 0.43.0 (#1522) 2024-06-25 12:51:17 +00:00
.github Use different Go cache key for goreleaser jobs (#1558) 2024-07-04 11:39:55 +00:00
.vscode Cleanup after previous PR comments (#724) 2023-09-04 07:07:17 +00:00
bundle Track multiple locations associated with a `dyn.Value` (#1510) 2024-07-16 11:27:27 +00:00
cmd Print diagnostics in 'bundle deploy' (#1579) 2024-07-10 11:14:57 +00:00
docker Followup improvements to the Docker setup script (#1369) 2024-04-18 20:52:11 +00:00
docs Removed autogenerated docs for the CLI commands (#1392) 2024-04-23 13:25:25 +00:00
internal Allow artifacts (JARs, wheels) to be uploaded to UC Volumes (#1591) 2024-07-16 08:57:04 +00:00
libs [Fix] Do not buffer files in memory when downloading (#1599) 2024-07-17 07:14:02 +00:00
.codegen.json Allow referencing bundle resources by name (#872) 2024-01-04 21:04:42 +00:00
.gitattributes Update Go SDK to v0.41.0 (#1445) 2024-05-22 07:41:32 +00:00
.gitignore Add a foundation for built-in templates (#685) 2023-08-25 09:03:42 +00:00
.goreleaser.yaml Use latest version of goreleaser action (#1477) 2024-06-05 11:33:51 +00:00
.mockery.yaml Use mockery to generate mocks compatible with testify/mock (#1190) 2024-02-08 15:18:53 +00:00
CHANGELOG.md Release v0.223.2 (#1587) 2024-07-10 12:04:59 +00:00
Dockerfile Followup improvements to the Docker setup script (#1369) 2024-04-18 20:52:11 +00:00
LICENSE Trim trailing whitespace (#1206) 2024-02-15 16:23:48 +00:00
Makefile `make snapshot` to build file in `.databricks/databricks` (#927) 2023-10-27 10:50:41 +00:00
NOTICE Add NOTICE for using Terraform 1.5.5 licensed under MPL 2.0 (#1377) 2024-04-19 11:44:05 +00:00
README.md Add link to documentation for Homebrew installation to README (#1505) 2024-06-19 11:01:04 +00:00
go.mod Bump github.com/databricks/databricks-sdk-go from 0.43.0 to 0.43.2 (#1594) 2024-07-16 11:40:12 +00:00
go.sum Bump github.com/databricks/databricks-sdk-go from 0.43.0 to 0.43.2 (#1594) 2024-07-16 11:40:12 +00:00
main.go Add regression tests for CLI error output (#1566) 2024-07-10 06:38:06 +00:00
main_test.go Consolidate environment variable interaction (#747) 2023-09-11 08:18:43 +00:00

README.md

Databricks CLI

build

This project is in Public Preview.

Documentation is available at https://docs.databricks.com/dev-tools/cli/databricks-cli.html.

Installation

This CLI is packaged as a dependency-free binary executable and may be located in any directory. See https://github.com/databricks/cli/releases for releases and the Databricks documentation for detailed information about installing the CLI.


Homebrew

We maintain a Homebrew tap for installing the Databricks CLI. You can find instructions for how to install, upgrade and downgrade the CLI using Homebrew here.


Docker

You can use the CLI via a Docker image by pulling the image from ghcr.io. You can find all available versions at: https://github.com/databricks/cli/pkgs/container/cli.

docker pull ghcr.io/databricks/cli:latest

Example of how to run the CLI using the Docker image. More documentation is available at https://docs.databricks.com/dev-tools/bundles/airgapped-environment.html.

docker run -e DATABRICKS_HOST=$YOUR_HOST_URL -e DATABRICKS_TOKEN=$YOUR_TOKEN ghcr.io/databricks/cli:latest current-user me

Authentication

This CLI follows the Databricks Unified Authentication principles.

You can find a detailed description at https://github.com/databricks/databricks-sdk-go#authentication.