Databricks CLI
Go to file
shreyas-goenka 5140a9a902
Add docker images for the CLI (#1353)
## Changes
This PR makes changes to support creating a docker image for the CLI
with the `terraform` dependencies built in. This is useful for customers
that operate in a network-restricted environment. Normally DABs makes
API calls to registry.terraform.io to setup the terraform dependencies,
with this setup the CLI/DABs will rely on the provider binaries bundled
in the docker image.

### Specifically this PR makes the following changes:
----------------
Modifies the CLI release workflow to publish the docker images in the
Github Container Registry. URL:
https://github.com/databricks/cli/pkgs/container/cli.

We use docker support in `goreleaser` to build and publish the images.
Using goreleaser ensures the CLI packaged in the docker image is the
same release artifact as the normal releases. For more information see:
1. https://goreleaser.com/cookbooks/multi-platform-docker-images
2. https://goreleaser.com/customization/docker/

Other choices made include:
1. Using `alpine` as the base image. The reason is `alpine` is a small
and lightweight linux distribution (~5MB) and an industry standard.
2. Not using [docker
manifest](https://docs.docker.com/reference/cli/docker/manifest) to
create a multi-arch build. This is because the functionality is still
experimental.

------------------

Make the `DATABRICKS_TF_VERSION` and `DATABRICKS_TF_PROVIDER_VERSION`
environment variables optional for using the terraform file mirror.
While it's not strictly necessary to make the docker image work, it's
the "right" behaviour and reduces complexity. The rationale is:
- These environment variables here are needed so the Databricks CLI does
not accidentally use the file mirror bundled with VSCode if it's
incompatible. This does not require the env vars to be mandatory.
context: https://github.com/databricks/cli/pull/1294
- This makes the `Dockerfile` and `setup.sh` simpler. We don't need an
[entrypoint.sh script to set the version environment
variables](https://medium.com/@leonardo5621_66451/learn-how-to-use-entrypoint-scripts-in-docker-images-fede010f172d).
This also makes using an interactive terminal with `docker run -it ...`
work out of the box.

 
## Tests


Tested manually. 

--------------------

To test the release pipeline I triggered a couple of dummy releases and
verified that the images are built successfully and uploaded to Github.
1. https://github.com/databricks/cli/pkgs/container/cli
3. workflow for release:
https://github.com/databricks/cli/actions/runs/8646106333

--------------------

I tested the docker container itself by setting up
[Charles](https://www.charlesproxy.com/) as an HTTP proxy and verifying
that no HTTP requests are made to `registry.terraform.io`

Before:
FYI, The Charles web proxy is hosted at localhost:8888.
```
shreyas.goenka@THW32HFW6T bundle-playground % rm -r .databricks 
shreyas.goenka@THW32HFW6T bundle-playground %  HTTP_PROXY="http://localhost:8888" HTTPS_PROXY="http://localhost:8888" cli bundle deploy
Uploading bundle files to /Users/shreyas.goenka@databricks.com/.bundle/bundle-playground/default/files...
Deploying resources...
Updating deployment state...
Deployment complete!
```
<img width="1275" alt="Screenshot 2024-04-11 at 3 21 45 PM"
src="https://github.com/databricks/cli/assets/88374338/15f37324-afbd-47c0-a40e-330ab232656b">

After:
This time bundle deploy is run from inside the docker container. We use
`host.docker.internal` to map to localhost on the host machine, and -v
to mount the host file system as a volume.
```
shreyas.goenka@THW32HFW6T bundle-playground % docker run -v ~/projects/bundle-playground:/bundle -v ~/.databrickscfg:/root/.databrickscfg  -it --entrypoint /bin/sh -e HTTP_PROXY="http://host.docker.internal:8888" -e HTTPS_PROXY="http://host.docker.internal:8888" --network host ghcr.io/databricks/cli:latest-arm64           
/ # cd /bundle/
/bundle # rm -r .databricks/
/bundle # databricks bundle deploy
Uploading bundle files to /Users/shreyas.goenka@databricks.com/.bundle/bundle-playground/default/files...
Deploying resources...
Updating deployment state...
Deployment complete!
```

<img width="1275" alt="Screenshot 2024-04-11 at 3 22 54 PM"
src="https://github.com/databricks/cli/assets/88374338/2a8f097e-734b-4b3e-8075-c02e98a1b275">
2024-04-12 15:22:30 +00:00
.codegen Bump github.com/databricks/databricks-sdk-go from 0.36.0 to 0.37.0 (#1326) 2024-04-03 10:39:53 +00:00
.github Add docker images for the CLI (#1353) 2024-04-12 15:22:30 +00:00
.vscode Cleanup after previous PR comments (#724) 2023-09-04 07:07:17 +00:00
bundle Add docker images for the CLI (#1353) 2024-04-12 15:22:30 +00:00
cmd Fixed typo in error template for auth describe (#1341) 2024-04-08 11:19:13 +00:00
docker Add docker images for the CLI (#1353) 2024-04-12 15:22:30 +00:00
docs Rename bricks -> databricks (#389) 2023-05-16 18:35:39 +02:00
internal Fixed typo in error template for auth describe (#1341) 2024-04-08 11:19:13 +00:00
libs Do not emit warning on YAML anchor blocks (#1354) 2024-04-10 09:55:02 +00:00
.codegen.json Allow referencing bundle resources by name (#872) 2024-01-04 21:04:42 +00:00
.gitattributes Update Go SDK to v0.34.0 (#1256) 2024-03-06 09:53:44 +00:00
.gitignore Add a foundation for built-in templates (#685) 2023-08-25 09:03:42 +00:00
.goreleaser.yaml Add docker images for the CLI (#1353) 2024-04-12 15:22:30 +00:00
.mockery.yaml Use mockery to generate mocks compatible with testify/mock (#1190) 2024-02-08 15:18:53 +00:00
CHANGELOG.md Release v0.217.1 (#1355) 2024-04-10 11:17:21 +00:00
Dockerfile Add docker images for the CLI (#1353) 2024-04-12 15:22:30 +00:00
LICENSE Trim trailing whitespace (#1206) 2024-02-15 16:23:48 +00:00
Makefile `make snapshot` to build file in `.databricks/databricks` (#927) 2023-10-27 10:50:41 +00:00
NOTICE Allow specifying CLI version constraints required to run the bundle (#1320) 2024-04-02 12:55:21 +00:00
README.md Fix typo in README.md (#510) 2023-06-21 20:26:09 +02:00
go.mod Bump golang.org/x/sync from 0.6.0 to 0.7.0 (#1346) 2024-04-08 15:11:59 +00:00
go.sum Bump golang.org/x/sync from 0.6.0 to 0.7.0 (#1346) 2024-04-08 15:11:59 +00:00
main.go Consolidate environment variable interaction (#747) 2023-09-11 08:18:43 +00:00
main_test.go Consolidate environment variable interaction (#747) 2023-09-11 08:18:43 +00:00

README.md

Databricks CLI

build

This project is in Public Preview.

Documentation about the full REST API coverage is available in the docs folder.

Documentation is available at https://docs.databricks.com/dev-tools/cli/databricks-cli.html.

Installation

This CLI is packaged as a dependency-free binary executable and may be located in any directory. See https://github.com/databricks/cli/releases for releases and the docs pages for installation instructions.

Authentication

This CLI follows the Databricks Unified Authentication principles.

You can find a detailed description at https://github.com/databricks/databricks-sdk-go#authentication.