steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/Dockerfile

26 lines
567 B
Docker
Raw Permalink Normal View History

Add docker images for the CLI (#1353) ## Changes This PR makes changes to support creating a docker image for the CLI with the `terraform` dependencies built in. This is useful for customers that operate in a network-restricted environment. Normally DABs makes API calls to registry.terraform.io to setup the terraform dependencies, with this setup the CLI/DABs will rely on the provider binaries bundled in the docker image. ### Specifically this PR makes the following changes: ---------------- Modifies the CLI release workflow to publish the docker images in the Github Container Registry. URL: https://github.com/databricks/cli/pkgs/container/cli. We use docker support in `goreleaser` to build and publish the images. Using goreleaser ensures the CLI packaged in the docker image is the same release artifact as the normal releases. For more information see: 1. https://goreleaser.com/cookbooks/multi-platform-docker-images 2. https://goreleaser.com/customization/docker/ Other choices made include: 1. Using `alpine` as the base image. The reason is `alpine` is a small and lightweight linux distribution (~5MB) and an industry standard. 2. Not using [docker manifest](https://docs.docker.com/reference/cli/docker/manifest) to create a multi-arch build. This is because the functionality is still experimental. ------------------ Make the `DATABRICKS_TF_VERSION` and `DATABRICKS_TF_PROVIDER_VERSION` environment variables optional for using the terraform file mirror. While it's not strictly necessary to make the docker image work, it's the "right" behaviour and reduces complexity. The rationale is: - These environment variables here are needed so the Databricks CLI does not accidentally use the file mirror bundled with VSCode if it's incompatible. This does not require the env vars to be mandatory. context: https://github.com/databricks/cli/pull/1294 - This makes the `Dockerfile` and `setup.sh` simpler. We don't need an [entrypoint.sh script to set the version environment variables](https://medium.com/@leonardo5621_66451/learn-how-to-use-entrypoint-scripts-in-docker-images-fede010f172d). This also makes using an interactive terminal with `docker run -it ...` work out of the box. ## Tests Tested manually. -------------------- To test the release pipeline I triggered a couple of dummy releases and verified that the images are built successfully and uploaded to Github. 1. https://github.com/databricks/cli/pkgs/container/cli 3. workflow for release: https://github.com/databricks/cli/actions/runs/8646106333 -------------------- I tested the docker container itself by setting up [Charles](https://www.charlesproxy.com/) as an HTTP proxy and verifying that no HTTP requests are made to `registry.terraform.io` Before: FYI, The Charles web proxy is hosted at localhost:8888. ``` shreyas.goenka@THW32HFW6T bundle-playground % rm -r .databricks shreyas.goenka@THW32HFW6T bundle-playground % HTTP_PROXY="http://localhost:8888" HTTPS_PROXY="http://localhost:8888" cli bundle deploy Uploading bundle files to /Users/shreyas.goenka@databricks.com/.bundle/bundle-playground/default/files... Deploying resources... Updating deployment state... Deployment complete! ``` <img width="1275" alt="Screenshot 2024-04-11 at 3 21 45 PM" src="https://github.com/databricks/cli/assets/88374338/15f37324-afbd-47c0-a40e-330ab232656b"> After: This time bundle deploy is run from inside the docker container. We use `host.docker.internal` to map to localhost on the host machine, and -v to mount the host file system as a volume. ``` shreyas.goenka@THW32HFW6T bundle-playground % docker run -v ~/projects/bundle-playground:/bundle -v ~/.databrickscfg:/root/.databrickscfg -it --entrypoint /bin/sh -e HTTP_PROXY="http://host.docker.internal:8888" -e HTTPS_PROXY="http://host.docker.internal:8888" --network host ghcr.io/databricks/cli:latest-arm64 / # cd /bundle/ /bundle # rm -r .databricks/ /bundle # databricks bundle deploy Uploading bundle files to /Users/shreyas.goenka@databricks.com/.bundle/bundle-playground/default/files... Deploying resources... Updating deployment state... Deployment complete! ``` <img width="1275" alt="Screenshot 2024-04-11 at 3 22 54 PM" src="https://github.com/databricks/cli/assets/88374338/2a8f097e-734b-4b3e-8075-c02e98a1b275">
2024-04-12 15:22:30 +00:00
FROM alpine:3.19 as builder
RUN ["apk", "add", "jq"]
Followup improvements to the Docker setup script (#1369) ## Changes This PR: 1. Uses bash to run the setup.sh script instead of the native busybox sh shipped with alpine. 2. Verifies the checksums of the installed terraform CLI binaries. ## Tests Manually. The docker image successfully builds. --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2024-04-18 20:52:11 +00:00
RUN ["apk", "add", "bash"]
Add docker images for the CLI (#1353) ## Changes This PR makes changes to support creating a docker image for the CLI with the `terraform` dependencies built in. This is useful for customers that operate in a network-restricted environment. Normally DABs makes API calls to registry.terraform.io to setup the terraform dependencies, with this setup the CLI/DABs will rely on the provider binaries bundled in the docker image. ### Specifically this PR makes the following changes: ---------------- Modifies the CLI release workflow to publish the docker images in the Github Container Registry. URL: https://github.com/databricks/cli/pkgs/container/cli. We use docker support in `goreleaser` to build and publish the images. Using goreleaser ensures the CLI packaged in the docker image is the same release artifact as the normal releases. For more information see: 1. https://goreleaser.com/cookbooks/multi-platform-docker-images 2. https://goreleaser.com/customization/docker/ Other choices made include: 1. Using `alpine` as the base image. The reason is `alpine` is a small and lightweight linux distribution (~5MB) and an industry standard. 2. Not using [docker manifest](https://docs.docker.com/reference/cli/docker/manifest) to create a multi-arch build. This is because the functionality is still experimental. ------------------ Make the `DATABRICKS_TF_VERSION` and `DATABRICKS_TF_PROVIDER_VERSION` environment variables optional for using the terraform file mirror. While it's not strictly necessary to make the docker image work, it's the "right" behaviour and reduces complexity. The rationale is: - These environment variables here are needed so the Databricks CLI does not accidentally use the file mirror bundled with VSCode if it's incompatible. This does not require the env vars to be mandatory. context: https://github.com/databricks/cli/pull/1294 - This makes the `Dockerfile` and `setup.sh` simpler. We don't need an [entrypoint.sh script to set the version environment variables](https://medium.com/@leonardo5621_66451/learn-how-to-use-entrypoint-scripts-in-docker-images-fede010f172d). This also makes using an interactive terminal with `docker run -it ...` work out of the box. ## Tests Tested manually. -------------------- To test the release pipeline I triggered a couple of dummy releases and verified that the images are built successfully and uploaded to Github. 1. https://github.com/databricks/cli/pkgs/container/cli 3. workflow for release: https://github.com/databricks/cli/actions/runs/8646106333 -------------------- I tested the docker container itself by setting up [Charles](https://www.charlesproxy.com/) as an HTTP proxy and verifying that no HTTP requests are made to `registry.terraform.io` Before: FYI, The Charles web proxy is hosted at localhost:8888. ``` shreyas.goenka@THW32HFW6T bundle-playground % rm -r .databricks shreyas.goenka@THW32HFW6T bundle-playground % HTTP_PROXY="http://localhost:8888" HTTPS_PROXY="http://localhost:8888" cli bundle deploy Uploading bundle files to /Users/shreyas.goenka@databricks.com/.bundle/bundle-playground/default/files... Deploying resources... Updating deployment state... Deployment complete! ``` <img width="1275" alt="Screenshot 2024-04-11 at 3 21 45 PM" src="https://github.com/databricks/cli/assets/88374338/15f37324-afbd-47c0-a40e-330ab232656b"> After: This time bundle deploy is run from inside the docker container. We use `host.docker.internal` to map to localhost on the host machine, and -v to mount the host file system as a volume. ``` shreyas.goenka@THW32HFW6T bundle-playground % docker run -v ~/projects/bundle-playground:/bundle -v ~/.databrickscfg:/root/.databrickscfg -it --entrypoint /bin/sh -e HTTP_PROXY="http://host.docker.internal:8888" -e HTTPS_PROXY="http://host.docker.internal:8888" --network host ghcr.io/databricks/cli:latest-arm64 / # cd /bundle/ /bundle # rm -r .databricks/ /bundle # databricks bundle deploy Uploading bundle files to /Users/shreyas.goenka@databricks.com/.bundle/bundle-playground/default/files... Deploying resources... Updating deployment state... Deployment complete! ``` <img width="1275" alt="Screenshot 2024-04-11 at 3 22 54 PM" src="https://github.com/databricks/cli/assets/88374338/2a8f097e-734b-4b3e-8075-c02e98a1b275">
2024-04-12 15:22:30 +00:00
WORKDIR /build
COPY ./docker/setup.sh /build/docker/setup.sh
COPY ./databricks /app/databricks
COPY ./docker/config.tfrc /app/config/config.tfrc
ARG ARCH
RUN /build/docker/setup.sh
# Start from a fresh base image, to remove any build artifacts and scripts.
FROM alpine:3.19
ENV DATABRICKS_TF_EXEC_PATH "/app/bin/terraform"
ENV DATABRICKS_TF_CLI_CONFIG_FILE "/app/config/config.tfrc"
ENV PATH="/app:${PATH}"
COPY --from=builder /app /app
ENTRYPOINT ["/app/databricks"]
CMD ["-h"]