50f62692ce
## Changes Include a materialized copy of built-in templates as reference output. This updates the output comparison logic to work against an output directory. The `doComparison` function now always works on real files. It can now tell apart non-existing files and empty files (e.g., the `.gitkeep` files in templates). |
||
---|---|---|
.. | ||
.vscode | ||
fixtures | ||
resources | ||
scratch | ||
src | ||
tests | ||
.gitignore | ||
README.md | ||
databricks.yml | ||
pytest.ini | ||
requirements-dev.txt | ||
setup.py |
README.md
my_default_python
The 'my_default_python' project was generated by using the default-python template.
Getting started
-
Install the Databricks CLI from https://docs.databricks.com/dev-tools/cli/databricks-cli.html
-
Authenticate to your Databricks workspace, if you have not done so already:
$ databricks configure
-
To deploy a development copy of this project, type:
$ databricks bundle deploy --target dev
(Note that "dev" is the default target, so the
--target
parameter is optional here.)This deploys everything that's defined for this project. For example, the default template would deploy a job called
[dev yourname] my_default_python_job
to your workspace. You can find that job by opening your workpace and clicking on Workflows. -
Similarly, to deploy a production copy, type:
$ databricks bundle deploy --target prod
Note that the default job from the template has a schedule that runs every day (defined in resources/my_default_python.job.yml). The schedule is paused when deploying in development mode (see https://docs.databricks.com/dev-tools/bundles/deployment-modes.html).
-
To run a job or pipeline, use the "run" command:
$ databricks bundle run
-
Optionally, install developer tools such as the Databricks extension for Visual Studio Code from https://docs.databricks.com/dev-tools/vscode-ext.html. Or read the "getting started" documentation for Databricks Connect for instructions on running the included Python code from a different IDE.
-
For documentation on the Databricks asset bundles format used for this project, and for CI/CD configuration, see https://docs.databricks.com/dev-tools/bundles/index.html.