mirror of https://github.com/databricks/cli.git
fa89db57e9
## Changes
Previously local JAR paths were transformed to remote path during
initialisation and thus artifact building logic did not recognise such
libraries as local to be handled and uploaded.
Now it's possible to use spark_jar_tasks with local JAR libraries on
14.1+ DBR clusters
Example configuration
```
bundle:
name: spark-jar
workspace:
host: ***
artifacts:
my_java_code:
path: ./sample-java
build: "javac PrintArgs.java && jar cvfm PrintArgs.jar META-INF/MANIFEST.MF PrintArgs.class"
files:
- source: "/Users/andrew.nester/dabs/wheel/sample-java/PrintArgs.jar"
resources:
jobs:
print_args:
name: "Print Args"
tasks:
- task_key: Print
new_cluster:
num_workers: 0
spark_version: 14.2.x-scala2.12
node_type_id: i3.xlarge
spark_conf:
"spark.databricks.cluster.profile": "singleNode"
"spark.master": "local[*]"
custom_tags:
ResourceClass: "SingleNode"
spark_jar_task:
main_class_name: PrintArgs
libraries:
- jar: ./sample-java/PrintArgs.jar
```
## Tests
Manually running `bundle deploy and bundle run`
|
||
|---|---|---|
| .. | ||
| artifacts | ||
| config | ||
| deploy | ||
| deployer | ||
| env | ||
| internal/tf | ||
| libraries | ||
| metadata | ||
| permissions | ||
| phases | ||
| python | ||
| run | ||
| schema | ||
| scripts | ||
| tests | ||
| bundle.go | ||
| bundle_test.go | ||
| context.go | ||
| context_test.go | ||
| deferred.go | ||
| deferred_test.go | ||
| mutator.go | ||
| mutator_test.go | ||
| root.go | ||
| root_test.go | ||
| seq.go | ||
| seq_test.go | ||