steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/acceptance/bundle/integration_whl/base/databricks_template_schema....

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

30 lines
850 B
JSON
Raw Normal View History

Added end-to-end test for deploying and running Python wheel task (#741) ## Changes Added end-to-end test for deploying and running Python wheel task ## Tests Test successfully passed on all environments, takes about 9-10 minutes to pass. ``` Deleted snapshot file at /var/folders/nt/xjv68qzs45319w4k36dhpylc0000gp/T/TestAccPythonWheelTaskDeployAndRun1845899209/002/.databricks/bundle/default/sync-snapshots/1f7cc766ffe038d6.json Successfully deleted files! 2023/09/06 17:50:50 INFO Releasing deployment lock mutator=destroy mutator=seq mutator=seq mutator=deferred mutator=lock:release --- PASS: TestAccPythonWheelTaskDeployAndRun (508.16s) PASS coverage: 77.9% of statements in ./... ok github.com/databricks/cli/internal/bundle 508.810s coverage: 77.9% of statements in ./... ``` --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2023-09-07 14:08:16 +00:00
{
"properties": {
"project_name": {
"type": "string",
"default": "my_test_code",
"description": "Unique name for this project"
},
"spark_version": {
"type": "string",
"description": "Spark version used for job cluster"
},
"node_type_id": {
"type": "string",
"description": "Node type id for job cluster"
Generate unique name for a job in Python wheel test (#745) ## Changes Generate unique name for a job in Python wheel test
2023-09-07 20:02:26 +00:00
},
"unique_id": {
"type": "string",
"description": "Unique ID for job name"
Make a notebook wrapper for Python wheel tasks optional (#797) ## Changes Instead of always using notebook wrapper for Python wheel tasks, let's make this an opt-in option. Now by default Python wheel tasks will be deployed as is to Databricks platform. If notebook wrapper required (DBR < 13.1 or other configuration differences), users can provide a following experimental setting ``` experimental: python_wheel_wrapper: true ``` Fixes #783, https://github.com/databricks/databricks-asset-bundles-dais2023/issues/8 ## Tests Added unit tests. Integration tests passed for both cases ``` helpers.go:163: [databricks stdout]: Hello from my func helpers.go:163: [databricks stdout]: Got arguments: helpers.go:163: [databricks stdout]: ['my_test_code', 'one', 'two'] ... Bundle remote directory is ***/.bundle/ac05d5e8-ed4b-4e34-b3f2-afa73f62b021 Deleted snapshot file at /var/folders/nt/xjv68qzs45319w4k36dhpylc0000gp/T/TestAccPythonWheelTaskDeployAndRunWithWrapper3733431114/001/.databricks/bundle/default/sync-snapshots/cac1e02f3941a97b.json Successfully deleted files! --- PASS: TestAccPythonWheelTaskDeployAndRunWithWrapper (214.18s) PASS coverage: 93.5% of statements in ./... ok github.com/databricks/cli/internal/bundle 214.495s coverage: 93.5% of statements in ./... ``` ``` helpers.go:163: [databricks stdout]: Hello from my func helpers.go:163: [databricks stdout]: Got arguments: helpers.go:163: [databricks stdout]: ['my_test_code', 'one', 'two'] ... Bundle remote directory is ***/.bundle/0ef67aaf-5960-4049-bf1d-dc9e29157421 Deleted snapshot file at /var/folders/nt/xjv68qzs45319w4k36dhpylc0000gp/T/TestAccPythonWheelTaskDeployAndRunWithoutWrapper2340216760/001/.databricks/bundle/default/sync-snapshots/edf0b322cee93b13.json Successfully deleted files! --- PASS: TestAccPythonWheelTaskDeployAndRunWithoutWrapper (192.36s) PASS coverage: 93.5% of statements in ./... ok github.com/databricks/cli/internal/bundle 195.130s coverage: 93.5% of statements in ./... ```
2023-09-26 14:32:20 +00:00
},
"python_wheel_wrapper": {
"type": "boolean",
"description": "Whether or not to enable python wheel wrapper"
Fix python wheel task integration tests (#1648) ## Changes A new Service Control Policy has removed the `ec2.RunInstances` permission from our service principal for our AWS integration tests. This PR switches over to using the instance pool which does not require creating new clusters. ## Tests The integration tests pass now.
2024-08-01 13:55:22 +00:00
},
"instance_pool_id": {
"type": "string",
"description": "Instance pool id for job cluster"
Added end-to-end test for deploying and running Python wheel task (#741) ## Changes Added end-to-end test for deploying and running Python wheel task ## Tests Test successfully passed on all environments, takes about 9-10 minutes to pass. ``` Deleted snapshot file at /var/folders/nt/xjv68qzs45319w4k36dhpylc0000gp/T/TestAccPythonWheelTaskDeployAndRun1845899209/002/.databricks/bundle/default/sync-snapshots/1f7cc766ffe038d6.json Successfully deleted files! 2023/09/06 17:50:50 INFO Releasing deployment lock mutator=destroy mutator=seq mutator=seq mutator=deferred mutator=lock:release --- PASS: TestAccPythonWheelTaskDeployAndRun (508.16s) PASS coverage: 77.9% of statements in ./... ok github.com/databricks/cli/internal/bundle 508.810s coverage: 77.9% of statements in ./... ``` --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2023-09-07 14:08:16 +00:00
}
}
}