steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/acceptance/bundle/integration_whl/wrapper/output.txt

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

71 lines
1.8 KiB
Plaintext
Raw Normal View History

acc: Fix interactive_cluster test to use correct config; add NODE_TYPE_ID (#2478) This is a follow up to #2471 where incorrect config was used in interactive_cluster test. ## Changes - Fixed interactive_cluster to use proper config, it was accidentally referring to config from ../base - Add $NODE_TYPE_ID env var and replacement to acceptance tests, this is necessary for interactive_cluster test. - Disable acceptance/bundle/override on cloud. This started failing because it has real node type that gets replaced with NODE_TYPE_ID but only in AWS env. Since the test is focussed on config merging, there is no need to run it against real workspaces. - Modify all tests in integration_whl to print rendered databricks.yml, to prevent this kind of error.
2025-03-12 15:49:24 +00:00
>>> cat input.json
Convert python_wheel_test.go to acceptance test (#2471) Convert integration/bundle/integration/bundle/python_wheel_test.go to acceptance tests. I plan to expand these tests to check patchwheel functionality. Inside each test there were two runs - with params and without, I've expanded each run into separate test to reduce total time as this runs can be done in parallel. Also add new env var DEFAULT_SPARK_VERSION that matches the one in integration tests. The tests are currently enabled on every PR (`CloudLong=true` is commented out), this can be changed after landing.
2025-03-12 10:51:31 +00:00
{
"project_name": "my_test_code",
"spark_version": "12.2.x-scala2.12",
acc: Fix interactive_cluster test to use correct config; add NODE_TYPE_ID (#2478) This is a follow up to #2471 where incorrect config was used in interactive_cluster test. ## Changes - Fixed interactive_cluster to use proper config, it was accidentally referring to config from ../base - Add $NODE_TYPE_ID env var and replacement to acceptance tests, this is necessary for interactive_cluster test. - Disable acceptance/bundle/override on cloud. This started failing because it has real node type that gets replaced with NODE_TYPE_ID but only in AWS env. Since the test is focussed on config merging, there is no need to run it against real workspaces. - Modify all tests in integration_whl to print rendered databricks.yml, to prevent this kind of error.
2025-03-12 15:49:24 +00:00
"node_type_id": "[NODE_TYPE_ID]",
Convert python_wheel_test.go to acceptance test (#2471) Convert integration/bundle/integration/bundle/python_wheel_test.go to acceptance tests. I plan to expand these tests to check patchwheel functionality. Inside each test there were two runs - with params and without, I've expanded each run into separate test to reduce total time as this runs can be done in parallel. Also add new env var DEFAULT_SPARK_VERSION that matches the one in integration tests. The tests are currently enabled on every PR (`CloudLong=true` is commented out), this can be changed after landing.
2025-03-12 10:51:31 +00:00
"unique_id": "[UNIQUE_NAME]",
"python_wheel_wrapper": true,
"instance_pool_id": "[TEST_INSTANCE_POOL_ID]"
}
✨ Successfully initialized template
acc: Fix interactive_cluster test to use correct config; add NODE_TYPE_ID (#2478) This is a follow up to #2471 where incorrect config was used in interactive_cluster test. ## Changes - Fixed interactive_cluster to use proper config, it was accidentally referring to config from ../base - Add $NODE_TYPE_ID env var and replacement to acceptance tests, this is necessary for interactive_cluster test. - Disable acceptance/bundle/override on cloud. This started failing because it has real node type that gets replaced with NODE_TYPE_ID but only in AWS env. Since the test is focussed on config merging, there is no need to run it against real workspaces. - Modify all tests in integration_whl to print rendered databricks.yml, to prevent this kind of error.
2025-03-12 15:49:24 +00:00
>>> cat databricks.yml
bundle:
name: wheel-task
workspace:
root_path: "~/.bundle/[UNIQUE_NAME]"
experimental:
python_wheel_wrapper: true
resources:
jobs:
some_other_job:
name: "[${bundle.target}] Test Wheel Job [UNIQUE_NAME]"
tasks:
- task_key: TestTask
new_cluster:
num_workers: 1
spark_version: "12.2.x-scala2.12"
node_type_id: "[NODE_TYPE_ID]"
data_security_mode: USER_ISOLATION
instance_pool_id: "[TEST_INSTANCE_POOL_ID]"
python_wheel_task:
package_name: my_test_code
entry_point: run
parameters:
- "one"
- "two"
libraries:
- whl: ./dist/*.whl
Convert python_wheel_test.go to acceptance test (#2471) Convert integration/bundle/integration/bundle/python_wheel_test.go to acceptance tests. I plan to expand these tests to check patchwheel functionality. Inside each test there were two runs - with params and without, I've expanded each run into separate test to reduce total time as this runs can be done in parallel. Also add new env var DEFAULT_SPARK_VERSION that matches the one in integration tests. The tests are currently enabled on every PR (`CloudLong=true` is commented out), this can be changed after landing.
2025-03-12 10:51:31 +00:00
>>> [CLI] bundle deploy
Building python_artifact...
Uploading my_test_code-0.0.1-py3-none-any.whl...
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/files...
Deploying resources...
Updating deployment state...
Deployment complete!
>>> [CLI] bundle run some_other_job
Run URL: [DATABRICKS_URL]/?o=[NUMID]#job/[NUMID]/run/[NUMID]
[TIMESTAMP] "[default] Test Wheel Job [UNIQUE_NAME]" RUNNING
[TIMESTAMP] "[default] Test Wheel Job [UNIQUE_NAME]" TERMINATED SUCCESS
Hello from my func
Got arguments:
['my_test_code', 'one', 'two']
>>> [CLI] bundle destroy --auto-approve
The following resources will be deleted:
delete job some_other_job
All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]
Deleting files...
Destroy complete!