steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
databricks-cli/acceptance/bundle/integration_whl/wrapper/script

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

13 lines
745 B
Plaintext
Raw Normal View History

Convert python_wheel_test.go to acceptance test (#2471) Convert integration/bundle/integration/bundle/python_wheel_test.go to acceptance tests. I plan to expand these tests to check patchwheel functionality. Inside each test there were two runs - with params and without, I've expanded each run into separate test to reduce total time as this runs can be done in parallel. Also add new env var DEFAULT_SPARK_VERSION that matches the one in integration tests. The tests are currently enabled on every PR (`CloudLong=true` is commented out), this can be changed after landing.
2025-03-12 10:51:31 +00:00
# Installing wheels from Workspace file system is only supported starting from DBR 13.1+.
# But before users used older DBRs and python wheel tasks but installed it from DBFS.
# We still want to support older DBRs and did the trampoline workaround (https://github.com/databricks/cli/pull/635)
# Hence this is to test that python wheel tasks in DABs are working for older DBRs
acc: Simplify integration_whl tests (#2479) ## Changes Instead of full templates, only template databricks.yml and use envsubst for that.
2025-03-14 08:52:20 +00:00
export DEFAULT_SPARK_VERSION=12.2.x-scala2.12
export EXTRA_CONFIG=python_wheel_wrapper.yml
envsubst < $TESTDIR/../base/databricks.yml.tmpl > databricks.yml
cp -r $TESTDIR/../base/{$EXTRA_CONFIG,setup.py,my_test_code} .
acc: Fix interactive_cluster test to use correct config; add NODE_TYPE_ID (#2478) This is a follow up to #2471 where incorrect config was used in interactive_cluster test. ## Changes - Fixed interactive_cluster to use proper config, it was accidentally referring to config from ../base - Add $NODE_TYPE_ID env var and replacement to acceptance tests, this is necessary for interactive_cluster test. - Disable acceptance/bundle/override on cloud. This started failing because it has real node type that gets replaced with NODE_TYPE_ID but only in AWS env. Since the test is focussed on config merging, there is no need to run it against real workspaces. - Modify all tests in integration_whl to print rendered databricks.yml, to prevent this kind of error.
2025-03-12 15:49:24 +00:00
trace cat databricks.yml
Convert python_wheel_test.go to acceptance test (#2471) Convert integration/bundle/integration/bundle/python_wheel_test.go to acceptance tests. I plan to expand these tests to check patchwheel functionality. Inside each test there were two runs - with params and without, I've expanded each run into separate test to reduce total time as this runs can be done in parallel. Also add new env var DEFAULT_SPARK_VERSION that matches the one in integration tests. The tests are currently enabled on every PR (`CloudLong=true` is commented out), this can be changed after landing.
2025-03-12 10:51:31 +00:00
trap "errcode trace '$CLI' bundle destroy --auto-approve" EXIT
trace $CLI bundle deploy
trace $CLI bundle run some_other_job