Commit Graph

7 Commits

Author SHA1 Message Date
Andrew Nester e64463ba47
Fixed marking libraries from DBFS as remote (#750)
## Changes
Fixed marking libraries from DBFS as remote

## Tests
Updated unit tests to catch the regression
2023-09-08 09:53:57 +00:00
Andrew Nester 5f6289e3a7
Allow referencing local Python wheels without artifacts section defined (#703)
## Changes
Now if the user reference local Python wheel files and do not specify
"artifacts" section, this file will be automatically uploaded by CLI.

Fixes #693 

## Tests
Added unit tests

Ran bundle deploy for this configuration
```
resources:
  jobs:
    some_other_job:
      name: "[${bundle.environment}] My Wheel Job"
      tasks:
        - task_key: TestTask
          existing_cluster_id: ${var.job_existing_cluster}
          python_wheel_task:
            package_name: "my_test_code"
            entry_point: "run"
          libraries:
          - whl: ./dist/*.whl
 ```
 
 Result
 
 ```
andrew.nester@HFW9Y94129 wheel % databricks bundle deploy
artifacts.whl.AutoDetect: Detecting Python wheel project...
artifacts.whl.AutoDetect: No Python wheel project found at bundle root folder
Starting upload of bundle files
Uploaded bundle files at /Users/andrew.nester@databricks.com/.bundle/wheel-task/default/files!

artifacts.Upload(my_test_code-0.0.1-py3-none-any.whl): Uploading...
artifacts.Upload(my_test_code-0.0.1-py3-none-any.whl): Upload succeeded
 
 ```
2023-08-28 16:29:04 +00:00
Pieter Noordhuis 8656c4a1fa
Log the bundle root configuration file if applicable (#657)
## Changes

Pass through the `context.Context` to the bundle loader functions.

## Tests

Unit tests pass.
2023-08-11 12:28:05 +00:00
Andrew Nester f7a76ff5d8
Fixed processing jobs libraries with remote path (#638)
## Changes
Some library paths such as for Spark jobs, can reference a lib on remote
path, for example DBFS.
This PR fixes how CLI handles such libraries and do not report them as
missing locally.

## Tests
Added unit tests + ran `databricks bundle deploy` manually
2023-08-07 09:55:30 +00:00
Andrew Nester cfff140815
Auto detect Python wheel packages and infer build command (#603) 2023-07-26 10:07:26 +00:00
Andrew Nester 5e0a096722
Fixed python wheel test (#608)
## Changes
Fixed python wheel test

## Tests
<!-- How is this tested? -->
2023-07-26 11:02:17 +02:00
Andrew Nester 9a88fa602d
Added support for artifacts building for bundles (#583)
## Changes
Added support for artifacts building for bundles. 

Now it allows to specify `artifacts` block in bundle.yml and define a
resource (at the moment Python wheel) to be build and uploaded during
`bundle deploy`

Built artifact will be automatically attached to corresponding job task
or pipeline where it's used as a library

Follow-ups:
1. If artifact is used in job or pipeline, but not found in the config,
try to infer and build it anyway
2. If build command is not provided for Python wheel artifact, infer it
2023-07-25 13:35:08 +02:00