steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,385 Commits 85 Branches 104 Tags 56 MiB
Commit Graph

13 Commits

Author SHA1 Message Date
Shreyas Goenka 68dc6c1ce4
Merge remote-tracking branch 'origin' into feature/uc-volumes 2024-11-18 16:56:37 +01:00
dependabot[bot] 25838ee0af
Bump github.com/databricks/databricks-sdk-go from 0.49.0 to 0.51.0 (#1878) 
Known issues:

- [ ] _(non-blocking with a command override)_ `apps.Update` requires 2
`name` params (one from path, one from request body)
- [ ] _(non-blocking)_ `lakeview.Create` does not require positional
argument `display_name` anymore because it's not marked as required in
request body

Bumps
[github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go)
from 0.49.0 to 0.51.0.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Nester <andrew.nester@databricks.com>
 2024-11-13 13:40:53 +00:00
Shreyas Goenka 1218178e64
- 2024-10-31 18:01:37 +01:00
Shreyas Goenka 8a2fe4969c
merge 2024-10-29 11:38:42 +01:00
Pieter Noordhuis 11f75fd320
Add support for AI/BI dashboards (#1743) 
## Changes

This change adds support for modeling [AI/BI dashboards][docs] in DABs.


[Example bundle configuration][example] is located in the
`bundle-examples` repository.

[docs]: https://docs.databricks.com/en/dashboards/index.html#dashboards
[example]:
https://github.com/databricks/bundle-examples/tree/main/knowledge_base/dashboard_nyc_taxi

## Tests

* Added unit tests for self-contained parts
* Integration test for e2e dashboard deployment and remote change
modification
 2024-10-29 09:11:08 +00:00
Shreyas Goenka 810da66809
- 2024-10-16 13:36:55 +02:00
Shreyas Goenka 6192835d63
add custom prefixing behaviour for volumes 2024-10-15 18:04:20 +02:00
Shreyas Goenka e43f566579
Merge remote-tracking branch 'origin' into feature/uc-volumes 2024-10-14 15:04:40 +02:00
Lennart Kats (databricks) da3b4f7c72
Fix panic in `apply_presets.go` (#1796) 
## Changes

This fixes the user-reported panic in `apply_presets.go`. I'm still
unsure how to reproduce this, since the CLI just reports `ob broken_job
is not defined` when I try to use `bundle deploy` with an empty job.
That said — we may as well be defensive here and I see we have lots of
checks for empty job/cluster/etc. settings scattered throughout our code
base so at least we're somewhat consistent.
 2024-09-29 14:08:10 +00:00
Andrew Nester 56ed9bebf3
Added support for creating all-purpose clusters (#1698) 
## Changes
Added support for creating all-purpose clusters

Example of configuration

```
bundle:
  name: clusters

resources:
  clusters:
    test_cluster:
      cluster_name: "Test Cluster"
      num_workers: 2
      node_type_id: "i3.xlarge"
      autoscale:
        min_workers: 2
        max_workers: 7
      spark_version: "13.3.x-scala2.12"
      spark_conf:
        "spark.executor.memory": "2g"

  jobs:
    test_job:
      name: "Test Job"
      tasks:
        - task_key: test_task
          existing_cluster_id: ${resources.clusters.test_cluster.id}
          notebook_task:
            notebook_path: "./src/test.py"

targets:
    development:
      mode: development
      compute_id: ${resources.clusters.test_cluster.id}

```

## Tests
Added unit, config and E2E tests
 2024-09-23 10:42:34 +00:00
Shreyas Goenka 9b66cd523b
add apply target mode prefix functionality 2024-09-09 15:11:57 +02:00
shreyas-goenka f5df211320
Fix prefix preset used for UC schemas (#1704) 
## Changes
In https://github.com/databricks/cli/pull/1490 we regressed and started
using the development mode prefix for UC schemas regardless of the mode
of the bundle target.

This PR fixes the regression and adds a regression test

## Tests
Failing integration tests pass now.
 2024-08-21 12:53:54 +00:00
Lennart Kats (databricks) 78d0ac5c6a
Add configurable presets for name prefixes, tags, etc. (#1490) 
## Changes

This adds configurable transformations based on the transformations
currently seen in `mode: development`.

Example databricks.yml showcasing how some transformations:

```
bundle:
  name: my_bundle

targets:
  dev:
    presets:
      prefix: "myprefix_"          # prefix all resource names with myprefix_
      pipelines_development: true  # set development to true by default for pipelines
      trigger_pause_status: PAUSED # set pause_status to PAUSED by default for all triggers and schedules
      jobs_max_concurrent_runs: 10 # set max_concurrent runs to 10 by default for all jobs
      tags:
        dev: true
```

## Tests

* Existing process_target_mode tests that were adapted to use this new
code
* Unit tests specific for the new mutator
* Unit tests for config loading and merging
* Manual e2e testing
 2024-08-19 18:18:50 +00:00