Commit Graph

204 Commits

Author SHA1 Message Date
Pieter Noordhuis a390271cd8
Remove unused retries package (#326)
## Changes

The retries package used elsewhere in this repository originates from
https://github.com/databricks/databricks-sdk-go.

## Tests

No impact.
2023-04-12 13:51:33 +02:00
shreyas-goenka 375eb1c502
Remove package project (#321)
## Changes
<!-- Summary of your changes that are easy to understand -->

This PR removes the project package and it's dependents in the bricks
repo

## Tests
<!-- How is this tested? -->
2023-04-11 16:59:27 +02:00
dependabot[bot] 2a0f2f70b7
Bump golang.org/x/term from 0.6.0 to 0.7.0 (#317) 2023-04-11 11:39:21 +02:00
dependabot[bot] a8e6156ed7
Bump github.com/spf13/cobra from 1.6.1 to 1.7.0 (#318) 2023-04-11 11:37:42 +02:00
dependabot[bot] ef2b8c4ecb
Bump golang.org/x/text from 0.8.0 to 0.9.0 (#316) 2023-04-11 11:35:55 +02:00
Fabian Jakobs 1ad5fdaaf0
Update go SDK to 0.7.0 (#310)
## Changes
Update go SDK to 0.7.0

## Tests
`make test`
2023-04-05 15:26:09 +02:00
dependabot[bot] 57cf66d3a8
Bump github.com/databricks/databricks-sdk-go from 0.5.0 to 0.6.0 (#299) 2023-04-03 21:33:21 +02:00
dependabot[bot] e553ea1cf6
Bump github.com/hashicorp/hc-install from 0.5.0 to 0.5.1 (#297) 2023-04-03 21:03:21 +02:00
shreyas-goenka 8fd3dccca9
Add progress logs for job runs (#276) 2023-03-29 14:58:09 +02:00
Pieter Noordhuis 32a29c6af4
Add structured logging infrastructure (#246)
New global flags:
* `--log-file FILE`: can be literal `stdout`, `stderr`, or a file name (default `stderr`)
* `--log-level LEVEL`: can be `error`, `warn`, `info`, `debug`, `trace`, or `disabled` (default `disabled`)
* `--log-format TYPE`: can be `text` or `json` (default `text`)

New functions in the `log` package take a `context.Context` and retrieve
the logger from said context.

Because we carry the logger in a context, adding
[attributes](https://pkg.go.dev/golang.org/x/exp/slog#hdr-Attrs_and_Values)
to the logger can be done as follows:

```go
ctx = log.NewContext(ctx, log.GetLogger(ctx).With("foo", "bar"))
```
2023-03-16 14:46:53 +01:00
dependabot[bot] dccb0aafce
Bump github.com/fatih/color from 1.14.1 to 1.15.0 (#245) 2023-03-15 19:02:08 +01:00
dependabot[bot] 5b0e62d37f
Bump github.com/databricks/databricks-sdk-go from 0.4.1 to 0.5.0 (#247) 2023-03-15 19:01:05 +01:00
dependabot[bot] 54f6f69cb8
Bump github.com/hashicorp/terraform-json from 0.15.0 to 0.16.0 (#241) 2023-03-10 11:57:33 +01:00
Pieter Noordhuis c567b2abc0
Bump databricks-sdk-go to 0.4.1 (#240) 2023-03-09 14:29:53 +01:00
dependabot[bot] 7ade32c734
Bump golang.org/x/mod from 0.8.0 to 0.9.0 (#232) 2023-03-09 10:29:51 +01:00
dependabot[bot] 16efd6960a
Bump github.com/hashicorp/terraform-exec from 0.17.3 to 0.18.1 (#233) 2023-03-09 10:29:36 +01:00
dependabot[bot] 1530065e87
Bump golang.org/x/text from 0.7.0 to 0.8.0 (#231) 2023-03-09 10:28:53 +01:00
dependabot[bot] c5ffa60ec1
Bump github.com/stretchr/testify from 1.8.1 to 1.8.2 (#227) 2023-03-09 10:27:51 +01:00
dependabot[bot] 0ea3d28c3e
Bump github.com/databricks/databricks-sdk-go from 0.3.2 to 0.3.3 (#225) 2023-03-09 10:27:33 +01:00
shreyas-goenka f93b541b63
Show detailed error logs for jobs (#209)
PR for how to render errors on console for jobs. 
Here is the bundle used for the logs below:
```
bundle:
  name: deco-438

workspace:
  host: https://adb-309687753508875.15.azuredatabricks.net

resources:
  jobs:
    foo:
      name: "[${bundle.name}][${bundle.environment}] a test notebook"

      tasks:
        - task_key: alpha
          existing_cluster_id: 1109-115254-ox7poobk
          notebook_task:
            notebook_path: "/Users/shreyas.goenka@databricks.com/[deco-438] invalid notebook"
        - task_key: beta
          existing_cluster_id: 1109-115254-ox7poobk
          notebook_task:
            notebook_path: "/does-not-exist"
        - task_key: gamma
          existing_cluster_id: 1109-115254-ox7poobk
          notebook_task:
            notebook_path: "/Users/shreyas.goenka@databricks.com/[deco-438] valid notebook"
```

And this is a screenshot of the logs from the console:
<img width="1057" alt="Screenshot 2023-02-17 at 7 12 29 PM"
src="https://user-images.githubusercontent.com/88374338/219744768-ab7f1e79-db8f-466a-ad6d-f2b6f85ed17c.png">

Here are the logs when only tasks gamma is executed (successfully):
<img width="1059" alt="Screenshot 2023-02-17 at 7 13 04 PM"
src="https://user-images.githubusercontent.com/88374338/219744992-011d8b91-ec1d-44f0-a849-83c81816dd9f.png">


TODO: Investigate more possible job errors, and make sure state for them
is handled in a robust way here
2023-02-20 23:40:14 +01:00
Pieter Noordhuis 414ea4f891
Bump databricks-sdk-go to 0.3.2 (#215) 2023-02-20 16:00:20 +01:00
dependabot[bot] ded7d8c404
Bump golang.org/x/net from 0.1.0 to 0.7.0 (#210) 2023-02-20 11:23:25 +01:00
dependabot[bot] 1810a8ba61
Bump github.com/hashicorp/terraform-json from 0.14.0 to 0.15.0 (#202) 2023-02-20 10:57:33 +01:00
dependabot[bot] a94d023776
Bump golang.org/x/text from 0.6.0 to 0.7.0 (#201) 2023-02-20 10:56:53 +01:00
dependabot[bot] 04f16b20b1
Bump golang.org/x/mod from 0.7.0 to 0.8.0 (#200) 2023-02-20 10:56:31 +01:00
dependabot[bot] aa4ba29879
Bump github.com/hashicorp/hc-install from 0.4.0 to 0.5.0 (#198) 2023-02-20 10:21:47 +01:00
dependabot[bot] c6a0d53566
Bump golang.org/x/text from 0.5.0 to 0.6.0 (#163)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.5.0 to
0.6.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ec5565b1b7"><code>ec5565b</code></a>
README.md: update documentation of module versioning</li>
<li>See full diff in <a
href="https://github.com/golang/text/compare/v0.5.0...v0.6.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/text&package-manager=go_modules&previous-version=0.5.0&new-version=0.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-13 15:27:40 +01:00
Serge Smertin 6d04eb0f0c
Update to Go SDK v0.2.0 (#157)
This PR updates Go SDK to v0.2.0
2022-12-28 11:32:04 +01:00
Pieter Noordhuis 7f83463ca3
Bump SDK to latest (#151) 2022-12-22 09:46:17 +01:00
dependabot[bot] fa458406ea
Bump github.com/databricks/databricks-sdk-go from 0.1.0 to 0.1.1 (#121) 2022-12-14 11:21:44 +01:00
dependabot[bot] 4631c70279
Bump github.com/spf13/cobra from 1.5.0 to 1.6.1 (#115) 2022-12-14 11:00:44 +01:00
dependabot[bot] 7f3094e4d7
Bump golang.org/x/mod from 0.6.0 to 0.7.0 (#114) 2022-12-14 10:58:30 +01:00
shreyas-goenka d9d295f2a9
Implement Terraform state synchronization and deploy (#98)
https://user-images.githubusercontent.com/88374338/203669797-abebf99e-8fa6-4d6e-b57a-abd172d8020d.mov
2022-12-06 00:40:45 +01:00
Serge Smertin 487bf6fd5c
Use Databricks Go SDK v0.1.0 (#110)
This PR pins the version of Databricks SDK for Go to v0.1.0
2022-12-01 12:17:36 +01:00
Pieter Noordhuis e47fa61951
Skeleton for configuration loading and mutation (#92)
Load a tree of configuration files anchored at `bundle.yml` into the
`config.Root` struct.

All mutations (from setting defaults to merging files) are observable
through the `mutator.Mutator` interface.
2022-11-18 10:57:31 +01:00
Pieter Noordhuis 6a8c9f22b8
Update databricks-sdk-go to latest (#93) 2022-11-17 11:24:09 +01:00
Pieter Noordhuis af5659ae8e
Update databricks-sdk-go to latest (#89) 2022-11-09 15:01:47 +01:00
shreyas-goenka d587531cbd
Added creation of .gitignore for bricks project with cache dir path (#88)
It works, please trust me
2022-11-08 13:51:08 +01:00
Shreyas Goenka efe1f28b36
Revert "Added creation of .gitignore for bricks project with cache dir path"
This reverts commit 1505c63426.
2022-11-03 21:09:29 +01:00
Shreyas Goenka 1505c63426
Added creation of .gitignore for bricks project with cache dir path 2022-11-03 21:06:21 +01:00
shreyas-goenka 0b754e6de8
[DECO-94] Execute uploads in parallel instead of sequentailly (#81)
Tested manually

Upload seems fast enough, delete API calls though have a much longer
turn around times

Additional optimizations that can be done if/when the need arises:
1. First time upload can be done using zip batching of the files


https://user-images.githubusercontent.com/88374338/192783332-9b2b19bc-d6c4-4a66-8dbc-e78287e6af1a.mov
2022-10-05 00:12:57 +02:00
Pieter Noordhuis a7701cc8f3
Store project object in context.Context instead of global (#61)
* Load project root from `BRICKS_ROOT` environment variable
* Rename project.Project -> project.Config
* Rename project.inner -> project.project
* Upgrade cobra to 1.5.0 for cmd.SetContext
2022-09-16 11:06:58 +02:00
Pieter Noordhuis 14a45e701e
Run go mod tidy (#55) 2022-09-14 10:48:51 +02:00
shreyas-goenka 96efd0e2e4
Replace terraform dependency with go sdk (#19)
Issue: https://github.com/databricks/bricks/issues/17

`./bricks fs ls ...` command works
`./bricks launch ...` command works

Did not test other changes as the readme claims other commands don't
work anyways :) cc: @nfx

TODO left for this PR:
2. Replace terraform scim.Me once its there in go SDK
(https://github.com/databricks/databricks-sdk-go/issues/56)
2022-09-07 11:55:59 +02:00
Kartik Gupta 457f3ad3c2
Add `bricks configure` command to bricks CLI (#18)
* bricks configure

* remove t.setenv

* Read token and host from stdin

* Update .vscode/testing.code-snippets

Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com>

Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com>
2022-09-05 20:25:54 +02:00
dependabot[bot] bb50563c60
Bump github.com/stretchr/testify from 1.7.1 to 1.8.0 (#13)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.7.1 to 1.8.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.7.1...v1.8.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-02 14:00:38 +02:00
dependabot[bot] 081f039a93
Bump gopkg.in/ini.v1 from 1.66.4 to 1.67.0 (#16)
Bumps [gopkg.in/ini.v1](https://github.com/go-ini/ini) from 1.66.4 to 1.67.0.
- [Release notes](https://github.com/go-ini/ini/releases)
- [Commits](https://github.com/go-ini/ini/compare/v1.66.4...v1.67.0)

---
updated-dependencies:
- dependency-name: gopkg.in/ini.v1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-02 14:00:16 +02:00
Shreyas Goenka f70d42fff6
Revert "Added go sdk as a dependency"
This reverts commit 0ae8b60573.
2022-09-01 13:42:35 +02:00
Shreyas Goenka 0ae8b60573
Added go sdk as a dependency 2022-08-31 19:05:05 +02:00
Serge Smertin 32ae59c1bc Experimental sync command 2022-07-07 20:56:59 +02:00
dependabot[bot] 82438b9f1c
Bump github.com/databrickslabs/terraform-provider-databricks (#1)
Bumps [github.com/databrickslabs/terraform-provider-databricks](https://github.com/databrickslabs/terraform-provider-databricks) from 0.5.7 to 0.5.8.
- [Release notes](https://github.com/databrickslabs/terraform-provider-databricks/releases)
- [Changelog](https://github.com/databrickslabs/terraform-provider-databricks/blob/master/CHANGELOG.md)
- [Commits](https://github.com/databrickslabs/terraform-provider-databricks/compare/v0.5.7...v0.5.8)

---
updated-dependencies:
- dependency-name: github.com/databrickslabs/terraform-provider-databricks
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-23 10:49:35 +02:00
Serge Smertin 3907dcdba9 wrap `terraform` execution PoC 2022-05-21 14:31:26 +02:00
Serge Smertin 3d3b722eda updated dependencies 2022-05-14 19:56:09 +02:00
Serge Smertin 15fd93a012 Initial commit 2022-05-13 15:30:22 +02:00