Commit Graph

771 Commits

Author SHA1 Message Date
shreyas-goenka 0b754e6de8
[DECO-94] Execute uploads in parallel instead of sequentailly (#81)
Tested manually

Upload seems fast enough, delete API calls though have a much longer
turn around times

Additional optimizations that can be done if/when the need arises:
1. First time upload can be done using zip batching of the files


https://user-images.githubusercontent.com/88374338/192783332-9b2b19bc-d6c4-4a66-8dbc-e78287e6af1a.mov
2022-10-05 00:12:57 +02:00
Pieter Noordhuis a1b6fdb2e8
Update SDK (#79) 2022-09-27 09:58:55 -07:00
Serge Smertin f27cb7149d
Reduce nesting (#72) 2022-09-22 15:03:30 +02:00
Pieter Noordhuis 6258a1637d
Add environments to project configuration (#68) 2022-09-22 13:40:11 +02:00
shreyas-goenka 6bcb33bf07
Use env vars to read profile if databricks.yml is absent (#70)
Tested manually and by running sync_test.go integration test bricks sync
works
2022-09-21 13:55:37 +02:00
shreyas-goenka 731679cb4b
Add `persist-snapshot` to bricks sync (#66)
Tested manually

We are adding this flag because the default bricks sync is not robust
against changing the profile and other project config changes. This will
be used in the initial version of the vscode extention
2022-09-19 16:47:55 +02:00
Pieter Noordhuis 7cad8bda81
Respect project root in sync command (#63) 2022-09-16 15:18:46 +02:00
Pieter Noordhuis ec6c58f1d1
Fix relative path derivation if root path can be cleaned (#64) 2022-09-16 15:13:49 +02:00
Pieter Noordhuis a7701cc8f3
Store project object in context.Context instead of global (#61)
* Load project root from `BRICKS_ROOT` environment variable
* Rename project.Project -> project.Config
* Rename project.inner -> project.project
* Upgrade cobra to 1.5.0 for cmd.SetContext
2022-09-16 11:06:58 +02:00
shreyas-goenka 836ab58473
Fix flaky test `TestDiff` (#59) 2022-09-15 15:40:47 +02:00
Pieter Noordhuis 192a790155
Comment out flavor.go (#60)
The code here isn't used at the moment.
2022-09-15 09:46:11 +02:00
shreyas-goenka f9b66b3536
Make `bricks sync` feature work (#48)
Tested manually and partially by unit tests
2022-09-14 17:50:29 +02:00
Pieter Noordhuis 281f25a627
Don't depend on working directory in folders.FindDirWithLeaf (#54)
This is plumbing for allowing a configurable root that doesn't depend on
the working directory.
2022-09-14 15:08:55 +02:00
Pieter Noordhuis 14a45e701e
Run go mod tidy (#55) 2022-09-14 10:48:51 +02:00
Pieter Noordhuis 40194cc1a2
Scope snippets to Go (#51)
Confirmed they only show up in Go code now.
2022-09-13 17:58:59 +02:00
Fabian Jakobs a168eece47
Build 32 bit Windows (#47)
Win32 is needed by VSCode as a fallback.
2022-09-09 10:31:41 +02:00
shreyas-goenka 21bc774491
Replace scim Me terraform call with go sdk (#46)
This PR:
1. Replaces scim.Me call to use the go SDK instead of the terraform
client
2. Removes terraform client from bricks project

Tested manually that the scim.Me call works now and returns the correct
user
go build works
2022-09-08 15:50:00 +02:00
Fabian Jakobs d829ce36d5
fix release (#45) 2022-09-08 15:12:26 +02:00
Fabian Jakobs f309f68444
Fix release action and make sure to also build for Windows (#43) 2022-09-08 14:57:33 +02:00
Fabian Jakobs 1ddba0caa8
Allow manually triggering releases 2022-09-08 14:13:43 +02:00
Fabian Jakobs ea19417336
Also build for windows (#42) 2022-09-08 13:27:05 +02:00
Pieter Noordhuis d214f44df4
Use actions/setup-go@v3 with caching support (#41) 2022-09-08 11:21:00 +02:00
Pieter Noordhuis 2e12a2aa01
Make tests pass (#40)
By:
* Add .gitkeep to retain test fixture directories under
./python/testdata
* Move GitHub related functionality to ./experimental (it is not in use)
* Comment out test in ./cmd/sync
* Fix test in ./git
2022-09-07 20:08:42 +02:00
Pieter Noordhuis 5a55cad7c3
Ensure Go code is formatted (#37) 2022-09-07 15:15:23 +02:00
Pieter Noordhuis 80a4c47d62
Fix lint error regarding struct with unexported fields (#38)
Unexported fields are skipped during marshalling, so this would be a
nop.

The code in `./cmd/sync/github*.go` is currently unused.

Confirmed that staticcheck passes when run with:

```
staticcheck -checks SA9005 ./...
```
2022-09-07 15:15:07 +02:00
Pieter Noordhuis c00db56d84
Fix lint errors for using deprecated functionality (#35)
Functionality from `io/ioutil` has moved to the `io` and `os` packages
in go1.16 ([reference](https://pkg.go.dev/io/ioutil)).

Confirmed that staticcheck passes when run with:

```
staticcheck -checks SA1019 ./...
```
2022-09-07 14:30:10 +02:00
Pieter Noordhuis 31a841ff33
Fix lint errors for unused functions (#36)
The functionality under terraform/ isn't used anywhere at the moment and
the test doesn't pass for me. It will be useful down the line so
commenting out instead of removing.

Confirmed that staticcheck passes when run with:
```
staticcheck -checks U1000 ./...
```
2022-09-07 14:26:31 +02:00
Pieter Noordhuis 67b2e4206f
Use t.Setenv instead of custom implementation (#34)
Follow up for #18. This function was introduced in go1.17. As of #19 we
require go1.18, so we can use it.
2022-09-07 13:24:11 +02:00
shreyas-goenka 96efd0e2e4
Replace terraform dependency with go sdk (#19)
Issue: https://github.com/databricks/bricks/issues/17

`./bricks fs ls ...` command works
`./bricks launch ...` command works

Did not test other changes as the readme claims other commands don't
work anyways :) cc: @nfx

TODO left for this PR:
2. Replace terraform scim.Me once its there in go SDK
(https://github.com/databricks/databricks-sdk-go/issues/56)
2022-09-07 11:55:59 +02:00
Kartik Gupta f070c24399
Write default header only if `DEFAULT` section exists (#31) 2022-09-06 18:04:05 +02:00
Kartik Gupta 30a7de621a
Add more flags to `configure` command (#29) 2022-09-06 16:37:58 +02:00
Kartik Gupta 457f3ad3c2
Add `bricks configure` command to bricks CLI (#18)
* bricks configure

* remove t.setenv

* Read token and host from stdin

* Update .vscode/testing.code-snippets

Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com>

Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com>
2022-09-05 20:25:54 +02:00
dependabot[bot] bb50563c60
Bump github.com/stretchr/testify from 1.7.1 to 1.8.0 (#13)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.7.1 to 1.8.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.7.1...v1.8.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-02 14:00:38 +02:00
dependabot[bot] 081f039a93
Bump gopkg.in/ini.v1 from 1.66.4 to 1.67.0 (#16)
Bumps [gopkg.in/ini.v1](https://github.com/go-ini/ini) from 1.66.4 to 1.67.0.
- [Release notes](https://github.com/go-ini/ini/releases)
- [Commits](https://github.com/go-ini/ini/compare/v1.66.4...v1.67.0)

---
updated-dependencies:
- dependency-name: gopkg.in/ini.v1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-02 14:00:16 +02:00
Shreyas Goenka f70d42fff6
Revert "Added go sdk as a dependency"
This reverts commit 0ae8b60573.
2022-09-01 13:42:35 +02:00
Shreyas Goenka 9951068c62
Revert "Using go sdk for ./bricks fs ls DIR_NAME command"
This reverts commit 6217d20e57.
2022-09-01 13:42:34 +02:00
Shreyas Goenka 5e2c5888e0
Revert "Replaced usage of Cluster struct from terraform provider to ClusterInfo from go-sdk in config.go"
This reverts commit a9a7672725.
2022-09-01 13:42:31 +02:00
Shreyas Goenka a9a7672725
Replaced usage of Cluster struct from terraform provider to ClusterInfo from go-sdk in config.go 2022-08-31 19:46:35 +02:00
Shreyas Goenka 6217d20e57
Using go sdk for ./bricks fs ls DIR_NAME command 2022-08-31 19:31:58 +02:00
Shreyas Goenka 0ae8b60573
Added go sdk as a dependency 2022-08-31 19:05:05 +02:00
Serge Smertin 32ae59c1bc Experimental sync command 2022-07-07 20:56:59 +02:00
Serge Smertin 95a68937fe rename `internal/test` to idiomatic `testdata` 2022-05-23 11:46:37 +02:00
Serge Smertin 0d76ab3d28 upload wheel to pep503-compatible DBFS path 2022-05-23 11:41:55 +02:00
dependabot[bot] 82438b9f1c
Bump github.com/databrickslabs/terraform-provider-databricks (#1)
Bumps [github.com/databrickslabs/terraform-provider-databricks](https://github.com/databrickslabs/terraform-provider-databricks) from 0.5.7 to 0.5.8.
- [Release notes](https://github.com/databrickslabs/terraform-provider-databricks/releases)
- [Changelog](https://github.com/databrickslabs/terraform-provider-databricks/blob/master/CHANGELOG.md)
- [Commits](https://github.com/databrickslabs/terraform-provider-databricks/compare/v0.5.7...v0.5.8)

---
updated-dependencies:
- dependency-name: github.com/databrickslabs/terraform-provider-databricks
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-23 10:49:35 +02:00
Serge Smertin 515a6930f8 interation on `.tfstate` sync stability
* read `serial` from `.tfstate`.
* defer local `.tfstate` handle closing on method exits.
* document private methods not to forget their intentions.
2022-05-23 10:34:58 +02:00
Serge Smertin 06db8376bc local and remote `.tfstate` utilities 2022-05-21 15:23:37 +02:00
Serge Smertin 3907dcdba9 wrap `terraform` execution PoC 2022-05-21 14:31:26 +02:00
Serge Smertin b85e63684b added essential Python metadata detection 2022-05-20 21:40:03 +02:00
Serge Smertin ae2dc104f9 add some comments to commands package 2022-05-20 20:43:29 +02:00
Serge Smertin 99ce4f71de added dummy wheel 2022-05-20 20:42:53 +02:00