steve/databricks-cli
steve
/
databricks-cli
mirror of https://github.com/databricks/cli.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
178 Commits 85 Branches 104 Tags 56 MiB
Commit Graph

178 Commits

Author SHA1 Message Date
Shreyas Goenka bd6d0dca57
added test for map of map 2023-01-17 17:57:48 +01:00
Shreyas Goenka 753c54e899
added support for additionalProperties = false for validation and a test for a map of struct 2023-01-17 17:53:58 +01:00
Shreyas Goenka 91617e50c1
added test for map of primitives 2023-01-17 15:47:31 +01:00
Shreyas Goenka f556e62c3b
added tests for struct of map and struct of slice 2023-01-17 15:43:35 +01:00
Shreyas Goenka 2021ca1886
Added test for struct of structs 2023-01-17 15:34:49 +01:00
Shreyas Goenka 019b7d7fa9
added tests for schemas for primitives 2023-01-17 15:08:17 +01:00
Shreyas Goenka ce7b2133bf
Added comments for the schema struct fields 2023-01-17 13:59:56 +01:00
Shreyas Goenka d2f466afc5
replaced properties and items with schema 2023-01-17 13:47:31 +01:00
Shreyas Goenka cc350f761a
correct typo and some comments 2023-01-17 13:43:50 +01:00
Shreyas Goenka e79c52ff37
Added doc string for type mapping 2023-01-17 12:25:25 +01:00
Shreyas Goenka aa81d83970
added ignoring - fields and tests for ignoring dashed and non annotated fields 2023-01-17 11:58:32 +01:00
Shreyas Goenka 2a66a2fc74
added comment for what to do about cycles in json schema 2023-01-17 11:26:46 +01:00
Shreyas Goenka 7385f37df6
fixed typo 2023-01-16 21:53:20 +01:00
Shreyas Goenka 2945408f0d
add test for errWithTrace 2023-01-16 13:19:31 +01:00
Shreyas Goenka 42d82ede6e
- 2023-01-16 02:30:41 +01:00
Shreyas Goenka 4438d1b37c
some todo and comment cleanup 2023-01-16 02:27:49 +01:00
Shreyas Goenka ae201f1bd3
comments and typo correction 2023-01-16 02:23:35 +01:00
Shreyas Goenka 276ba00067
fixed debugTrace 2023-01-13 20:03:24 +01:00
Shreyas Goenka 1ee35c3633
corrected enum 2023-01-13 19:40:09 +01:00
Shreyas Goenka d605ab1aa4
using list for error tracing 2023-01-13 19:37:35 +01:00
Shreyas Goenka 1538422d27
some cleanup 2023-01-13 19:26:50 +01:00
Shreyas Goenka 0f67d7c0d7
added test for embedded structs 2023-01-13 19:25:42 +01:00
Shreyas Goenka b1e85e20f6
cleaned up test boilerplate, added support for maps and embedded types 2023-01-13 19:08:21 +01:00
Shreyas Goenka 010f60d498
added slice of object handling 2023-01-13 18:01:53 +01:00
Shreyas Goenka babd7fa4eb
corrected item typo 2023-01-13 17:28:46 +01:00
Shreyas Goenka cc5ed2bc76
fixed typo, and tested schema is working 2023-01-13 17:13:21 +01:00
Shreyas Goenka 926db54a6d
added tests for primitives and basic objects/arrays 2023-01-13 16:45:01 +01:00
Shreyas Goenka 987dd29357
added recursive limit 2023-01-13 12:09:19 +01:00
Shreyas Goenka d46dbbce6a
WIP json schema support for bundles 2023-01-13 04:52:25 +01:00
Pieter Noordhuis 3582037be6
Add nil check for retries.Info.Info (#166) 2023-01-12 18:58:36 +01:00
Kartik Gupta e48eb6ff50
Fix integration tests for windows (#165) 
Successful run: 
Azure:
https://github.com/databricks/eng-dev-ecosystem/actions/runs/3892131225
AWS:
https://github.com/databricks/eng-dev-ecosystem/actions/runs/3892432562
 2023-01-11 13:32:02 +01:00
shreyas-goenka bf9f0c06b0
Fix sync integration test failing on aws-prod (#164) 
The tests were failing because we were the list of expected files did
not include .gitignore

aws and azure test runs:

https://github.com/databricks/eng-dev-ecosystem/actions/runs/3891837322/jobs/6642552850


https://github.com/databricks/eng-dev-ecosystem/actions/runs/3891836306/jobs/6642550633
 2023-01-11 11:50:27 +01:00
shreyas-goenka 0d9ecb5643
Refactor and cover edge cases in sync integration tests (#160) 
This PR:
1. Refactors the sync integration tests to make them more readable
2. Adds additional tests for edge cases we encountered during vscode
runs
3. Intensional side effect: sync integration tests are also green on
windows (see
https://github.com/databricks/eng-dev-ecosystem/actions/runs/3817365642/jobs/6493576727)

Change in coverage

- We now test for python notebook <-> python file interconversion and
python notebook deletion being synced to workspace
- Tests are split up and are more focused on testing specific edge cases
 2023-01-10 13:16:30 +01:00
Serge Smertin b87b4b0f40
Added `bricks auth login` and `bricks auth token` (#158) 
# Auth challenge (happy path)

Simplified description of [PKCE](https://oauth.net/2/pkce/)
implementation:

```mermaid
sequenceDiagram
    autonumber
    actor User
    
    User ->> CLI: type `bricks auth login HOST`
    CLI ->>+ HOST: request OIDC endpoints
    HOST ->>- CLI: auth & token endpoints
    CLI ->> CLI: start embedded server to consume redirects (lock)
    CLI -->>+ Auth Endpoint: open browser with RND1 + SHA256(RND2)

    User ->>+ Auth Endpoint: Go through SSO
    Auth Endpoint ->>- CLI: AUTH CODE + 'RND1 (redirect)

    CLI ->>+ Token Endpoint: Exchange: AUTH CODE + RND2
    Token Endpoint ->>- CLI: Access Token (JWT) + refresh + expiry
    CLI ->> Token cache: Save Access Token (JWT) + refresh + expiry
    CLI ->> User: success
```

# Token refresh (happy path)

```mermaid
sequenceDiagram
    autonumber
    actor User
    
    User ->> CLI: type `bricks token HOST`
    
    CLI ->> CLI: acquire lock (same local addr as redirect server)
    CLI ->>+ Token cache: read token

    critical token not expired
    Token cache ->>- User: JWT (without refresh)

    option token is expired
    CLI ->>+ HOST: request OIDC endpoints
    HOST ->>- CLI: auth & token endpoints
    CLI ->>+ Token Endpoint: refresh token
    Token Endpoint ->>- CLI: JWT (refreshed)
    CLI ->> Token cache: save JWT (refreshed)
    CLI ->> User: JWT (refreshed)
    
    option no auth for host
    CLI -X User: no auth configured
    end
```
 2023-01-06 16:15:57 +01:00
Pieter Noordhuis a59136f77f
Use []byte for files in workspace (#162) 2023-01-05 12:03:31 +01:00
Serge Smertin a9b82aa1c7
Remove version suffix from snapshot binaries (#159) 
After this change it's possible to have a stable symlink for
development:

<img width="172" alt="image"
src="https://user-images.githubusercontent.com/259697/209996850-68bcb6da-b8f9-4b5a-9ac3-ff3ef35cd751.png">
 2023-01-03 12:15:21 +01:00
Serge Smertin 6d04eb0f0c
Update to Go SDK v0.2.0 (#157) 
This PR updates Go SDK to v0.2.0
 2022-12-28 11:32:04 +01:00
Pieter Noordhuis 8f4461904b
Define flags for running jobs and pipelines (#146) 2022-12-23 15:17:16 +01:00
Pieter Noordhuis 49aa858b89
Run command must always take a single argument (#156) 2022-12-22 16:19:38 +01:00
Pieter Noordhuis 61ef0ba8c6
Handle nil environment (#154) 2022-12-22 15:31:32 +01:00
shreyas-goenka 198eefcf39
Fix folder syncing on windows (#149) 
Tested by running the unit and integration tests locally

Tested manually on windows

Screenshot from windows sync logs indicating that the correct slashed
for paths were used:
<img width="623" alt="Screenshot 2022-12-21 at 9 09 13 PM"
src="https://user-images.githubusercontent.com/88374338/208943937-146670b2-1afd-4e0b-8f4e-6091c8c7e17a.png">

@pietern with this the state machine for syncing becomes slightly more
complicated, indicating a stronger need for a tree based approach herre

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
 2022-12-22 11:35:32 +01:00
Pieter Noordhuis f3112d90ad
Work on release actions (#153) 
* Include Go version in cache key
* Store build artifacts by operating systems
 2022-12-22 11:01:50 +01:00
Pieter Noordhuis f5cb9e06a9
Add workflow for building release snapshots (#152) 
Use custom caching because goreleaser performs a cross platform build
and therefore we cannot share a cache with the test builds.
 2022-12-22 10:25:24 +01:00
Pieter Noordhuis 7f83463ca3
Bump SDK to latest (#151) 2022-12-22 09:46:17 +01:00
Pieter Noordhuis ce55284398
Run go mod tidy in fmt job (#150) 2022-12-22 09:33:46 +01:00
Pieter Noordhuis fdb8c97f6b
Exit with non-zero status on errors (#148) 2022-12-21 11:58:51 +01:00
Pieter Noordhuis a354fa1f77
Only display usage string on flag errors (#147) 2022-12-21 11:38:30 +01:00
Pieter Noordhuis f70a5814f6
Run initialize phase from validate command (#145) 
This lets the user know if their workspace connection is setup
correctly.
 2022-12-16 15:00:48 +01:00
Pieter Noordhuis 4026b2cda2
Mutator to convert paths to local notebooks files into artifacts (#144) 
This lets you write:
```yaml
libraries:
  - notebook:
      path: ./events.sql
```

Instead of:
```yaml
artifacts:
  events_sql:
    notebook:
      path: ./events.sql

libraries:
  - notebook:
      path: "${artifacts.events_sql.notebook.remote_path}"
```
 2022-12-16 14:49:23 +01:00
Pieter Noordhuis 1a9a431b97
No need for nil check on map (#143) 2022-12-15 21:28:27 +01:00