Commit Graph

35 Commits

Author SHA1 Message Date
Gleb Kanterov f582c7962b
Merge bbcbffd7f4 into 85c0d2d3ee 2024-11-26 18:19:04 +00:00
Pieter Noordhuis 4a2d866f9a
Use Go 1.23 (#1871)
## Changes

This was released 2+ months ago so it has baked enough.

Blog post: https://go.dev/blog/go1.23.

## Tests

None other than unit and integration tests.
2024-10-31 12:42:14 +00:00
Pieter Noordhuis 7f1121d8d8
Pin Go toolchain to 1.22.7 (#1790)
## Changes

Relates to https://github.com/databricks/cli/pull/1758.

More information about toolchains:
* https://go.dev/blog/toolchain
* https://go.dev/doc/toolchain

We need to specify the toolchain as we need to bump Go to 1.22.0 for the
`mod` upgrade and want to use the latest toolchain on the 1.22 series.

## Tests

The previous release was made with Go 1.22.7 so we should continue to
use it.
2024-09-25 15:45:28 +00:00
Gleb Kanterov 6fd701ec84
Bump python version 2024-09-25 16:24:59 +02:00
shreyas-goenka c61358407f
Add end to end integration tests for bundle JSON schema (#1726) 2024-09-11 09:15:56 +00:00
Pieter Noordhuis 80136dea5f
Use Go 1.22 to build and test (#1562)
## Changes

This has been released for a while. Blog post:
https://go.dev/blog/go1.22.

## Tests

None besides the unit tests.
2024-07-04 06:54:41 +00:00
shreyas-goenka cddc5f97f8
Fix the generated DABs JSON schema (#1322)
## Changes
This PR fixes bundle schema being broken because `for_each_task: null`
was set in the generated schema. This is not valid according to the JSON
schema specification and thus the Red Hat YAML VSCode extension was
failing to parse the YAML configuration.

This PR fixes: https://github.com/databricks/cli/issues/1312

## Tests
The fix itself was tested manually. I asserted that the autocompletion
works now. This was mistakenly overlooked the first time around when the
regression was introduced in https://github.com/databricks/cli/pull/1204
because the YAML extension provides best-effort autocomplete suggestions
even if the JSON schema fails to load.

To prevent future regressions we also add a test to assert that the JSON
schema generated itself is a valid JSON schema object. This is done via
using the `ajv-cli` to validate the schema. This package is also used by
the Red Hat YAML extension and thus provides a high fidelity check for
ensuring the JSON schema is valid.

Before, with the old schema:
```
shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-216.json -d ../bundle-playground-3/databricks.yml
schema proj/schema-216.json is invalid
error: schema is invalid: data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items/properties/for_each_task must be object,boolean, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must be array, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must match a schema in anyOf
```

After, with the new schema:
```
shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-dev.json -d ../bundle-playground-3/databricks.yml
../bundle-playground-3/databricks.yml valid
```

After, autocomplete suggestions:

<img width="600" alt="Screenshot 2024-03-27 at 6 35 57 PM"
src="https://github.com/databricks/cli/assets/88374338/d0a62402-e323-4f36-854d-332b33cbeab8">
2024-03-28 11:25:36 +00:00
Pieter Noordhuis 4f2c6150e7
Update codecov/codecov-action to v4 (#1291)
## Changes

This addresses a Node 16 deprecation warning in our GHA output.

Full release notes of v4 at
https://github.com/codecov/codecov-action/releases/tag/v4.0.0
2024-03-18 12:48:59 +00:00
Pieter Noordhuis c49c8cc04c
Update actions/setup-python to v5 (#1290)
## Changes

This addresses a Node 16 deprecation warning in our GHA output.
2024-03-18 12:48:50 +00:00
Pieter Noordhuis 2d38d14703
Use latest patch release of Go toolchain (#1152)
## Changes

This was pinned to 1.21.0 and included a vulnerability as reported in
#1150. The vulnerability does not affect the prior CLI releases as it
requires a user to execute Go commands from within compromised module
directories.

Fixes #1150.
2024-01-25 12:18:35 +00:00
Pieter Noordhuis 8988920a3e
Update codecov/codecov-action to v3 (#1149)
## Changes

v1 has been deprecated for a long time and produces warnings in action
output.
2024-01-25 10:47:03 +00:00
Pieter Noordhuis 8922bf916c
Update actions/setup-go to v5 (#1148)
## Changes

This silences the following warning as seen in action output:

> Node.js 16 actions are deprecated. Please update the following actions
to use Node.js 20: actions/setup-go@v4.
2024-01-25 10:44:37 +00:00
Andrew Nester 9cb098506a
Added codecov integration on code push (#1095)
Added codecov integration on code push
2024-01-02 09:32:17 +00:00
Pieter Noordhuis 94a9fe4385
No need to fetch repository history when running tests (#1022)
Test runs don't need access to the repository history and only need the
commit being tested.
2023-11-29 13:50:17 +00:00
Pieter Noordhuis b91fab7d09
Run make fmt from fmt job (#929)
## Changes

I noticed we weren't running `goimports` from our formatting job. If we
run `make fmt`, we do.

## Tests

The fmt job passes.
2023-10-27 15:43:44 +00:00
Andrew Nester 4a9dcd3231
Added setup Python action (#789)
## Changes
Added setup Python action

---------

Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
2023-09-21 12:21:39 +00:00
Pieter Noordhuis 2f2386ef5a
Work on GitHub Action (#733)
## Changes

* Run the build workflow on push to main to properly use the build cache

Same as https://github.com/databricks/databricks-sdk-go/pull/601.

## Tests

n/a
2023-09-05 09:58:56 +00:00
Pieter Noordhuis 437263eb58
Upgrade to actions/checkout@v4 (#731)
## Changes

This should fix intermittent failures with v3 (see
https://github.com/actions/checkout/issues/1448)
2023-09-05 08:27:18 +00:00
Pieter Noordhuis cc1038fbd5
Upgrade to actions/setup-go@v4 (#718)
## Changes

Version 4 enables caching by default so we no longer need to explicitly
enable it: https://github.com/actions/setup-go#v4.

The build cache only reuses a cache from a repo's default branch, which
for this repository is `main`. After enabling the merge queue, we no
longer run builds on the `main` branch after push, but on merge queue
branches. With no more builds on the `main` branch there is no longer a
cache to reuse.

This change fixes that by making the `release(-snapshot)?` workflows use
the same caching mechanism. These run off of the `main` branch, so the
cache they save can be reused by builds triggered on PRs or from the
merge queue.

## Tests

We have to merge this to see if it works.
2023-08-30 14:57:34 +00:00
Andrew Nester 6e708da6fc
Upgraded Go version to 1.21 (#664)
## Changes
Upgraded Go version to 1.21

Upgraded to use `slices` and `slog` from core instead of experimental.

Still use `exp/maps` as our code relies on `maps.Keys` which is not part
of core package and therefore refactoring required.

### Tests

Integration tests passed

```
[DEBUG] Test execution command:  /opt/homebrew/opt/go@1.21/bin/go test ./... -json -timeout 1h -run ^TestAcc
[DEBUG] Test execution directory:  /Users/andrew.nester/cli
2023/08/15 13:20:51 [INFO]  TestAccAlertsCreateErrWhenNoArguments (2.150s)
2023/08/15 13:20:52 [INFO]  TestAccApiGet (0.580s)
2023/08/15 13:20:53 [INFO]  TestAccClustersList (0.900s)
2023/08/15 13:20:54 [INFO]  TestAccClustersGet (0.870s)
2023/08/15 13:21:06 [INFO]  TestAccFilerWorkspaceFilesReadWrite (11.980s)
2023/08/15 13:21:13 [INFO]  TestAccFilerWorkspaceFilesReadDir (7.060s)
2023/08/15 13:21:25 [INFO]  TestAccFilerDbfsReadWrite (12.810s)
2023/08/15 13:21:33 [INFO]  TestAccFilerDbfsReadDir (7.380s)
2023/08/15 13:21:41 [INFO]  TestAccFilerWorkspaceNotebookConflict (7.760s)
2023/08/15 13:21:49 [INFO]  TestAccFilerWorkspaceNotebookWithOverwriteFlag (8.660s)
2023/08/15 13:21:49 [INFO]  TestAccFilerLocalReadWrite (0.020s)
2023/08/15 13:21:49 [INFO]  TestAccFilerLocalReadDir (0.010s)
2023/08/15 13:21:52 [INFO]  TestAccFsCatForDbfs (3.190s)
2023/08/15 13:21:53 [INFO]  TestAccFsCatForDbfsOnNonExistentFile (0.890s)
2023/08/15 13:21:54 [INFO]  TestAccFsCatForDbfsInvalidScheme (0.600s)
2023/08/15 13:21:57 [INFO]  TestAccFsCatDoesNotSupportOutputModeJson (2.960s)
2023/08/15 13:22:28 [INFO]  TestAccFsCpDir (31.480s)
2023/08/15 13:22:43 [INFO]  TestAccFsCpFileToFile (14.530s)
2023/08/15 13:22:58 [INFO]  TestAccFsCpFileToDir (14.610s)
2023/08/15 13:23:29 [INFO]  TestAccFsCpDirToDirFileNotOverwritten (31.810s)
2023/08/15 13:23:47 [INFO]  TestAccFsCpFileToDirFileNotOverwritten (17.500s)
2023/08/15 13:24:04 [INFO]  TestAccFsCpFileToFileFileNotOverwritten (17.260s)
2023/08/15 13:24:37 [INFO]  TestAccFsCpDirToDirWithOverwriteFlag (32.690s)
2023/08/15 13:24:56 [INFO]  TestAccFsCpFileToFileWithOverwriteFlag (19.290s)
2023/08/15 13:25:15 [INFO]  TestAccFsCpFileToDirWithOverwriteFlag (19.230s)
2023/08/15 13:25:17 [INFO]  TestAccFsCpErrorsWhenSourceIsDirWithoutRecursiveFlag (2.010s)
2023/08/15 13:25:18 [INFO]  TestAccFsCpErrorsOnInvalidScheme (0.610s)
2023/08/15 13:25:33 [INFO]  TestAccFsCpSourceIsDirectoryButTargetIsFile (14.900s)
2023/08/15 13:25:37 [INFO]  TestAccFsLsForDbfs (3.770s)
2023/08/15 13:25:41 [INFO]  TestAccFsLsForDbfsWithAbsolutePaths (4.160s)
2023/08/15 13:25:44 [INFO]  TestAccFsLsForDbfsOnFile (2.990s)
2023/08/15 13:25:46 [INFO]  TestAccFsLsForDbfsOnEmptyDir (1.870s)
2023/08/15 13:25:46 [INFO]  TestAccFsLsForDbfsForNonexistingDir (0.850s)
2023/08/15 13:25:47 [INFO]  TestAccFsLsWithoutScheme (0.560s)
2023/08/15 13:25:49 [INFO]  TestAccFsMkdirCreatesDirectory (2.310s)
2023/08/15 13:25:52 [INFO]  TestAccFsMkdirCreatesMultipleDirectories (2.920s)
2023/08/15 13:25:55 [INFO]  TestAccFsMkdirWhenDirectoryAlreadyExists (2.320s)
2023/08/15 13:25:57 [INFO]  TestAccFsMkdirWhenFileExistsAtPath (2.820s)
2023/08/15 13:26:01 [INFO]  TestAccFsRmForFile (4.030s)
2023/08/15 13:26:05 [INFO]  TestAccFsRmForEmptyDirectory (3.530s)
2023/08/15 13:26:08 [INFO]  TestAccFsRmForNonEmptyDirectory (3.190s)
2023/08/15 13:26:09 [INFO]  TestAccFsRmForNonExistentFile (0.830s)
2023/08/15 13:26:13 [INFO]  TestAccFsRmForNonEmptyDirectoryWithRecursiveFlag (3.580s)
2023/08/15 13:26:13 [INFO]  TestAccGitClone (0.800s)
2023/08/15 13:26:14 [INFO]  TestAccGitCloneWithOnlyRepoNameOnAlternateBranch (0.790s)
2023/08/15 13:26:15 [INFO]  TestAccGitCloneErrorsWhenRepositoryDoesNotExist (0.540s)
2023/08/15 13:26:23 [INFO]  TestAccLock (8.630s)
2023/08/15 13:26:27 [INFO]  TestAccLockUnlockWithoutAllowsLockFileNotExist (3.490s)
2023/08/15 13:26:30 [INFO]  TestAccLockUnlockWithAllowsLockFileNotExist (3.130s)
2023/08/15 13:26:39 [INFO]  TestAccSyncFullFileSync (9.370s)
2023/08/15 13:26:50 [INFO]  TestAccSyncIncrementalFileSync (10.390s)
2023/08/15 13:27:00 [INFO]  TestAccSyncNestedFolderSync (10.680s)
2023/08/15 13:27:11 [INFO]  TestAccSyncNestedFolderDoesntFailOnNonEmptyDirectory (10.970s)
2023/08/15 13:27:22 [INFO]  TestAccSyncNestedSpacePlusAndHashAreEscapedSync (10.930s)
2023/08/15 13:27:29 [INFO]  TestAccSyncIncrementalFileOverwritesFolder (7.020s)
2023/08/15 13:27:37 [INFO]  TestAccSyncIncrementalSyncPythonNotebookToFile (7.380s)
2023/08/15 13:27:43 [INFO]  TestAccSyncIncrementalSyncFileToPythonNotebook (6.050s)
2023/08/15 13:27:48 [INFO]  TestAccSyncIncrementalSyncPythonNotebookDelete (5.390s)
2023/08/15 13:27:51 [INFO]  TestAccSyncEnsureRemotePathIsUsableIfRepoDoesntExist (2.570s)
2023/08/15 13:27:56 [INFO]  TestAccSyncEnsureRemotePathIsUsableIfRepoExists (5.540s)
2023/08/15 13:27:58 [INFO]  TestAccSyncEnsureRemotePathIsUsableInWorkspace (1.840s)
2023/08/15 13:27:59 [INFO]  TestAccWorkspaceList (0.790s)
2023/08/15 13:28:08 [INFO]  TestAccExportDir (8.860s)
2023/08/15 13:28:11 [INFO]  TestAccExportDirDoesNotOverwrite (3.090s)
2023/08/15 13:28:14 [INFO]  TestAccExportDirWithOverwriteFlag (3.500s)
2023/08/15 13:28:23 [INFO]  TestAccImportDir (8.330s)
2023/08/15 13:28:34 [INFO]  TestAccImportDirDoesNotOverwrite (10.970s)
2023/08/15 13:28:44 [INFO]  TestAccImportDirWithOverwriteFlag (10.130s)
2023/08/15 13:28:44 [INFO]  68/68 passed, 0 failed, 3 skipped
```
2023-08-15 13:50:40 +00:00
Pieter Noordhuis 28bfc0c73d
Remove push to main trigger for build workflow (#621)
## Changes

Commits going through the merge queue are tested there using their final
SHA as if they were already in main. The push-to-main trigger therefore
duplicates the builds that were already triggered from the merge queue.

## Tests

![Screenshot 2023-07-27 at 15 37
17](https://github.com/databricks/cli/assets/9845/ff7af5dd-0d2c-48c2-89b2-7ecf3d121071)
2023-07-27 14:19:40 +00:00
Pieter Noordhuis 8cdf7284f8
Add merge_group trigger for build (#612)
## Changes

Also see [GitHub
docs](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue#triggering-merge-group-checks-with-github-actions).

## Tests

n/a
2023-07-26 12:48:05 +00:00
Andrew Nester cfff140815
Auto detect Python wheel packages and infer build command (#603) 2023-07-26 10:07:26 +00:00
Andrew Nester 5e0a096722
Fixed python wheel test (#608)
## Changes
Fixed python wheel test

## Tests
<!-- How is this tested? -->
2023-07-26 11:02:17 +02:00
Pieter Noordhuis cd789366b9
Bump Go version to 1.19 (#197) 2023-02-03 15:13:29 +01:00
Pieter Noordhuis ce55284398
Run go mod tidy in fmt job (#150) 2022-12-22 09:33:46 +01:00
Pieter Noordhuis d0bd74c116
Run Go formatting with 1.19 (#137)
See https://tip.golang.org/doc/go1.19#go-doc.
2022-12-14 15:59:47 +01:00
Pieter Noordhuis 58e6ba3483
Work on GitHub actions (#133)
This does:
* Use actions/checkout@v3 (fixes node.js v12 deprecation warning)
* Pin Go version to 1.18.8 to make caching work better
* Remove checkout of submodules (we don't have any anymore)
2022-12-12 16:51:27 +01:00
shreyas-goenka 2ebfa5f369
Run unit tests on windows and macos (#103)
Unit tests are now run in all three big OS. 

Some of the changes are to make the tests green for windows while we are
skipping some of the other tests on windows/macOS to make the tests
pass. This is a temporary measure and we will incrementally migrate
these tests over so there is parity in unit testing along all three
environments!
2022-11-28 11:34:25 +01:00
Pieter Noordhuis d214f44df4
Use actions/setup-go@v3 with caching support (#41) 2022-09-08 11:21:00 +02:00
Pieter Noordhuis 2e12a2aa01
Make tests pass (#40)
By:
* Add .gitkeep to retain test fixture directories under
./python/testdata
* Move GitHub related functionality to ./experimental (it is not in use)
* Comment out test in ./cmd/sync
* Fix test in ./git
2022-09-07 20:08:42 +02:00
Pieter Noordhuis 5a55cad7c3
Ensure Go code is formatted (#37) 2022-09-07 15:15:23 +02:00
shreyas-goenka 96efd0e2e4
Replace terraform dependency with go sdk (#19)
Issue: https://github.com/databricks/bricks/issues/17

`./bricks fs ls ...` command works
`./bricks launch ...` command works

Did not test other changes as the readme claims other commands don't
work anyways :) cc: @nfx

TODO left for this PR:
2. Replace terraform scim.Me once its there in go SDK
(https://github.com/databricks/databricks-sdk-go/issues/56)
2022-09-07 11:55:59 +02:00
Serge Smertin 5d8613b21b Also on `main` branch 2022-05-16 13:03:02 +02:00
Serge Smertin bf801c2605 run tests on push or PR 2022-05-16 13:02:12 +02:00