5ba0aaa5c5
Add support for UC Volumes to the `databricks fs` commands ( #1209 )
...
## Changes
```
shreyas.goenka@THW32HFW6T cli % databricks fs -h
Commands to do file system operations on DBFS and UC Volumes.
Usage:
databricks fs [command]
Available Commands:
cat Show file content.
cp Copy files and directories.
ls Lists files.
mkdir Make directories.
rm Remove files and directories.
```
This PR adds support for UC Volumes to the fs commands. The fs commands
for UC volumes work the same as they currently do for DBFS. This is
ensured by running the same test matrix we across both DBFS and UC
Volumes versions of the fs commands.
## Tests
Support for UC volumes is tested by running the same tests as we did
originally for DBFS commands. The tests require a `main` catalog to
exist in the workspace, which does in our test workspaces environments
which have the `TEST_METASTORE_ID` environment variable set.
For the Files API filer, we do the same by running mostly common tests
to ensure the filers for "local", "wsfs", "dbfs" and "files API" are
consistent.
The tests are also made to all run in parallel to reduce the time taken.
To ensure the separation of the tests, each test creates its own UC
schema (for UC volumes tests) or DBFS directories (for DBFS tests).
2024-02-20 16:14:37 +00:00
1752e29885
Update Go SDK to v0.19.0 ( #729 )
...
## Changes
* Update Go SDK to v0.19.0
* Update commands per OpenAPI spec from Go SDK
* Incorporate `client.Do()` signature change to include a (nil) header
map
* Update `workspace.WorkspaceService` mock with permissions methods
* Skip `files` service in codegen; already implemented under the `fs`
command
## Tests
Unit and integration tests pass.
2023-09-05 09:43:57 +00:00
6e708da6fc
Upgraded Go version to 1.21 ( #664 )
...
## Changes
Upgraded Go version to 1.21
Upgraded to use `slices` and `slog` from core instead of experimental.
Still use `exp/maps` as our code relies on `maps.Keys` which is not part
of core package and therefore refactoring required.
### Tests
Integration tests passed
```
[DEBUG] Test execution command: /opt/homebrew/opt/go@1.21/bin/go test ./... -json -timeout 1h -run ^TestAcc
[DEBUG] Test execution directory: /Users/andrew.nester/cli
2023/08/15 13:20:51 [INFO] ✅ TestAccAlertsCreateErrWhenNoArguments (2.150s)
2023/08/15 13:20:52 [INFO] ✅ TestAccApiGet (0.580s)
2023/08/15 13:20:53 [INFO] ✅ TestAccClustersList (0.900s)
2023/08/15 13:20:54 [INFO] ✅ TestAccClustersGet (0.870s)
2023/08/15 13:21:06 [INFO] ✅ TestAccFilerWorkspaceFilesReadWrite (11.980s)
2023/08/15 13:21:13 [INFO] ✅ TestAccFilerWorkspaceFilesReadDir (7.060s)
2023/08/15 13:21:25 [INFO] ✅ TestAccFilerDbfsReadWrite (12.810s)
2023/08/15 13:21:33 [INFO] ✅ TestAccFilerDbfsReadDir (7.380s)
2023/08/15 13:21:41 [INFO] ✅ TestAccFilerWorkspaceNotebookConflict (7.760s)
2023/08/15 13:21:49 [INFO] ✅ TestAccFilerWorkspaceNotebookWithOverwriteFlag (8.660s)
2023/08/15 13:21:49 [INFO] ✅ TestAccFilerLocalReadWrite (0.020s)
2023/08/15 13:21:49 [INFO] ✅ TestAccFilerLocalReadDir (0.010s)
2023/08/15 13:21:52 [INFO] ✅ TestAccFsCatForDbfs (3.190s)
2023/08/15 13:21:53 [INFO] ✅ TestAccFsCatForDbfsOnNonExistentFile (0.890s)
2023/08/15 13:21:54 [INFO] ✅ TestAccFsCatForDbfsInvalidScheme (0.600s)
2023/08/15 13:21:57 [INFO] ✅ TestAccFsCatDoesNotSupportOutputModeJson (2.960s)
2023/08/15 13:22:28 [INFO] ✅ TestAccFsCpDir (31.480s)
2023/08/15 13:22:43 [INFO] ✅ TestAccFsCpFileToFile (14.530s)
2023/08/15 13:22:58 [INFO] ✅ TestAccFsCpFileToDir (14.610s)
2023/08/15 13:23:29 [INFO] ✅ TestAccFsCpDirToDirFileNotOverwritten (31.810s)
2023/08/15 13:23:47 [INFO] ✅ TestAccFsCpFileToDirFileNotOverwritten (17.500s)
2023/08/15 13:24:04 [INFO] ✅ TestAccFsCpFileToFileFileNotOverwritten (17.260s)
2023/08/15 13:24:37 [INFO] ✅ TestAccFsCpDirToDirWithOverwriteFlag (32.690s)
2023/08/15 13:24:56 [INFO] ✅ TestAccFsCpFileToFileWithOverwriteFlag (19.290s)
2023/08/15 13:25:15 [INFO] ✅ TestAccFsCpFileToDirWithOverwriteFlag (19.230s)
2023/08/15 13:25:17 [INFO] ✅ TestAccFsCpErrorsWhenSourceIsDirWithoutRecursiveFlag (2.010s)
2023/08/15 13:25:18 [INFO] ✅ TestAccFsCpErrorsOnInvalidScheme (0.610s)
2023/08/15 13:25:33 [INFO] ✅ TestAccFsCpSourceIsDirectoryButTargetIsFile (14.900s)
2023/08/15 13:25:37 [INFO] ✅ TestAccFsLsForDbfs (3.770s)
2023/08/15 13:25:41 [INFO] ✅ TestAccFsLsForDbfsWithAbsolutePaths (4.160s)
2023/08/15 13:25:44 [INFO] ✅ TestAccFsLsForDbfsOnFile (2.990s)
2023/08/15 13:25:46 [INFO] ✅ TestAccFsLsForDbfsOnEmptyDir (1.870s)
2023/08/15 13:25:46 [INFO] ✅ TestAccFsLsForDbfsForNonexistingDir (0.850s)
2023/08/15 13:25:47 [INFO] ✅ TestAccFsLsWithoutScheme (0.560s)
2023/08/15 13:25:49 [INFO] ✅ TestAccFsMkdirCreatesDirectory (2.310s)
2023/08/15 13:25:52 [INFO] ✅ TestAccFsMkdirCreatesMultipleDirectories (2.920s)
2023/08/15 13:25:55 [INFO] ✅ TestAccFsMkdirWhenDirectoryAlreadyExists (2.320s)
2023/08/15 13:25:57 [INFO] ✅ TestAccFsMkdirWhenFileExistsAtPath (2.820s)
2023/08/15 13:26:01 [INFO] ✅ TestAccFsRmForFile (4.030s)
2023/08/15 13:26:05 [INFO] ✅ TestAccFsRmForEmptyDirectory (3.530s)
2023/08/15 13:26:08 [INFO] ✅ TestAccFsRmForNonEmptyDirectory (3.190s)
2023/08/15 13:26:09 [INFO] ✅ TestAccFsRmForNonExistentFile (0.830s)
2023/08/15 13:26:13 [INFO] ✅ TestAccFsRmForNonEmptyDirectoryWithRecursiveFlag (3.580s)
2023/08/15 13:26:13 [INFO] ✅ TestAccGitClone (0.800s)
2023/08/15 13:26:14 [INFO] ✅ TestAccGitCloneWithOnlyRepoNameOnAlternateBranch (0.790s)
2023/08/15 13:26:15 [INFO] ✅ TestAccGitCloneErrorsWhenRepositoryDoesNotExist (0.540s)
2023/08/15 13:26:23 [INFO] ✅ TestAccLock (8.630s)
2023/08/15 13:26:27 [INFO] ✅ TestAccLockUnlockWithoutAllowsLockFileNotExist (3.490s)
2023/08/15 13:26:30 [INFO] ✅ TestAccLockUnlockWithAllowsLockFileNotExist (3.130s)
2023/08/15 13:26:39 [INFO] ✅ TestAccSyncFullFileSync (9.370s)
2023/08/15 13:26:50 [INFO] ✅ TestAccSyncIncrementalFileSync (10.390s)
2023/08/15 13:27:00 [INFO] ✅ TestAccSyncNestedFolderSync (10.680s)
2023/08/15 13:27:11 [INFO] ✅ TestAccSyncNestedFolderDoesntFailOnNonEmptyDirectory (10.970s)
2023/08/15 13:27:22 [INFO] ✅ TestAccSyncNestedSpacePlusAndHashAreEscapedSync (10.930s)
2023/08/15 13:27:29 [INFO] ✅ TestAccSyncIncrementalFileOverwritesFolder (7.020s)
2023/08/15 13:27:37 [INFO] ✅ TestAccSyncIncrementalSyncPythonNotebookToFile (7.380s)
2023/08/15 13:27:43 [INFO] ✅ TestAccSyncIncrementalSyncFileToPythonNotebook (6.050s)
2023/08/15 13:27:48 [INFO] ✅ TestAccSyncIncrementalSyncPythonNotebookDelete (5.390s)
2023/08/15 13:27:51 [INFO] ✅ TestAccSyncEnsureRemotePathIsUsableIfRepoDoesntExist (2.570s)
2023/08/15 13:27:56 [INFO] ✅ TestAccSyncEnsureRemotePathIsUsableIfRepoExists (5.540s)
2023/08/15 13:27:58 [INFO] ✅ TestAccSyncEnsureRemotePathIsUsableInWorkspace (1.840s)
2023/08/15 13:27:59 [INFO] ✅ TestAccWorkspaceList (0.790s)
2023/08/15 13:28:08 [INFO] ✅ TestAccExportDir (8.860s)
2023/08/15 13:28:11 [INFO] ✅ TestAccExportDirDoesNotOverwrite (3.090s)
2023/08/15 13:28:14 [INFO] ✅ TestAccExportDirWithOverwriteFlag (3.500s)
2023/08/15 13:28:23 [INFO] ✅ TestAccImportDir (8.330s)
2023/08/15 13:28:34 [INFO] ✅ TestAccImportDirDoesNotOverwrite (10.970s)
2023/08/15 13:28:44 [INFO] ✅ TestAccImportDirWithOverwriteFlag (10.130s)
2023/08/15 13:28:44 [INFO] ✅ 68/68 passed, 0 failed, 3 skipped
```
2023-08-15 13:50:40 +00:00
30efe91c6d
Make local files default for fs commands ( #506 )
...
## Changes
<!-- Summary of your changes that are easy to understand -->
## Tests
<!-- How is this tested? -->
2023-06-23 16:07:09 +02:00
e19eaca4d1
Add filer.Filer implementation backed by the Files API ( #474 )
...
## Tests
New integration test for the read/write parts of the other filers. The
integration test cannot be shared just yet because the Files API doesn't
include support for creating/listing/removing directories yet.
2023-06-19 18:29:13 +00:00
shreyas-goenka
Pieter Noordhuis
Andrew Nester