databricks-cli/.github/workflows/push.yml

138 lines
3.5 KiB
YAML
Raw Normal View History

2022-05-16 11:02:12 +00:00
name: build
on:
pull_request:
types: [opened, synchronize]
merge_group:
types: [checks_requested]
push:
# Always run on push to main. The build cache can only be reused
# if it was saved by a run from the repository's default branch.
# The run result will be identical to that from the merge queue
# because the commit is identical, yet we need to perform it to
# seed the build cache.
branches:
- main
2022-05-16 11:02:12 +00:00
jobs:
tests:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os:
- macos-latest
- ubuntu-latest
- windows-latest
2022-05-16 11:02:12 +00:00
steps:
- name: Checkout repository and submodules
uses: actions/checkout@v4
2022-05-16 11:02:12 +00:00
- name: Setup Go
uses: actions/setup-go@v5
2022-05-16 11:02:12 +00:00
with:
go-version: 1.23.2
2022-05-16 11:02:12 +00:00
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.9'
2022-05-16 11:02:12 +00:00
- name: Set go env
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
go install gotest.tools/gotestsum@v1.12.0
2022-05-16 11:02:12 +00:00
- name: Pull external libraries
run: |
make vendor
pip3 install wheel
2022-05-16 11:02:12 +00:00
- name: Run tests
2024-12-04 17:40:19 +00:00
run: make testonly
2022-09-07 13:15:23 +00:00
- name: Publish test coverage
uses: codecov/codecov-action@v4
2022-09-07 13:15:23 +00:00
fmt:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
2022-09-07 13:15:23 +00:00
- name: Setup Go
uses: actions/setup-go@v5
2022-09-07 13:15:23 +00:00
with:
go-version: 1.23.2
2022-09-07 13:15:23 +00:00
# No need to download cached dependencies when running gofmt.
cache: false
- name: Install goimports
2022-09-07 13:15:23 +00:00
run: |
go install golang.org/x/tools/cmd/goimports@latest
- name: Run make fmt
run: |
make fmt
2022-09-07 13:15:23 +00:00
2022-12-22 08:33:46 +00:00
- name: Run go mod tidy
run: |
go mod tidy
2022-09-07 13:15:23 +00:00
- name: Fail on differences
run: |
# Exit with status code 1 if there are differences (i.e. unformatted files)
git diff --exit-code
Fix the generated DABs JSON schema (#1322) ## Changes This PR fixes bundle schema being broken because `for_each_task: null` was set in the generated schema. This is not valid according to the JSON schema specification and thus the Red Hat YAML VSCode extension was failing to parse the YAML configuration. This PR fixes: https://github.com/databricks/cli/issues/1312 ## Tests The fix itself was tested manually. I asserted that the autocompletion works now. This was mistakenly overlooked the first time around when the regression was introduced in https://github.com/databricks/cli/pull/1204 because the YAML extension provides best-effort autocomplete suggestions even if the JSON schema fails to load. To prevent future regressions we also add a test to assert that the JSON schema generated itself is a valid JSON schema object. This is done via using the `ajv-cli` to validate the schema. This package is also used by the Red Hat YAML extension and thus provides a high fidelity check for ensuring the JSON schema is valid. Before, with the old schema: ``` shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-216.json -d ../bundle-playground-3/databricks.yml schema proj/schema-216.json is invalid error: schema is invalid: data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items/properties/for_each_task must be object,boolean, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must be array, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must match a schema in anyOf ``` After, with the new schema: ``` shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-dev.json -d ../bundle-playground-3/databricks.yml ../bundle-playground-3/databricks.yml valid ``` After, autocomplete suggestions: <img width="600" alt="Screenshot 2024-03-27 at 6 35 57 PM" src="https://github.com/databricks/cli/assets/88374338/d0a62402-e323-4f36-854d-332b33cbeab8">
2024-03-28 11:25:36 +00:00
2024-12-04 17:40:19 +00:00
golangci:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: 1.23.2
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v1.62.2
args: --timeout=15m
Fix the generated DABs JSON schema (#1322) ## Changes This PR fixes bundle schema being broken because `for_each_task: null` was set in the generated schema. This is not valid according to the JSON schema specification and thus the Red Hat YAML VSCode extension was failing to parse the YAML configuration. This PR fixes: https://github.com/databricks/cli/issues/1312 ## Tests The fix itself was tested manually. I asserted that the autocompletion works now. This was mistakenly overlooked the first time around when the regression was introduced in https://github.com/databricks/cli/pull/1204 because the YAML extension provides best-effort autocomplete suggestions even if the JSON schema fails to load. To prevent future regressions we also add a test to assert that the JSON schema generated itself is a valid JSON schema object. This is done via using the `ajv-cli` to validate the schema. This package is also used by the Red Hat YAML extension and thus provides a high fidelity check for ensuring the JSON schema is valid. Before, with the old schema: ``` shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-216.json -d ../bundle-playground-3/databricks.yml schema proj/schema-216.json is invalid error: schema is invalid: data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items/properties/for_each_task must be object,boolean, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must be array, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must match a schema in anyOf ``` After, with the new schema: ``` shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-dev.json -d ../bundle-playground-3/databricks.yml ../bundle-playground-3/databricks.yml valid ``` After, autocomplete suggestions: <img width="600" alt="Screenshot 2024-03-27 at 6 35 57 PM" src="https://github.com/databricks/cli/assets/88374338/d0a62402-e323-4f36-854d-332b33cbeab8">
2024-03-28 11:25:36 +00:00
validate-bundle-schema:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: 1.23.2
Fix the generated DABs JSON schema (#1322) ## Changes This PR fixes bundle schema being broken because `for_each_task: null` was set in the generated schema. This is not valid according to the JSON schema specification and thus the Red Hat YAML VSCode extension was failing to parse the YAML configuration. This PR fixes: https://github.com/databricks/cli/issues/1312 ## Tests The fix itself was tested manually. I asserted that the autocompletion works now. This was mistakenly overlooked the first time around when the regression was introduced in https://github.com/databricks/cli/pull/1204 because the YAML extension provides best-effort autocomplete suggestions even if the JSON schema fails to load. To prevent future regressions we also add a test to assert that the JSON schema generated itself is a valid JSON schema object. This is done via using the `ajv-cli` to validate the schema. This package is also used by the Red Hat YAML extension and thus provides a high fidelity check for ensuring the JSON schema is valid. Before, with the old schema: ``` shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-216.json -d ../bundle-playground-3/databricks.yml schema proj/schema-216.json is invalid error: schema is invalid: data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items/properties/for_each_task must be object,boolean, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must be array, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must match a schema in anyOf ``` After, with the new schema: ``` shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-dev.json -d ../bundle-playground-3/databricks.yml ../bundle-playground-3/databricks.yml valid ``` After, autocomplete suggestions: <img width="600" alt="Screenshot 2024-03-27 at 6 35 57 PM" src="https://github.com/databricks/cli/assets/88374338/d0a62402-e323-4f36-854d-332b33cbeab8">
2024-03-28 11:25:36 +00:00
# Github repo: https://github.com/ajv-validator/ajv-cli
- name: Install ajv-cli
run: npm install -g ajv-cli@5.0.0
# Assert that the generated bundle schema is a valid JSON schema by using
# ajv-cli to validate it against bundle configuration files.
Fix the generated DABs JSON schema (#1322) ## Changes This PR fixes bundle schema being broken because `for_each_task: null` was set in the generated schema. This is not valid according to the JSON schema specification and thus the Red Hat YAML VSCode extension was failing to parse the YAML configuration. This PR fixes: https://github.com/databricks/cli/issues/1312 ## Tests The fix itself was tested manually. I asserted that the autocompletion works now. This was mistakenly overlooked the first time around when the regression was introduced in https://github.com/databricks/cli/pull/1204 because the YAML extension provides best-effort autocomplete suggestions even if the JSON schema fails to load. To prevent future regressions we also add a test to assert that the JSON schema generated itself is a valid JSON schema object. This is done via using the `ajv-cli` to validate the schema. This package is also used by the Red Hat YAML extension and thus provides a high fidelity check for ensuring the JSON schema is valid. Before, with the old schema: ``` shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-216.json -d ../bundle-playground-3/databricks.yml schema proj/schema-216.json is invalid error: schema is invalid: data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items/properties/for_each_task must be object,boolean, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must be array, data/properties/resources/properties/jobs/additionalProperties/properties/tasks/items must match a schema in anyOf ``` After, with the new schema: ``` shreyas.goenka@THW32HFW6T cli-versions % ajv validate -s proj/schema-dev.json -d ../bundle-playground-3/databricks.yml ../bundle-playground-3/databricks.yml valid ``` After, autocomplete suggestions: <img width="600" alt="Screenshot 2024-03-27 at 6 35 57 PM" src="https://github.com/databricks/cli/assets/88374338/d0a62402-e323-4f36-854d-332b33cbeab8">
2024-03-28 11:25:36 +00:00
# By default the ajv-cli runs in strict mode which will fail if the schema
# itself is not valid. Strict mode is more strict than the JSON schema
# specification. See for details: https://ajv.js.org/options.html#strict-mode-options
- name: Validate bundle schema
run: |
go run main.go bundle schema > schema.json
for file in ./bundle/internal/schema/testdata/pass/*.yml; do
ajv test -s schema.json -d $file --valid
done
for file in ./bundle/internal/schema/testdata/fail/*.yml; do
ajv test -s schema.json -d $file --invalid
done