2022-12-12 11:49:25 +00:00
|
|
|
package phases
|
|
|
|
|
|
|
|
|
|
import (
|
Remove bundle.{Seq,If,Defer,newPhase,logString}, switch to regular functions (#2390)
## Changes
- Instead of constructing chains of mutators and then executing them,
execute them directly.
- Remove functionality related to chain-building: Seq, If, Defer,
newPhase, logString.
- Phases become functions that apply the changes directly rather than
construct mutator chains that will be called later.
- Add a helper ApplySeq to call multiple mutators, use it where
Apply+Seq were used before.
This is intended to be a refactoring without functional changes, but
there are a few behaviour changes:
- Since defer() is used to call unlock instead of bundle.Defer()
unlocking will now happen even in case of panics.
- In --debug, the phase names are are still logged once before start of
the phase but each entry no longer has 'seq' or phase name in it.
- The message "Deployment complete!" was printed even if
terraform.Apply() mutator had an error. It no longer does that.
## Motivation
The use of the chains was necessary when mutators were returning a list
of other mutators instead of calling them directly. But that has since
been removed, so now the chain machinery have no purpose anymore.
Use of direct functions simplifies the logic and makes bugs more
apparent and easy to fix.
Other improvements that this unlocks:
- Simpler stacktraces/debugging (breakpoints).
- Use of functions with narrowly scoped API: instead of mutators that
receive full bundle config, we can use focused functions that only deal
with sections they care about prepareGitSettings(currentGitSection) ->
updatedGitSection. This makes the data flow more apparent.
- Parallel computations across mutators (within phase): launch
goroutines fetching data from APIs at the beggining, process them once
they are ready.
## Tests
Existing tests.
2025-02-27 11:41:58 +00:00
|
|
|
"context"
|
|
|
|
|
|
2023-05-16 16:35:39 +00:00
|
|
|
"github.com/databricks/cli/bundle"
|
2025-01-13 16:43:48 +00:00
|
|
|
"github.com/databricks/cli/bundle/apps"
|
2023-09-14 10:14:13 +00:00
|
|
|
"github.com/databricks/cli/bundle/config"
|
2023-05-16 16:35:39 +00:00
|
|
|
"github.com/databricks/cli/bundle/config/mutator"
|
2024-06-20 08:43:08 +00:00
|
|
|
pythonmutator "github.com/databricks/cli/bundle/config/mutator/python"
|
2024-08-13 12:50:15 +00:00
|
|
|
"github.com/databricks/cli/bundle/config/validate"
|
2023-12-19 07:38:52 +00:00
|
|
|
"github.com/databricks/cli/bundle/deploy/metadata"
|
2023-05-16 16:35:39 +00:00
|
|
|
"github.com/databricks/cli/bundle/deploy/terraform"
|
2023-11-13 11:29:40 +00:00
|
|
|
"github.com/databricks/cli/bundle/permissions"
|
2023-09-14 10:14:13 +00:00
|
|
|
"github.com/databricks/cli/bundle/scripts"
|
2024-09-27 09:32:54 +00:00
|
|
|
"github.com/databricks/cli/bundle/trampoline"
|
Remove bundle.{Seq,If,Defer,newPhase,logString}, switch to regular functions (#2390)
## Changes
- Instead of constructing chains of mutators and then executing them,
execute them directly.
- Remove functionality related to chain-building: Seq, If, Defer,
newPhase, logString.
- Phases become functions that apply the changes directly rather than
construct mutator chains that will be called later.
- Add a helper ApplySeq to call multiple mutators, use it where
Apply+Seq were used before.
This is intended to be a refactoring without functional changes, but
there are a few behaviour changes:
- Since defer() is used to call unlock instead of bundle.Defer()
unlocking will now happen even in case of panics.
- In --debug, the phase names are are still logged once before start of
the phase but each entry no longer has 'seq' or phase name in it.
- The message "Deployment complete!" was printed even if
terraform.Apply() mutator had an error. It no longer does that.
## Motivation
The use of the chains was necessary when mutators were returning a list
of other mutators instead of calling them directly. But that has since
been removed, so now the chain machinery have no purpose anymore.
Use of direct functions simplifies the logic and makes bugs more
apparent and easy to fix.
Other improvements that this unlocks:
- Simpler stacktraces/debugging (breakpoints).
- Use of functions with narrowly scoped API: instead of mutators that
receive full bundle config, we can use focused functions that only deal
with sections they care about prepareGitSettings(currentGitSection) ->
updatedGitSection. This makes the data flow more apparent.
- Parallel computations across mutators (within phase): launch
goroutines fetching data from APIs at the beggining, process them once
they are ready.
## Tests
Existing tests.
2025-02-27 11:41:58 +00:00
|
|
|
"github.com/databricks/cli/libs/diag"
|
|
|
|
|
"github.com/databricks/cli/libs/log"
|
2022-12-12 11:49:25 +00:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
// The initialize phase fills in defaults and connects to the workspace.
|
|
|
|
|
// Interpolation of fields referring to the "bundle" and "workspace" keys
|
|
|
|
|
// happens upon completion of this phase.
|
Remove bundle.{Seq,If,Defer,newPhase,logString}, switch to regular functions (#2390)
## Changes
- Instead of constructing chains of mutators and then executing them,
execute them directly.
- Remove functionality related to chain-building: Seq, If, Defer,
newPhase, logString.
- Phases become functions that apply the changes directly rather than
construct mutator chains that will be called later.
- Add a helper ApplySeq to call multiple mutators, use it where
Apply+Seq were used before.
This is intended to be a refactoring without functional changes, but
there are a few behaviour changes:
- Since defer() is used to call unlock instead of bundle.Defer()
unlocking will now happen even in case of panics.
- In --debug, the phase names are are still logged once before start of
the phase but each entry no longer has 'seq' or phase name in it.
- The message "Deployment complete!" was printed even if
terraform.Apply() mutator had an error. It no longer does that.
## Motivation
The use of the chains was necessary when mutators were returning a list
of other mutators instead of calling them directly. But that has since
been removed, so now the chain machinery have no purpose anymore.
Use of direct functions simplifies the logic and makes bugs more
apparent and easy to fix.
Other improvements that this unlocks:
- Simpler stacktraces/debugging (breakpoints).
- Use of functions with narrowly scoped API: instead of mutators that
receive full bundle config, we can use focused functions that only deal
with sections they care about prepareGitSettings(currentGitSection) ->
updatedGitSection. This makes the data flow more apparent.
- Parallel computations across mutators (within phase): launch
goroutines fetching data from APIs at the beggining, process them once
they are ready.
## Tests
Existing tests.
2025-02-27 11:41:58 +00:00
|
|
|
func Initialize(ctx context.Context, b *bundle.Bundle) diag.Diagnostics {
|
|
|
|
|
log.Info(ctx, "Phase: initialize")
|
|
|
|
|
|
|
|
|
|
return bundle.ApplySeq(ctx, b,
|
|
|
|
|
validate.AllResourcesHaveValues(),
|
2025-02-27 15:58:48 +00:00
|
|
|
validate.NoInterpolationInAuthConfig(),
|
Remove bundle.{Seq,If,Defer,newPhase,logString}, switch to regular functions (#2390)
## Changes
- Instead of constructing chains of mutators and then executing them,
execute them directly.
- Remove functionality related to chain-building: Seq, If, Defer,
newPhase, logString.
- Phases become functions that apply the changes directly rather than
construct mutator chains that will be called later.
- Add a helper ApplySeq to call multiple mutators, use it where
Apply+Seq were used before.
This is intended to be a refactoring without functional changes, but
there are a few behaviour changes:
- Since defer() is used to call unlock instead of bundle.Defer()
unlocking will now happen even in case of panics.
- In --debug, the phase names are are still logged once before start of
the phase but each entry no longer has 'seq' or phase name in it.
- The message "Deployment complete!" was printed even if
terraform.Apply() mutator had an error. It no longer does that.
## Motivation
The use of the chains was necessary when mutators were returning a list
of other mutators instead of calling them directly. But that has since
been removed, so now the chain machinery have no purpose anymore.
Use of direct functions simplifies the logic and makes bugs more
apparent and easy to fix.
Other improvements that this unlocks:
- Simpler stacktraces/debugging (breakpoints).
- Use of functions with narrowly scoped API: instead of mutators that
receive full bundle config, we can use focused functions that only deal
with sections they care about prepareGitSettings(currentGitSection) ->
updatedGitSection. This makes the data flow more apparent.
- Parallel computations across mutators (within phase): launch
goroutines fetching data from APIs at the beggining, process them once
they are ready.
## Tests
Existing tests.
2025-02-27 11:41:58 +00:00
|
|
|
|
|
|
|
|
// Update all path fields in the sync block to be relative to the bundle root path.
|
|
|
|
|
mutator.RewriteSyncPaths(),
|
|
|
|
|
|
|
|
|
|
// Configure the default sync path to equal the bundle root if not explicitly configured.
|
|
|
|
|
// By default, this means all files in the bundle root directory are synchronized.
|
|
|
|
|
mutator.SyncDefaultPath(),
|
|
|
|
|
|
|
|
|
|
// Figure out if the sync root path is identical or an ancestor of the bundle root path.
|
|
|
|
|
// If it is an ancestor, this updates all paths to be relative to the sync root path.
|
|
|
|
|
mutator.SyncInferRoot(),
|
|
|
|
|
|
|
|
|
|
mutator.PopulateCurrentUser(),
|
|
|
|
|
mutator.LoadGitDetails(),
|
|
|
|
|
|
|
|
|
|
// This mutator needs to be run before variable interpolation and defining default workspace paths
|
|
|
|
|
// because it affects how workspace variables are resolved.
|
|
|
|
|
mutator.ApplySourceLinkedDeploymentPreset(),
|
|
|
|
|
|
|
|
|
|
mutator.DefineDefaultWorkspaceRoot(),
|
|
|
|
|
mutator.ExpandWorkspaceRoot(),
|
|
|
|
|
mutator.DefineDefaultWorkspacePaths(),
|
|
|
|
|
mutator.PrependWorkspacePrefix(),
|
|
|
|
|
|
|
|
|
|
// This mutator needs to be run before variable interpolation because it
|
|
|
|
|
// searches for strings with variable references in them.
|
|
|
|
|
mutator.RewriteWorkspacePrefix(),
|
|
|
|
|
|
|
|
|
|
mutator.SetVariables(),
|
|
|
|
|
|
|
|
|
|
// Intentionally placed before ResolveVariableReferencesInLookup, ResolveResourceReferences,
|
|
|
|
|
// ResolveVariableReferencesInComplexVariables and ResolveVariableReferences.
|
|
|
|
|
// See what is expected in PythonMutatorPhaseInit doc
|
|
|
|
|
pythonmutator.PythonMutator(pythonmutator.PythonMutatorPhaseInit),
|
|
|
|
|
pythonmutator.PythonMutator(pythonmutator.PythonMutatorPhaseLoadResources),
|
|
|
|
|
pythonmutator.PythonMutator(pythonmutator.PythonMutatorPhaseApplyMutators),
|
|
|
|
|
mutator.ResolveVariableReferencesInLookup(),
|
|
|
|
|
mutator.ResolveResourceReferences(),
|
|
|
|
|
mutator.ResolveVariableReferences(
|
|
|
|
|
"bundle",
|
|
|
|
|
"workspace",
|
|
|
|
|
"variables",
|
|
|
|
|
),
|
|
|
|
|
|
|
|
|
|
mutator.MergeJobClusters(),
|
|
|
|
|
mutator.MergeJobParameters(),
|
|
|
|
|
mutator.MergeJobTasks(),
|
|
|
|
|
mutator.MergePipelineClusters(),
|
|
|
|
|
mutator.MergeApps(),
|
|
|
|
|
|
|
|
|
|
mutator.CaptureSchemaDependency(),
|
|
|
|
|
|
|
|
|
|
// Provide permission config errors & warnings after initializing all variables
|
|
|
|
|
permissions.PermissionDiagnostics(),
|
|
|
|
|
mutator.SetRunAs(),
|
|
|
|
|
mutator.OverrideCompute(),
|
|
|
|
|
mutator.ConfigureDashboardDefaults(),
|
|
|
|
|
mutator.ConfigureVolumeDefaults(),
|
|
|
|
|
mutator.ProcessTargetMode(),
|
|
|
|
|
mutator.ApplyPresets(),
|
|
|
|
|
mutator.DefaultQueueing(),
|
|
|
|
|
mutator.ExpandPipelineGlobPaths(),
|
|
|
|
|
|
|
|
|
|
// Configure use of WSFS for reads if the CLI is running on Databricks.
|
|
|
|
|
mutator.ConfigureWSFS(),
|
|
|
|
|
|
|
|
|
|
mutator.TranslatePaths(),
|
|
|
|
|
trampoline.WrapperWarning(),
|
|
|
|
|
|
|
|
|
|
apps.Validate(),
|
|
|
|
|
|
|
|
|
|
permissions.ValidateSharedRootPermissions(),
|
|
|
|
|
permissions.ApplyBundlePermissions(),
|
|
|
|
|
permissions.FilterCurrentUser(),
|
|
|
|
|
|
|
|
|
|
metadata.AnnotateJobs(),
|
|
|
|
|
metadata.AnnotatePipelines(),
|
|
|
|
|
terraform.Initialize(),
|
|
|
|
|
scripts.Execute(config.ScriptPostInit),
|
2022-12-12 11:49:25 +00:00
|
|
|
)
|
|
|
|
|
}
|
