From ddd45e25ee24cfad9ec5834ed66b71bb278b168d Mon Sep 17 00:00:00 2001 From: Denis Bilenko Date: Thu, 23 Jan 2025 13:48:47 +0100 Subject: [PATCH] Pass USE_SDK_V2_{RESOURCES,DATA_SOURCES} to terraform (#2207) ## Changes - Propagate env vars USE_SDK_V2_RESOURCES and $USE_SDK_V2_DATA_SOURCES to terraform - This are troubleshooting helpers for resources migrated to new plugin framework, recommended here: https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/troubleshooting#plugin-framework-migration-problems - This current unblocks deploying quality monitors, see https://github.com/databricks/terraform-provider-databricks/issues/4229#issuecomment-2520344690 ## Tests Manually testing that I can deploy quality monitor after this change with `USE_SDK_V2_RESOURCES="databricks_quality_monitor"` set ### Main branch: ``` ~/work/databricks_quality_monitor_repro % USE_SDK_V2_RESOURCES="databricks_quality_monitor" ../cli/cli-main bundle deploy Uploading bundle files to /Workspace/Users/denis.bilenko@databricks.com/.bundle/quality_monitor_bundle/default/files... Deploying resources... Updating deployment state... Deployment complete! Error: terraform apply: exit status 1 Error: Provider produced inconsistent result after apply When applying changes to databricks_quality_monitor.monitor_trips, provider "provider[\"registry.terraform.io/databricks/databricks\"]" produced an unexpected new value: .data_classification_config: block count changed from 0 to 1. This is a bug in the provider, which should be reported in the provider's own issue tracker. ``` ### This branch: ``` ~/work/databricks_quality_monitor_repro % USE_SDK_V2_RESOURCES="databricks_quality_monitor" ../cli/cli bundle deploy Uploading bundle files to /Workspace/Users/denis.bilenko@databricks.com/.bundle/quality_monitor_bundle/default/files... Deploying resources... Updating deployment state... Deployment complete! ``` ### Config: ``` ~/work/databricks_quality_monitor_repro % cat databricks.yml bundle: name: quality_monitor_bundle resources: quality_monitors: monitor_trips: table_name: main.denis-bilenko-cuj-pe34.trips_sanitized_1 output_schema_name: main.denis-bilenko-cuj-pe34 assets_dir: /Workspace/Users/${workspace.current_user.userName}/quality_monitor_issue snapshot: {} ``` --- bundle/deploy/terraform/init.go | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/bundle/deploy/terraform/init.go b/bundle/deploy/terraform/init.go index 6a014a7c1..5957611a4 100644 --- a/bundle/deploy/terraform/init.go +++ b/bundle/deploy/terraform/init.go @@ -108,6 +108,14 @@ var envCopy = []string{ // Include $TF_CLI_CONFIG_FILE to override terraform provider in development. // See: https://developer.hashicorp.com/terraform/cli/config/config-file#explicit-installation-method-configuration "TF_CLI_CONFIG_FILE", + + // Include $USE_SDK_V2_RESOURCES and $USE_SDK_V2_DATA_SOURCES, these are used to switch back from plugin framework to SDKv2. + // This is used for mitigation issues with resource migrated to plugin framework, as recommended here: + // https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/troubleshooting#plugin-framework-migration-problems + // It is currently a workaround for deploying quality_monitors + // https://github.com/databricks/terraform-provider-databricks/issues/4229#issuecomment-2520344690 + "USE_SDK_V2_RESOURCES", + "USE_SDK_V2_DATA_SOURCES", } // This function inherits some environment variables for Terraform CLI.