Skip to content

Replace regex-based interpolation with character scanner#4747

Draft
shreyas-goenka wants to merge 2 commits intomainfrom
interpolation-parser-pr1
Draft

Replace regex-based interpolation with character scanner#4747
shreyas-goenka wants to merge 2 commits intomainfrom
interpolation-parser-pr1

Conversation

@shreyas-goenka
Copy link
Contributor

@shreyas-goenka shreyas-goenka commented Mar 15, 2026

Summary

Split from #4715. This PR replaces the regex-based variable reference parser with a two-mode character scanner, addressing review feedback:

  • Move interpolation parser to libs/interpolation/ (out of libs/dyn/, per review)
  • Change escape mechanism from $$ to \$$ and \\\ (Bash convention, per core eng discussion)
  • Reject nested ${var.foo_${var.tail}} as error instead of silently handling (per review)
  • Replace strings.Replace interpolation with token-based concatenation
  • Add WarnMalformedReferences mutator for early parse error warnings
  • Add NewRefWithDiagnostics for surfacing parse errors as diagnostics
  • Flatten escape tests into table-driven tests (per review)
  • Update var_in_var test which relied on undocumented nested ref behavior (test itself noted: "not officially supported, might reject in the future")

Test plan

  • go test ./libs/interpolation/... — parser unit tests
  • go test ./libs/dyn/dynvar/... — ref, resolve tests
  • go test ./bundle/config/mutator/... — mutator tests
  • go test ./acceptance -run TestAccept/bundle/variables/malformed_reference
  • go test ./acceptance -run TestAccept/bundle/resource_deps/bad_syntax
  • go test ./acceptance -run TestAccept/bundle/variables/var_in_var
  • Full unit test suite passes
  • make fmtfull && make lintfull clean

This pull request was AI-assisted by Isaac.

- Move interpolation parser to libs/interpolation/ (independent of dyn)
- Use \$ -> $ and \\ -> \ escape sequences (Bash convention)
- Reject nested ${} constructs as errors
- Replace strings.Replace interpolation with token-based concatenation
- Add WarnMalformedReferences mutator for early parse error warnings
- Add NewRefWithDiagnostics for surfacing parse errors as diagnostics

Co-authored-by: Isaac
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Mar 15, 2026

Commit: 523e287

Run: 23122304388

Env 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 8 7 268 788 6:11
🔄​ aws windows 2 6 7 270 786 5:39
🔄​ aws-ucws linux 2 7 7 364 703 6:24
🔄​ aws-ucws windows 2 7 7 366 701 5:51
💚​ azure linux 2 9 271 786 5:57
💚​ azure windows 2 9 273 784 4:29
🔄​ azure-ucws linux 2 1 9 369 699 6:37
🔄​ azure-ucws windows 2 1 9 371 697 6:30
💚​ gcp linux 2 9 267 789 6:04
💚​ gcp windows 2 9 269 787 5:13
16 interesting tests: 7 SKIP, 6 RECOVERED, 3 flaky
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🔄​ TestAccept 💚​R 🔄​f 🔄​f 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/ssh/connect-serverless-gpu 🙈​s 🙈​s 🔄​f 🔄​f 🙈​s 🙈​s 🔄​f 🔄​f 🙈​s 🙈​s
🔄​ TestAccept/ssh/connection 💚​R 🔄​f 💚​R 🔄​f 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R
Top 20 slowest tests (at least 2 minutes):
duration env testname
3:38 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:36 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:12 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:10 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:09 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:09 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:05 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:04 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:54 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:51 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:47 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:44 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:39 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:38 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:34 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:12 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:09 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:08 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:06 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:05 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

The var_in_var test used ${var.foo_${var.tail}} which relied on
undocumented nested reference behavior (the test itself noted:
"not officially supported... might start to reject this in the future").
The new parser now rejects nested ${} constructs as intended.

Co-authored-by: Isaac
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants