Support bundle deployment bind for DMS-recorded deployments - #6590
Draft
shreyas-goenka wants to merge 5 commits into
Draft
Support bundle deployment bind for DMS-recorded deployments#6590shreyas-goenka wants to merge 5 commits into
shreyas-goenka wants to merge 5 commits into
Conversation
Bind and unbind now record their change with the deployment metadata service instead of refusing when experimental.deployment_history is on, using bind/unbind operation action types. Enables the DMS variant for the bind/unbind acceptance tests; the few that assert job/pipeline deployment stamps or dumped state stay opted out. Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
Integration test reportCommit: 633c9b5
|
- Always complete the DMS version (with failure on error) after starting it, so a bind/unbind recording failure does not leak the version's lease, matching the deploy and destroy paths. - FetchDeployment now returns the parsed last version id, dropping the duplicate parse in phases and process.go, and keeps the half-created-deployment TODO that was dropped when it moved to libs/dms. - Extract openRecordedState and tighten comments. - New bundle/dms/record-op-fault test asserts the version completes with failure when the operation upload fails; bundle/dms/bind-unbind now cleans up and asserts the resource is managed after bind and re-created after unbind. Co-authored-by: Isaac <no-reply@databricks.com>
Bind is now a first-class plan action. `bundle deployment bind` plans the resource as Bind (config already matches the existing workspace resource) or BindAndUpdate (config differs) and applies it immediately, because the metadata service is the source of truth - unlike the file-based bind, which stages the change for the next deploy. Adds Bind/BindAndUpdate to deployplan and the direct engine's plan/apply, maps them to the BIND / BIND_AND_UPDATE operation types, and drops the seed-state machinery the previous approach needed. The general bind/unbind acceptance tests stay on the file-based (deferred) path, since immediate DMS bind cannot share their goldens; DMS bind/unbind is covered by the dedicated bundle/dms/bind-unbind and record-op-fault tests. Co-authored-by: Isaac <no-reply@databricks.com>
The plan-based bind no longer builds a seed state, so drop the helper it needed. Co-authored-by: Isaac <no-reply@databricks.com>
Descope unbind under the deployment metadata service: no unbind operation action type exists yet, so it reverts to erroring rather than staging an unverified enum. Fix reference resolution so binding a resource that has permissions resolves the adopted id from the plan entry instead of state. Add end-to-end coverage for a plain bind (schema, unchanged) and a bind-and-update (job, reconciled) plus FilterToSelected multi-resource exclusion, and share the deployment create-and-stamp helper between the deploy and bind phases. Co-authored-by: Isaac <no-reply@databricks.com>
shreyas-goenka
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
databricks bundle deployment bindnow works for deployments that record history with the deployment metadata service (DMS). Bind is a first-class plan action: the resource is planned asBind(config already matches the existing workspace resource) orBindAndUpdate(config differs) and applied immediately — DMS is the source of truth — recording aBIND/BIND_AND_UPDATEoperation.Unbind under DMS is descoped for now: there is no unbind operation action type yet, so
bundle deployment unbindcontinues to error for a DMS-recorded deployment rather than staging an unverified enum.Why
The file-based bind stages a change for the next deploy; under DMS the recorded state is the source of truth, so bind plans and applies the adoption in one step (
Bind= adopt unchanged,BindAndUpdate= adopt and apply config). This addsBind/BindAndUpdateto the deployplan and the direct engine's plan/apply, and maps them to the DMSBIND/BIND_AND_UPDATEoperation action types.Tests
The existing file-based bind/unbind acceptance suites stay on the deferred path, since immediate DMS bind cannot share their goldens. New
bundle/dms/bindcovers a plainBIND(a schema whose config matches the existing resource) and aBIND_AND_UPDATE(a job whose config differs, plus its permissions), verifies only the bound resource's subtree is recorded while other resources are left untouched, and chains a second bind onto the first version.bundle/dms/record-op-faultasserts the version is completed with failure when the operation upload fails.This pull request and its description were written by Isaac.