Add technical steering proposal for live control plane migration #796
+384
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gardener-robot
added
the
reviewed/do-not-merge
gardener-github-actions
bot
added
the
reviewed/ok-to-test
gardener-robot
added
needs/review
github-actions
bot
added
needs/ok-to-test
gardener-github-actions
bot
added
the
reviewed/ok-to-test
ary1992
force-pushed
the
steering-live-cpm
branch
from
4b81714 to
b7792f6
Compare
gardener-robot
added
size/L
Member
|
Date proposal: Tue, 10.02., 3pm CET? |
Member
Works for me, but may be late for @ary1992 due to time zone differences. |
Member
|
True. What about Mon, 09.02., 1pm CET is better? |
n-boshnakov
requested changes
Jan 12, 2026
|  | ||
|
|
||
| #### Member removal from the cluster | ||
| - During the six-member cluster formation, the destination seed cluster’s etcd CR contains fields to bootstrap from the source cluster. To complete the migraion, the etcd member count needs to be brought back to three by removing the members that are part of the source seed cluster. For this, the bootstrap with source cluster field is removed from the destination cluster’s etcd CR. At the time of writing the GEP, it was decided that druid will use [EtcdOpsTask](https://github.com/gardener/etcd-druid/issues/1047) to remove members, and a new member removal task will be introduced. |
Contributor
There was a problem hiding this comment.
Suggested change
| - During the six-member cluster formation, the destination seed cluster’s etcd CR contains fields to bootstrap from the source cluster. To complete the migraion, the etcd member count needs to be brought back to three by removing the members that are part of the source seed cluster. For this, the bootstrap with source cluster field is removed from the destination cluster’s etcd CR. At the time of writing the GEP, it was decided that druid will use [EtcdOpsTask](https://github.com/gardener/etcd-druid/issues/1047) to remove members, and a new member removal task will be introduced. | |
| - During the six-member cluster formation, the destination seed cluster’s etcd CR contains fields to bootstrap from the source cluster. To complete the migration, the etcd member count needs to be brought back to three by removing the members that are part of the source seed cluster. For this, the bootstrap with source cluster field is removed from the destination cluster’s etcd CR. At the time of writing the GEP, it was decided that druid will use [EtcdOpsTask](https://github.com/gardener/etcd-druid/issues/1047) to remove members, and a new member removal task will be introduced. |
|
|
||
| ### Failures and Recovery Strategy | ||
|
|
||
| #### **ETCD Quorum loss** |
Contributor
There was a problem hiding this comment.
Suggested change
| #### **ETCD Quorum loss** | |
| #### etcd Quorum Loss |
|
|
||
| ### etcd-druid | ||
|
|
||
| #### Six member etcd cluster |
Contributor
There was a problem hiding this comment.
Suggested change
| #### Six member etcd cluster | |
| #### 6-Member etcd Cluster |
|
|
||
| If the Kube API Server (KAPI) is down because the underlying etcd has lost quorum, follow the recovery steps outlined under ETCD Quorum Loss. | ||
|
|
||
| ##### ETCD is healthy |
Contributor
There was a problem hiding this comment.
Suggested change
| ##### ETCD is healthy | |
| ##### ETCD is Healthy |
|
|
||
| #### Kube API Server is unhealthy | ||
|
|
||
| ##### ETCD is unhealthy |
Contributor
There was a problem hiding this comment.
Suggested change
| ##### ETCD is unhealthy | |
| ##### ETCD is Unhealthy |
| To trigger an abort, the annotation `migration.shoot.gardener.cloud/abort-live-migration=true` should be added to the Shoot. This is permitted only while the `liveMigration` status with step `SixMemberETCDReady` is in `Failed` state. Once annotated, you can switch the seed name back to the source seed in the Shoot spec; the gardenlet will then orchestrate the cleanup of migration-specific resources across both the source and destination seeds. The destination gardenlet will also take care to remove the newly added members from the etcd cluster. | ||
|
|
||
|
|
||
| ##### Quorum is lost at any other stage |
Contributor
There was a problem hiding this comment.
Suggested change
| ##### Quorum is lost at any other stage | |
| ##### Quorum Is Lost at Any Other Stage |
|
|
||
| #### **ETCD Quorum loss** | ||
|
|
||
| ##### The destination members are unable to join |
Contributor
There was a problem hiding this comment.
Suggested change
| ##### The destination members are unable to join | |
| ##### The Destination Members Are Unable to Join |
| Each etcd member pod is individually exposed to enable direct and controlled peer communication during migration required for the [six member etcd cluster](#six-member-etcd-cluster), allowing it to communicate with its peers in the destination Seed cluster. This exposure is achieved via Istio, using dedicated `Gateway` and `VirtualService` configurations. | ||
|
|
||
| #### Components with Shoot webhooks/Controller and Shoot-managed resources (TBD) |
Contributor
There was a problem hiding this comment.
Suggested change
| #### Components with Shoot webhooks/Controller and Shoot-managed resources (TBD) | |
| #### Components with Shoot Webhooks/Controller and Shoot-Managed Resources (TBD) |
Comment on lines
+40
to
+49
| - [Components with Shoot webhooks/Controller and Shoot-managed resources (TBD)](#components-with-shoot-webhookscontroller-and-shoot-managed-resources-tbd) | ||
| - [Live Migration Flow](#live-migration-flow) | ||
| - [etcd-druid](#etcd-druid) | ||
| - [Six member etcd cluster](#six-member-etcd-cluster) | ||
| - [Member removal from the cluster](#member-removal-from-the-cluster) | ||
| - [Decoupling Member Names from Pod Names](#decoupling-member-names-from-pod-names) | ||
| - [VPN](#vpn) | ||
| - [Failures and Recovery Strategy](#failures-and-recovery-strategy) | ||
| - [ETCD Quorum loss](#etcd-quorum-loss) | ||
| - [Kube API Server is unhealthy](#kube-api-server-is-unhealthy) |
Contributor
There was a problem hiding this comment.
Suggested change
| - [Components with Shoot webhooks/Controller and Shoot-managed resources (TBD)](#components-with-shoot-webhookscontroller-and-shoot-managed-resources-tbd) | |
| - [Live Migration Flow](#live-migration-flow) | |
| - [etcd-druid](#etcd-druid) | |
| - [Six member etcd cluster](#six-member-etcd-cluster) | |
| - [Member removal from the cluster](#member-removal-from-the-cluster) | |
| - [Decoupling Member Names from Pod Names](#decoupling-member-names-from-pod-names) | |
| - [VPN](#vpn) | |
| - [Failures and Recovery Strategy](#failures-and-recovery-strategy) | |
| - [ETCD Quorum loss](#etcd-quorum-loss) | |
| - [Kube API Server is unhealthy](#kube-api-server-is-unhealthy) | |
| - [Components with Shoot Webhooks/Controller and Shoot-Managed Resources (TBD)](#components-with-shoot-webhookscontroller-and-shoot-managed-resources-tbd) | |
| - [Live Migration Flow](#live-migration-flow) | |
| - [etcd-druid](#etcd-druid) | |
| - [6-Member etcd Cluster](#six-member-etcd-cluster) | |
| - [Member Removal from the Cluster](#member-removal-from-the-cluster) | |
| - [Decoupling Member Names from Pod Names](#decoupling-member-names-from-pod-names) | |
| - [VPN](#vpn) | |
| - [Failures and Recovery Strategy](#failures-and-recovery-strategy) | |
| - [ETCD Quorum Loss](#etcd-quorum-loss) | |
| - [Kube API Server Is Unhealthy](#kube-api-server-is-unhealthy) |
|
|
||
|  | ||
|
|
||
| #### Member removal from the cluster |
Contributor
There was a problem hiding this comment.
Suggested change
| #### Member removal from the cluster | |
| #### Member Removal from the Cluster |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Add technical steering proposal for live control plane migration (Live CPM)
This proposal describes the approach to be followed for implementing Live Control Plane Migration (Live CPM) in Gardener, enabling migration of a Shoot cluster’s control plane without API server downtime.
Which issue(s) this PR fixes:
Part of #gardener/gardener#10686
Special notes for your reviewer:
/cc @ScheererJ @rfranzke @vlerenc @timebertt
/hold
until a committee meeting has been scheduled
Release note: