[jlamillan]

Why

During Kubernetes node churn, backend entries for removed nodes can remain on an OCI load balancer. A backend-set update that adds replacement nodes before removing those stale entries can exceed the load balancer's total backend limit (backends-per-lb-limit) and fail, leaving backend sets out of sync.

Summary

• Remove obsolete backend registrations before submitting a backend-set update that adds replacements.
• Refresh the load balancer and ETag around each deletion, wait for OCI work requests, and retain the last existing backend until a replacement can be added when there is no overlap.
• Reject invalid desired backends before any deletion and serialize backend updates for the same load balancer.

Validation

• go test -mod=vendor ./pkg/...

Operational note

A failed OCI delete or update returns an error for normal reconciliation retry. The change does not remove a backend that is still in the desired set.

Also note, these proposed changes have been running in our production clusters for a number of weeks.