dataplane: default operator.apiKey.enabled=true (eager API key bootstrap) - #482
Merged
Merged
Conversation
Contributor
Current Aviator status
This PR was merged using Aviator (commit 16bc253).
See the real-time status of this PR on the
Aviator webapp.
Use the Aviator Chrome Extension
to see the status of your PR within GitHub.
|
mhotan
added a commit
that referenced
this pull request
Jul 16, 2026
…ver) Flip the actionsLeasor.enabled chart default false->true per the 2026-07-31 switch-over documented in values.yaml. Every selfhosted CP now routes its own UNION_ORG through the v2 actions service and rejects sub-2.0.4 SDK CreateRun (rejectLegacySDKVersions=true). Envs still on a legacy SDK set actionsLeasor.enabled=false explicitly. Also set global.UNION_ORG=test-org in the no-auth, external-authz, and aws.billing-enable fixtures (they omitted it, so the default-on flag rendered useActionsServiceForOrgs with an empty org); their snapshots now reflect a real single-tenant deployment. Rebased onto latest main. The dataplane operator.apiKey default moved to its own PR (#482). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
aviator-app Bot
pushed a commit
that referenced
this pull request
Jul 16, 2026
…ver) (#481) ## Overview Flips the `actionsLeasor.enabled` controlplane chart default `false → true`, per the switch-over timeline documented in `charts/controlplane/values.yaml`. With the default on, every selfhosted control plane routes its own `UNION_ORG` through the v2 actions service and stops depending on the legacy queue engine for eager/fan-out flows. This is the "chart default flips to `enabled: true`" step of the **2026-07-31** milestone. The queue + executor template removal (the other half of that milestone) is intentionally **not** in this PR. The paired dataplane default (`operator.apiKey.enabled`) is split into its own PR: #482. ## Behavior change For any deployment that does not set `actionsLeasor.enabled` explicitly, the `executions` configmap now renders: - `useActionsServiceForOrgs: [<UNION_ORG>]` (was `[]`) — the deployment's own org routes to the v2 actions service. - `rejectLegacySDKVersions: true` (was `false`) — **CreateRun from SDK < 2.0.4 is hard-rejected.** The `rejectLegacySDKVersions` flip is the real blast radius: an env still on a sub-2.0.4 SDK will hard-fail CreateRun. Opt out with `actionsLeasor.enabled: false`. The actions / leasor / scylla stack templates already rendered unconditionally, so there is **no** deployment/stack churn — the routing-config injection is the only functional delta. ## Note on timing The values.yaml timeline schedules this flip for **2026-07-31**; this lands it ahead of that date deliberately. Merging is a conscious call by the switch-over owner. ## Test Plan - Rebased onto latest `main`; `make generate-expected` + `make test` (helm-test + kubeconform) — green. - 6 controlplane snapshots change: the two routing keys above. The `no-auth`, `external-authz`, and `aws.billing-enable` fixtures gain `global.UNION_ORG: test-org` (they previously omitted it, so the default-on flag rendered an empty org); those snapshots now reflect a real single-tenant deployment. ## Rollback Revert this PR (`actionsLeasor.enabled: true → false`) and regenerate snapshots. State is reversible; no data migration.
mhotan
force-pushed
the
mike/dataplane-operator-apikey-default
branch
from
September 16, 2026 20:48
b7c69c8 to
73ca20a
Compare
mhotan
had a problem deploying
to
helm-charts-ci
September 16, 2026 20:49 — with 
GitHub Actions
Failure
GitHub Actions
Failure
config.operator.apiKey.enabled now defaults to true. The dataplane operator mints EAGER_API_KEY on the control plane and writes it to the task-pod secret store so eager/actions (v2) tasks can call back to the control plane. Under v2 every deployment runs eager/actions workloads, so this should be on. It relies on the proxy secret manager, which is already on by default. RELEASE.md documents the flipped default and the real dependency: the control plane must be able to mint EAGER_API_KEY. If the CP can't self-register OAuth clients on its IdP (selfhosted Okta/Entra), operators must first seed the credentials via the controlplane chart's identity.apiKeyOverrides (EAGER_API_KEY), else the bootstrap fails. Regenerated dataplane snapshots (apiKey block now renders in the operator configmap; configChecksum annotations updated). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Michael Hotan <mike@union.ai>
mhotan
force-pushed
the
mike/dataplane-operator-apikey-default
branch
from
September 16, 2026 22:03
73ca20a to
bda34e1
Compare
xjerod
approved these changes
Sep 16, 2026
Contributor
Author
|
/aviator merge |
Contributor
|
Aviator has accepted the merge request. It will enter the queue when all of the required status checks have passed. Aviator will update the sticky status comment as the pull request moves through the queue. |
aviator-app Bot
pushed a commit
that referenced
this pull request
Sep 18, 2026
## Overview Lowers the `uvolMountBroker` default CPU request from `500m` to `30m`, matching unionai/cloud#18499, which makes the same change to the operator chart that Union-managed (BYOC) dataplanes run. The broker is a DaemonSet at `system-node-critical`, so its request comes off each node's allocatable, and a request a task pod cannot fit around **preempts** that task pod rather than queueing behind it. Two managed customers sizing task pods to their node's full allocatable CPU have now been bitten by the 500m default — one lost 184 running pods to preemption, the other could not schedule at all. Selfmanaged and selfhosted clusters run this chart and have exactly the same exposure. The request is not a cap: there is deliberately no CPU limit, so the broker still bursts to whatever the node has idle. 500m bought a *guaranteed* share under contention. At the measured 60us of broker CPU per open with 8 volumes active, 30m guarantees ~500 opens/s against 500m's ~8,300 — far above what any cluster serving volumes through the CSI driver does today. A cluster that outgrows it should raise the value explicitly, and the rationale comment now says so. ## Test Plan - `./tests/run.sh generate` with the CI-pinned helm 4.2.0, then `make test` — exit 0. The only snapshot movement is the one line in `tests/generated/dataplane.uvol-broker.yaml`. - Committed with `--no-verify`: the local `trailing-whitespace` and `end-of-file-fixer` hooks want to rewrite lines this PR does not touch (three pre-existing trailing-whitespace lines already on `main`, and the trailing blank line the snapshot generator itself emits — stripping it would break `make test`). Neither hook runs in CI; `.github/workflows/checks.yaml` only mirrors the ruff hooks, and there is no Python here. - No `Chart.yaml` bump: per `charts/CONVENTIONS.md` a `version:` bump requires a matching `RELEASE.md` section, and recent values changes (#585, #594, #482, #588) all land unversioned, picked up by the next release commit. ## Rollout Picked up by ArgoCD on the next sync of each selfmanaged/selfhosted dataplane. `maxUnavailable: 25%` means brokers restart a quarter of nodes at a time; a task pod mounting a Volume during its node's restart window would fail `NodePublish`. The per-node effect is a 470m *reduction* in requested CPU, so it frees capacity rather than consuming it. ## Rollback Revert the commit; the broker returns to 500m. A cluster needing the old value sooner can set `uvolMountBroker.resources.requests.cpu` in its env overlay in the cloud repo. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Flips the dataplane chart default
config.operator.apiKey.enabled: false → true. With it on, the operator mints theEAGER_API_KEYon the control plane and writes it to the task-pod secret store, so the union-pod-webhook can inject it into task pods. Nothing mints the key without this, and it is required for v2/actions (eager) execution — see cloud#17056 review. Under v2 every deployment runs eager/actions workloads, so this belongs on by default.Split out of #481 as an isolated change so the controlplane
actionsLeasordefault and this dataplane default land independently.Behavior change
The operator configmap now renders the
apiKeyblock (enabled: true). The prereqproxy.secretManager.enabledis already the chart default (true), so no dataplane loses functionality. A dataplane that must opt out setsconfig.operator.apiKey.enabled: false.No deployments are added or removed — the operator already rendered unconditionally; the delta is the configmap block plus the operator deployment's recomputed
configChecksum.Control-plane dependency (selfhosted)
The dataplane bootstrap only requests the key — the control plane must be able to mint
EAGER_API_KEY. Normally the identity service mints it by registering an OAuth client on the IdP. If the control plane can't self-register clients (common with selfhosted Okta/Entra), operators must first seed pre-created OAuth client credentials via the controlplane chart'sidentity.apiKeyOverrides(system keyEAGER_API_KEY), or the mint fails. Deployments where the CP can self-register need no action.Release notes
RELEASE.mdgains an## Unreleasedentry documenting the flipped default and this control-plane dependency, with the explicit selfhosted guidance above.Test Plan
make generate-expected+make test(helm-test + kubeconform) — green.main; delta limited to theapiKey: {enabled: true}block and the recomputedconfigChecksum.Rollback
Revert this PR (
enabled: true → false) and regenerate snapshots. State is reversible; no data migration.