Repository navigation
K8s fleet onboarding — console UX for provider selection ('+ New fleet') #104
Description
Activity
Progress (usercron-k8s-onboarding, first tick): opened #105 —
list_aws_profiles+list_k8s_contextsMCP tools (the two dependency-free discovery tools;list_namespaces/list_service_accountsneed context+namespace already picked, so they're a separate next sub-item).Local build/test hit the known aws-sdk-ec2/s3 OOM constraint on this machine (see PR description) — couldn't get a green local test run, hand-verified the kube-client API surface against vendored crate source instead and am relying on CI as the actual gate. Flagging so a reviewer knows to wait for CI rather than assume local-verified.
Next tick:
list_namespaces/list_service_accounts, or console-side wiring, depending on how #105's CI/review lands.Progress (usercron-k8s-onboarding, second tick): opened #106 (stacked on #105, please merge #105 first) —
list_namespaces+list_service_accountsMCP tools. Also factored the from_kubeconfig+Client::try_from boilerplate into a sharedk8s_client_for()helper and refactoredobserve_k8s_identityto use it (no behavior change).#105 still open/unreviewed, CI was pending as of this tick's start (not red — just hasn't run to completion yet).
Remaining backend items:
k8s_fleet_config_write(wire the already-existingsave_k8s_bindings_text),deploy_provisionprovider param (dispatch toK8sDriver),K8sFleetBinding.expected_principalschema field. Console-side wiring (provider picker,<select>population) is separate again. Next tick: probablyK8sFleetBinding.expected_principal+k8s_fleet_config_writetogether (same file, related), unless #105/#106 need review-comment fixes by then.Progress (usercron-k8s-onboarding, third tick): opened #107 (stacked on #106 → #105, merge in order) — K8sFleetBinding.expected_principal schema field + k8s_fleet_config_write MCP tool. save_k8s_bindings_text needed zero changes (already generic).
Status: #105 CI-green, #106/#107 not yet checked this tick's end (opened just now). None merged/reviewed yet.
Remaining: deploy_provision provider param (dispatch to K8sDriver — the one item that needs real design thought, not just following the AWS-side pattern), plus all console-side wiring (provider picker, fleetsK8sToml.ts, population against the 4 list_* tools). Backend/tool surface for #104 is now feature-complete pending those two. Next tick: probably deploy_provision provider param, since console wiring depends on having something for it to call.
Stopping before writing code on the
deploy_provisionprovider param — found something that changes the shape of the problem, want to flag it rather than build on a wrong premise (same call as the bundle-carrier discovery during #97).Traced the actual call chain for
deploy_provision→t_provision→studio_cp::provision_from_library→oabctl::studio_api::redeploy:redeploy()requires a pre-existing stored manifest (load_manifest()→ S3 GetObject atmanifests/{namespace}/{name}.yaml) — if none exists it errors"no stored manifest ... create the agent before redeploying". It patches that existing manifest'simage/bundle_from, then callsprovision()→EcsDriver::apply().EcsDriver::apply()wrapsapply.rs::apply_manifests— which is where the S3 PUT of the manifest actually happens (apply.rs:~606, "1. Upload manifest to S3 (record of desired state)").K8sDriver::apply()(3b) is a fully independent implementation — directkubeApi<Deployment>::patch(server-side apply), doesn't touch S3 at all, doesn't store a manifest anywhere. It implements the sameProvisionDrivertrait, so swapping it in forEcsDriverinprovision()is mechanically easy — but doing so would mean k8s agents never get a stored manifest, soredeploy()'s "patch the existing one" pattern has nothing to patch.
The deeper finding:
deploy_provision/redeployhas never supported creating a brand-new agent from scratch, for any provider. It only patches an already-stored manifest. The only place that writes a manifest's first version isapply.rs::apply_manifests(ECS-only), which — percreate.rs's own AWS-coupled interactive wizard (the CLI tool, not Studio) — is how agents get created today. I don't see any code path where Studio's console (deploy_provision) creates an agent's first manifest. If that's right, the "+ New fleet" wizard's[+ Add instance]/[+ New fleet]flow may only work today for agents someone already created via the CLI — which would be a pre-existing gap independent of k8s, not something k8s support needs to fix, but it does mean "add a k8s branch todeploy_provision" can't just mean "swap the driver" — there's no manifest-creation entry point in the MCP surface at all to swap a driver into.Not proceeding on this sub-item without your read on it — is my trace right, or is there a manifest-creation path I'm missing (maybe
deploy_applyis meant to be called first by the console beforedeploy_provision, and I haven't found where that's documented/wired)? Pivoting this tick to console-side work that doesn't depend on this (the provider picker UI skeleton, wiring the fourlist_*tools already merged/PR'd) since that's independently useful regardless of how this resolves.Progress (usercron-k8s-onboarding, fourth tick): no response yet on the deploy_provision architecture question from last tick, so pivoted to independent console-side work as planned. Opened #108 (stacked on #107→#106→#105) — provider picker (AWS/k8s) in the "+ New fleet" identity form, context/namespace fields wired live to #105/#106's
list_k8s_contexts/list_namespacestools.k8s submission is intentionally blocked in this PR (clear "not available yet, tracked in #104" message) rather than wired through to a
deploy_provisioncall that can't actually work yet per the open architecture question — see #108's description for the reasoning. AWS path is unchanged.This one's fully locally verified (TS/vite, no OOM constraint) — typecheck clean, 102/102 tests pass, build succeeds. Unlike #105-107 this isn't relying on CI-as-the-only-gate.
Remaining:
list_service_accountsconsole wiring (small),fleetsK8sToml.ts(client-side write helper, needed once k8s identity form actually submits), and thedeploy_provision/K8sDriverdispatch design question (still open, blocking full end-to-end k8s onboarding). Next tick: check for a response on the architecture question first; if none,list_service_accountswiring is the next independent piece.Progress (usercron-k8s-onboarding, fifth tick): still no response on the deploy_provision architecture question. Opened #109 (stacked on #108→#107→#106→#105) — list_service_accounts wired into the k8s field group (select, silent-fallback-on-failure per the design). Also locally verified (typecheck/tests/build all pass).
Remaining open items in #104: fleetsK8sToml.ts (client-side write helper — lower priority until identity-form-for-k8s can actually submit anything), and the deploy_provision/K8sDriver dispatch design question (still the one blocking item, unresolved). All 5 PRs open (#105-#109), none merged/reviewed yet, all CI-green or pending-not-failed.
Next tick: check for a response on the architecture question again; if still none, will likely start fleetsK8sToml.ts (it's independently useful even before deploy_provision dispatch is resolved — the write path can be built now, submission gated same as today) since that's the last clearly-scoped, unblocked item.
Progress (usercron-k8s-onboarding, sixth tick — final for this job run): opened #110 (stacked on #109→#108→#107→#106→#105) —
fleetsK8sToml.ts, the client-side write helper (mirrorsfleetToml.ts, reusesappendMember/findFleetBlockrather than duplicating).This closes out every independently-buildable sub-item in #104's scope: all 5 new MCP tools (
list_aws_profiles,list_k8s_contexts,list_namespaces,list_service_accounts,k8s_fleet_config_write),K8sFleetBinding.expected_principal, and the full console UI (provider picker, context/namespace/service-account fields, write-path helper) are now open as PRs #105-110, stacked in dependency order, all CI-green or passing-so-far, none merged/reviewed yet.The one item still blocked:
deploy_provision's provider-param dispatch toK8sDriver. Traced this two ticks ago (comment above) —redeploy()requires an already-stored manifest to patch, andK8sDriver::applydoesn't persist one, so this isn't a simple driver swap. Posted the finding, no response yet. This is the one piece that needs your design input before it can move, not something further autonomous ticks can resolve — stopping the usercron job here rather than keep polling every 10 min for a response with nothing else to do in the meantime. Whenever you have a read on it, happy to pick this straight back up (either via a new usercron job or synchronously) — the write-path/UI/tools are all ready and waiting for it.Brett's decision (2026-08-27) on the deploy_provision architecture question: go with the "wire creation through deploy_apply" option (not "make K8sDriver mirror the missing behavior" and not "cram creation into redeploy"). Split into a dedicated issue since it's not k8s-specific — #111.
#104's
deploy_provisionprovider-param item now depends on #111. Once #111 lands (AWS-side "+ New fleet" actually able to create a new agent, viadeploy_apply's create-or-update path rather thanredeploy's patch-only path), k8s's provider dispatch happens at that same entry point. Resuming usercron-k8s-onboarding to work #111 first.Resumed (usercron-k8s-onboarding, after #111's #112/#113 landed the shared branch point): opened #114 (stacked on #113→#112) —
provision_from_library_k8s/provision_k8s/build_default_k8s_manifest. K8s side ofdeploy_provisionnow exists, mirroring the AWS path #113 just built.Still not wired:
oab-mcp'sdeploy_provisiontool (t_provision) has no provider param yet, andOabMcphas no k8s-fleet-binding state to resolvecontext/namespace/expected_principalfrom a fleet name — that's next. Once that's done, console's k8s "Next" button (blocked since #108/#109) can finally be un-blocked.- added a commit that references this issue
on Aug 27, 2026 Pushed a follow-up commit to #114 —
t_provisionnow actually dispatches on aproviderarg ("aws" default, unchanged; "k8s" callsprovision_from_library_k8s).context/expected_principalcome as direct MCP-call args (matching what the console's identity form already collects, #108/#109) rather than a fleet-name lookup — no existingK8sFleetBindingto resolve from for a brand-new fleet anyway. Also fixeddeploy_provision's tool description, which had gone stale after #113 (still said "the agent must already have been created").Deploy_provision now fully supports k8s end-to-end. With this, the only remaining piece for full k8s onboarding is un-blocking console's "Next" button placeholder from #108/#109 — separate follow-up, not required to call the backend done.
Flagging a branch-management issue, not hiding it: this #111/#104-resumed stack (#112→#113→#114) was cut from
mainbefore #104's original stack (#105-110) merged — not from #110. SoK8sFleetBinding.expected_principal(originally added in #107) got re-added here too. It's an identical field in both places — a trivial merge conflict, not a real divergence — but you'll hit it whichever order these two stacks merge in. Worth knowing before you start merging either chain.Stack: #112 → #113 → #114 (now includes the provider dispatch), all pending review/merge, no CI failures.
Context
Follow-up to #97 (K8s driver — ADR #63 slice 3, closed 2026-08-27, all 6 sub-slices merged). Driver backend for k8s (
K8sDriver,K8sFleetBinding/fleets-k8s.toml, k8s identity/context observation, k8s secret refs, bundle restore viapre_seed) is complete and onmain. This issue tracks the next phase: console-side onboarding — today "+ New fleet" only supports AWS (Region/Credential profile/Principal); there is no UI path to onboard a k8s-driven fleet, only manualfleets-k8s.tomlediting.Design discussion: Discord thread
1537474496406233130, 2026-08-26.Scope
Full deployment onboarding UX for k8s fleets in the desktop console, on par with the existing AWS "+ New fleet" wizard.
Design decisions (confirmed)
<select>(reusing the existingfillOptionspattern from the compose-library select indeploy.ts), not free-text — with a manual-entry fallback when enumeration is empty/fails.list_namespacestool (Api<Namespace>::list()), + free-text fallback for a namespace that doesn't exist yet.list_service_accounts(context, namespace)tool (Api<ServiceAccount>::list(namespace)) — depends on context+namespace being chosen first. If listing fails (including RBAC 403), silently fall back to leaving it blank (k8s uses the namespace'sdefaultServiceAccount) — no error UI needed for this one.K8sFleetBinding.expected_principal: Option<String>field — deliberately named to match the AWS side'sexpected_principal, sinceobserve_k8s_identity(studio-cp/lib.rs:281) +k8s_principal_kind(:259) already provide the same verify semantics as AWS'sobserve_identity/identity_matches— reuse, don't reinvent.deploy_provisiongets a provider parameter and branches internally to dispatch toK8sDriver— not a separate MCP tool.fleets-k8s.tomlwrite path: newk8s_fleet_config_writeMCP tool wired to the already-existingsave_k8s_bindings_text(studio-cp/lib.rs:677— has unit test coverage, just never wired to a tool). Client-side: newfleetsK8sToml.tsmirroringfleetToml.ts's pure text-mutation helpers (appendMember/appendFleetBlockequivalents for the k8s file).list_aws_profiles/list_k8s_contextsfailure UX — three tiers, not a single generic error:aws configureoraws sso login"; k8s: "install OrbStack/kind/minikube for local, or merge your cloud vendor's kubeconfig into~/.kube/config"). This is actionable becauseoab-mcpis a local Tauri sidecar child process (src-tauri/src/mcp.rs:1-16) reading the operator's own machine.servers_for(management, conn_id)(src-tauri/src/remote.rs:345) already declares theoabreverse-MCP server for management-bound connections; new tools are automatically reachable once built. Non-management agent consoles stay without access (least-privilege, unchanged).New MCP tools needed
list_aws_profiles— read~/.aws/config, return profile names (+ region if set)list_k8s_contexts—kube::config::Kubeconfig::read(), return context names + which iscurrent-contextlist_namespaces(context)—Api<Namespace>::list()list_service_accounts(context, namespace)—Api<ServiceAccount>::list(namespace)k8s_fleet_config_write— wire to existingsave_k8s_bindings_textdeploy_provision— extend with a provider parameter, dispatch toK8sDriverfor k8sSchema changes
K8sFleetBinding(studio-cp/lib.rs:562) — addexpected_principal: Option<String>Client changes
console/src/render.ts/deploy.ts— provider selector step in the identity formconsole/src/fleetsK8sToml.ts(new) — mirrorsfleetToml.tsExplicit non-goals for this phase
pre_seed), no work needed here.