Watch the cluster read-only, apply the way kubectl does, and mean it when verify is green

[cjimti]

Four tickets, three commits. #12 and #69 were written and committed in an earlier session on a stacked branch that never got a PR; #71 and #29 are new here. They ride together because the branch is the close-out of the up-front backlog, not because they share a mechanism — except the last two, which are the same defect from opposite ends.

Closes #69. Closes #71. Closes #29. (#12 is already closed; its code is a90583e.)

a90583e — Live cluster health for the open manifest (#12)

The read-only half of #12; drift detection went to #67 and watch pause/resume to #68.

internal/manifest indexes a checkout through an os.Root-confined walk — multi-doc YAML in, object identity plus originating file out — skipping dot-directories, node_modules and Helm chart subtrees, and reporting what it could not read as notices rather than failing. internal/kubewatch holds client-go list-and-watch connections, one per resource and namespace, with kstatus computing each object's health. Sessions are keyed by binding rather than by project, so a repository laid out one directory per cluster gets one session each and neither's verdicts can appear under the other's name. Failure is a phase, never an exit: a lost connection, an expired credential and an unreadable namespace are all states the panel reports while it retries.

The package is read-only by construction and readonly_test.go fails the build on any call to a mutating verb in its own sources. That is what justifies a second client to the same cluster living beside internal/kubeexec, whose mutations go through the confirm gate in DESIGN.md §6.1.

Two defects were caught in that commit's own adversarial review, each with a test that fails without its fix: an unbound project marshalled nil slices so objects crossed the bridge as null and the panel iterated it (a crash on the state every project starts in), and a watch that closed without delivering anything was reopened immediately — 234,805 reopens in 200ms against a proxy that hangs up on every watch, now paced by the backoff.

bbd6996 — Remove server-side apply (#69)

Found on a real cluster: with the setting on, Diff failed outright with Apply failed with 1 conflict: conflict with kubectl-client-side-apply. The diff was not wrong. kubectl diff --server-side is a server-side apply with dryRun=All, so it failed because the apply would have failed identically — it correctly previewed an impossible operation and offered no way through it.

Server-side apply records per-field ownership and refuses fields owned by another manager. Every object anyone has applied with plain kubectl apply is owned by kubectl-client-side-apply, which is essentially every object in a real cluster. That is a one-time migration only if nothing else ever writes to the cluster again — and m6t is a workbench used alongside colleagues and CI that keep running kubectl, so each of their applies hands ownership back and m6t's next apply conflicts again. Forcing past a conflict on every apply is server-side apply with the only property it buys switched off.

So the flag is not defaulted off or explained better. It is gone, with everything that carried it: the panel toggle, onServerSide, Kube.ServerSide, Binding.ServerSide, the serverSide parameter on Validate/Diff/Apply, and the --server-side argv. Anything left behind would be unreachable under the no-vaporware rule. A projects.yaml carrying serverSide: true from an older build still loads, and the key is dropped on the next write.

This resolves the issue's spike by answering it on the cluster that raised it, rather than keeping a force-conflicts affordance.

1436fc8 — Both halves of the verify/CI parity claim (#71, #29)

#71 — an /events subscriber can miss the publish it was dialed for. dial returns when the client reads the 101, which gorilla writes inside Upgrade; the handler reaches register only after that. publish snapshots the registry, so a connection that is open but not yet registered is absent from the snapshot, is skipped in silence, and the event is never repeated. On a loaded runner that is a red Test job on a diff that cannot reach internal/stream — which is what happened on PR #70.

The fan-out tests now wait for the subscription through a test-only subscriber count, not a sleep. That includes TestTheEventChannelDiscardsClientInput, which had the same exposure and was not named in the issue, and TestPublishHealthIsPublishedToEveryEventSubscriber, which arrived with #12 carrying the identical race.

Two tests keep it closed rather than leaving it to memory. TestPublishReachesRegisteredSubscribersAndNoOneElse drives the registry directly and registers a subscriber after the publish — the dialed-but-unregistered state made deterministic — and fails if publish ever grows a retry or ignores the subscribe flag. TestEventTestsWaitForTheirSubscriptions scans the package's own tests and fails if any of them dials /events and publishes without waiting; deleting a single wait makes it name the offending test.

#29 — make patch-coverage and codecov/patch enforced the same 85% by different arithmetic. A Go profile records blocks, blocks overlap, and a line inside one block that ran and one that did not is what Codecov calls a partial. Its ratio is hits / (hits + misses + partials), so a partial counts against you; go tool cover has no such category and the local gate inherited the optimistic view. On PR #28 that was worth 3.6 points in the direction that misleads — 88.0% and green locally, 84.35% and red in CI, on one diff.

The mapping moves to scripts/coverage-lines.awk, where a line is covered only if every block covering it ran and the reasoning sits beside the rule. Extracting it is what makes it testable: TestCoverageLinesAgreesWithCodecov states Codecov's arithmetic independently in Go and fails if the two part company. The local gate is now the strict side, which is the right direction for a pre-flight check — an over-strict one costs a contributor tests, an over-generous one costs them a red CI run on a diff verify called green.

Evidence

make verify is green on the combined tree — the tree at bbd6996, not on the pieces separately.

patch-coverage reports 92.5% (668/722) against the 85% floor, and that is under the new stricter rule, so the tightened gate lands without forcing coverage work on the code it first measured.

Each new test was checked by breaking what it guards: reverting the merge rule produces exactly 50.0% local against 37.5% Codecov; making publish ignore wantsEvents fails the routing test; deleting an awaitSubscribers call makes the registration gate name the test that lost it. The awk refactor was checked for behaviour equivalence against the real profile — same 2,848 lines, 43 status flips, all 1→0, none in the other direction.

For the reviewer

internal/stream/events_test.go and package_graph_test.go were the two cherry-pick conflicts. In the first, #12's health test and the #71 work both landed and the health test gained the wait it needed. In the second, #70's git → gitexec commentary and #12's app → manifest/kubewatch commentary both describe edges that exist, so both paragraphs are kept; the want map merged cleanly.

I did not adversarially review a90583e or bbd6996 — they carry their own review, recorded in their commit messages. What I verified about them is that they apply to post-#70 main, that the conflicts above are resolved correctly, and that the whole tree passes the gate.

Not in this PR

#26 (reap exited PTY sessions) was scoped and dropped deliberately. Both options in the issue fail one of its own acceptance criteria: a reaper on socket-close reaps immediately after the exit frame, which trades away replay while the tab is open, and a retention cap does not release on tab close. The only shape satisfying both is keeping the terminal socket open past the exit frame — a wire-contract change that also drops the trailing resync marker drain() writes. That is a design decision for the maintainer, not a change to slip into a close-out branch.

[codecov]

Codecov Report

❌ Patch coverage is 93.41421% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.48%. Comparing base (bddefab) to head (bbd6996).

Files with missing lines	Patch %	Lines
internal/kubewatch/pump.go	86.95%	7 Missing and 5 partials ⚠️
internal/kubewatch/session.go	94.55%	7 Missing and 1 partial ⚠️
internal/app/kube.go	86.95%	3 Missing and 3 partials ⚠️
internal/manifest/manifest.go	94.23%	4 Missing and 2 partials ⚠️
internal/kubewatch/connect.go	86.66%	2 Missing and 2 partials ⚠️
internal/kubewatch/plan.go	97.14%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #72      +/-   ##
==========================================
- Coverage   94.73%   94.48%   -0.26%     
==========================================
  Files          42       48       +6     
  Lines        2186     2737     +551     
==========================================
+ Hits         2071     2586     +515     
- Misses         71       95      +24     
- Partials       44       56      +12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.