fix(k8s): trim owner labels, bound owner cache, refresh discovery

[amir20]

Follow-up cleanup on the owner-chain grouping from #4811.

Changes

• Drop the duplicated legacy k8s.owner.* indexed labels. Only @k8s.owner.* synthetic labels plus top-level owner.kind/name/key are emitted now, halving the per-container label payload that ships over SSE/gRPC. The frontend already reads the synthetic set first, so the legacy indexed fallback was dead weight.
• Add a TTL (5m) and a hard size cap to the owner lookup cache. It could previously grow unbounded under ReplicaSet churn, and Forbidden/NotFound negatives were cached forever, so owner chains never recovered after RBAC was granted without a restart. Entries now expire and get swept.
• Reset the discovery REST mapper once and retry on mapping failure, so CRDs registered after startup (Argo Rollouts, etc.) resolve instead of staying unmapped until restart.

Notes

Owner lookups in the ContainerEvents watch loop are still synchronous, but the cache keys on owner UID so shared controllers (ReplicaSet/Deployment) resolve once per rollout. Leaving that on the hot path since moving it off would risk event reordering for little gain.

Tests updated and passing (go test -race ./internal/k8s, frontend k8s.spec.ts, typecheck).

🤖 Generated with Claude Code

[claude]

• internal/k8s/client.go:335-343: mapping failures still aren't cached (cacheable=false unchanged), but now every failed lookup also calls restMapper.Reset(), forcing a full discovery cache invalidation + rediscovery call. For a pod owned by a permanently-unmappable kind (typo, unsupported apiVersion, uninstalled CRD), this repeats on every pod event/relist instead of failing cheaply — hammers the API server's discovery endpoint under churn. Worth caching negative mapping results too, or rate-limiting the reset.
• client.go:309-317: pruneOwnerCache wholesale-clears the map if still at/over cap after the expiry sweep, instead of evicting oldest/LRU. In a large cluster with >4096 distinct live owners inside the 5m TTL, this thrashes — clear-and-refetch-all repeatedly rather than bounding growth gracefully.
• No new tests for the three actual behavior changes (TTL expiry, cache size cap/pruning, restMapper reset-and-retry). client_test.go only updates label-name assertions for the k8s.owner.* → @k8s.owner.* rename.
• Comment at client.go:333-334 says "reset once" — it resets once per failed call, not once globally; could misread as a one-time mitigation.

[amir20]

Addressed in 0c3766b:

• Discovery reset is now throttled to once/min, so an unmappable owner kind can't hammer the discovery endpoint on every pod event.
• pruneOwnerCache evicts down to 3/4 cap instead of clearing wholesale, no more refetch-all thrash past the cap.
• Added tests for TTL expiry refetch, graceful eviction, and the throttled reset.

On caching negative mapping results: skipped it on purpose so a CRD installed after startup still gets picked up within a minute rather than being stuck for the full TTL. The throttle keeps the cost bounded either way.

Fixed the "reset once" comment wording too.

[claude]

Reviewed the diff. Overall solid cleanup, one design nit:

• pruneOwnerCache (client.go:320): eviction trigger checks len(cache) <= ownerCacheEvictTo, not ownerCacheMaxSize. Since prune runs on every cacheable insert, the cache is trimmed back to evictTo (3072) the moment it crosses that line, so ownerCacheMaxSize (4096) is never actually reached or compared against anywhere. The "hysteresis" the comment describes (grow to max, then drop to evictTo) doesn't happen, it's a small eviction on essentially every insert once past evictTo. Not harmful (cache stays bounded, just at a lower/undocumented threshold), but the constant and comment are misleading. Either gate the eviction on > ownerCacheMaxSize or drop the unused constant.

Everything else checks out:

• TTL + eviction correctly fix unbounded growth and permanent negative caching.
• resetRESTMapper throttling and locking look correct, retry-once-after-reset is reasonable.
• Legacy k8s.owner.* label removal is clean, no remaining references in frontend or backend, isK8sMetadataLabel's @k8s. prefix check already covers the synthetic labels.
• Tests cover TTL expiry, eviction, and mapper reset throttling.

[amir20]

Addressed in 5f6ec34:

• pruneOwnerCache now gates eviction on `> ownerCacheMaxSize`, so the cache actually grows to max before dropping to evictTo. The hysteresis the comment describes now happens, and ownerCacheMaxSize is used instead of being dead. Fixed the comment wording too.

Existing eviction test still covers it (inserts maxSize+100, asserts trim down to evictTo).

[claude]

Reviewed the k8s owner-chain cleanup (label dedup, cache TTL/cap, mapper reset throttle).

• internal/k8s/client.go:315 pruneOwnerCache does a full map scan for expired entries on every cache insert (not just when over ownerCacheMaxSize), while holding ownerCacheMu. Since owner lookups are synchronous in the ContainerEvents watch loop, this adds O(n) work + lock contention on every cache miss as the cache approaches 4096 entries. Worth sweeping expired entries on a ticker instead of inline on every write.
• TTL expiry, eviction hysteresis, and throttled resetRESTMapper all look correct, nice hysteresis choice to avoid wholesale cache clears.
• Legacy k8s.owner.* label removal: confirmed no other references left in the repo (backend or frontend), and isK8sMetadataLabel correctly drops the now-dead strings.HasPrefix(key, "k8s.owner.") check.
• Tests cover the new TTL refetch, graceful eviction, and mapper-reset throttle paths well.