Skip to content

perf(scheduler): raise ClawSweeper worker budget#350

Draft
vincentkoc wants to merge 9 commits into
mainfrom
ops/clawsweeper-throughput
Draft

perf(scheduler): raise ClawSweeper worker budget#350
vincentkoc wants to merge 9 commits into
mainfrom
ops/clawsweeper-throughput

Conversation

@vincentkoc

Copy link
Copy Markdown
Member

What Problem This Solves

ClawSweeper background review can fall hours behind while the live fleet is healthy because the checked-in worker budget and derived lane caps are too conservative for the current queue.

Why This Change Was Made

Raise the single global budget from 32 to 48 and regenerate the derived workflow, dashboard, and operator-document values. The existing reserve and priority-lane rules stay unchanged.

User Impact

Normal review rises to 33 shards, hot intake to 16, commit review pages to 2, and repair defaults to 19 while preserving the existing reserve policy.

Evidence

  • pnpm run check:limits passes.
  • Live dashboard sample showed 24 active Codex jobs at 75% of the old 32-slot budget with zero recent failures.
  • Main still reports workers.max=32; this branch carries the intended 48 budget.

AI-assisted: yes.

@clawsweeper

clawsweeper Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 23, 2026, 5:50 AM ET / 09:50 UTC.

Summary
The PR raises ClawSweeper's global worker budget from 32 to 48 and updates derived scheduler, workflow, dashboard, documentation, script, and test capacity values.

Reproducibility: not applicable. This PR changes operational scheduler capacity rather than fixing a reproducible product bug. Source inspection can verify whether the derived capacity values and docs are synchronized, and the latest head now passes that focused stale-reference check.

Review metrics: 3 noteworthy metrics.

  • Diff surface: 17 files changed, +109/-80. The single budget knob fans out into workflow defaults, dashboard behavior, docs, source derivation, and tests that must stay synchronized.
  • Budget change: 32 -> 48 workers. This is a live automation capacity increase, not only a documentation refresh.
  • Derived fan-out: normal 22 -> 28, hot 11 -> 16, commit 1 -> 2, repair 12 -> 19. These derived limits are the operator-visible behavior maintainers need to evaluate before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🌊 off-meta tidepool
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Refresh the PR body so the user-impact section says normal review is capped at 28, not 33.
  • Provide or confirm current live headroom evidence before accepting the 48-worker capacity risk.

Risk before merge

  • [P1] Merging intentionally raises live automation fan-out to 28 normal review shards, 16 hot-intake shards, 2 commit-review pages, and 19 repair workers; ordinary CI cannot prove hosted GitHub/Codex fleet headroom.
  • [P1] The PR body still says normal review rises to 33 shards, while the latest branch caps normal review at 28, so the public merge context should be refreshed before capacity approval.
  • [P1] This PR is adjacent to ClawSweeper does not review fresh openclaw/openclaw PRs: per-target exact-review cap of 1 (PR #348) starves new PRs #349 but does not solve that issue's per-target exact-review starvation path.

Maintainer options:

  1. Refresh capacity context before approval (recommended)
    Update the PR body or maintainer-visible merge context so it matches the latest branch values before a maintainer accepts the live budget increase.
  2. Accept the higher fan-out after live review
    Maintainers can intentionally own the 48-worker automation risk after checking current dashboard and Actions headroom evidence.
  3. Stage or pause the increase
    If live headroom is not convincing, keep the PR draft/open or replace it with a smaller staged capacity increase.

Next step before merge

  • [P2] The remaining action is maintainer/operator review of the higher live automation budget and PR-body context, not a narrow source-code repair.

Security
Cleared: The diff changes capacity settings, workflow defaults, dashboard fallback values, docs, scripts, and tests without adding dependencies, secrets handling, permissions, or downloaded code execution.

Review details

Best possible solution:

Land the consistent 48-worker capacity model only after maintainers accept the live automation headroom, refresh the PR body to the final 28-shard normal cap, and keep the exact-review starvation issue tracked separately.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this PR changes operational scheduler capacity rather than fixing a reproducible product bug. Source inspection can verify whether the derived capacity values and docs are synchronized, and the latest head now passes that focused stale-reference check.

Is this the best way to solve the issue?

Yes for the code path: keeping workers.max as the main knob and deriving lane limits follows the existing source-of-truth pattern. It is not sufficient as a product/operator decision by itself, and it should not be treated as solving the separate exact-review starvation issue.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f94588d89c2f.

Label changes

Label changes:

  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate is not applicable because this PR is authored by a repository member; the body still provides check:limits and live dashboard context for maintainer review.
  • remove status: ⏳ waiting on author: Current PR status label is status: 👀 ready for maintainer look.
  • remove rating: 🦐 gold shrimp: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.

Label justifications:

  • P2: This is a bounded scheduler throughput improvement with limited blast radius but real automation-capacity impact.
  • merge-risk: 🚨 automation: The PR raises workflow shard and repair dispatch capacity, so green CI cannot prove live automation headroom after merge.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate is not applicable because this PR is authored by a repository member; the body still provides check:limits and live dashboard context for maintainer review.
Evidence reviewed

What I checked:

  • AGENTS.md policy applied: Repository policy was read fully; its throughput/default tuning guidance is relevant because this PR changes both scheduler defaults and .github/workflows/sweep.yml. (AGENTS.md:1, f94588d89c2f)
  • Current main is not already implemented: Current main still sets workers.max to 32, so the central 48-worker request is not already on main. (config/automation-limits.json:3, f94588d89c2f)
  • Latest release is not already implemented: The v0.3.0 tag points at dc82491 and its automation limits do not contain the requested 48-worker budget. (config/automation-limits.json:3, dc824915bb6c)
  • PR head implements capped 48-worker derivation: The branch sets workers.max to 48 and derives normal review as the smaller of 70% and capacity remaining after interactive and expansion reserves, producing the current 28-shard normal cap. (src/limits.ts:77, 768e8accce2a)
  • PR body still has stale public context: GitHub PR metadata shows the body still says normal review rises to 33 shards, while the current head derives and documents 28. (768e8accce2a)
  • Stale capacity references were rechecked: A targeted search of the changed files at PR head found no remaining stale 22/33/32-worker derived-capacity strings from the previous review findings. (768e8accce2a)

Likely related people:

  • vincentkoc: Recent merged PRs changed the same worker budget, workflow fanout, dashboard, scheduler docs, and capacity tests, and this PR continues that area work. (role: recent area contributor; confidence: high; commits: f69d9debf487, c66b2dedc6ec, f94588d89c2f; files: config/automation-limits.json, .github/workflows/sweep.yml, dashboard/worker.ts)
  • Peter Steinberger: Git blame ties the current derived-limit implementation and workflow scheduler capacity model to the v0.3.0 release-era implementation, with later exact queue expansion in the same capacity surface. (role: original surface contributor; confidence: medium; commits: dc824915bb6c, 30163dfd667b; files: src/limits.ts, src/repair/limits.ts, config/automation-limits.json)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. labels Jun 23, 2026
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jun 23, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. P2 Normal priority bug or improvement with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant