Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/sweep.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ on:
shard_count:
description: "Parallel shards (capped by config/automation-limits.json)"
required: false
default: "22"
default: "28"
item_number:
description: "Optional single issue/PR number to review"
required: false
Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -554,7 +554,7 @@ source ~/.profile
corepack enable
pnpm install
pnpm run build
pnpm run plan -- --target-repo openclaw/openclaw --batch-size 5 --shard-count 22 --max-pages 250 --codex-model internal --codex-reasoning-effort high
pnpm run plan -- --target-repo openclaw/openclaw --batch-size 5 --shard-count 28 --max-pages 250 --codex-model internal --codex-reasoning-effort high
pnpm run review -- --target-repo openclaw/openclaw --target-dir ../openclaw --batch-size 5 --max-pages 250 --artifact-dir artifacts/reviews --codex-model internal --codex-reasoning-effort high --codex-timeout-ms 600000
pnpm run apply-artifacts -- --target-repo openclaw/openclaw --artifact-dir artifacts/reviews --skip-dashboard
pnpm run audit -- --target-repo openclaw/openclaw --max-pages 250 --sample-limit 25 --update-dashboard
Expand Down Expand Up @@ -611,13 +611,13 @@ default, subject to the selected repository profile; pass `target_repo`,
`apply_kind=issue`, or `apply_kind=pull_request` to narrow a manual run.

Scheduled runs cover the configured product profiles. `openclaw/openclaw` runs
normal backfill every 5 minutes with up to 12 review shards when the system is
normal backfill every 5 minutes with up to 28 review shards when the system is
quiet; `openclaw/clawhub` runs on offset review/apply/audit crons so its reports
live under `records/openclaw-clawhub/` without colliding with default repo
records. `openclaw/clawsweeper` has a scheduled read-only audit row and is
available for manual and event self-review smoke tests. Broad hot-intake sweeps
cap scheduled fan-out at 11 one-item shards per run when quiet; manual normal
backfill can use up to 22 shards, while exact event reviews still use one shard.
cap scheduled fan-out at 16 one-item shards per run when quiet; manual normal
backfill can use up to 28 shards, while exact event reviews still use one shard.
Normal review, hot intake, and commit review are
background lanes, so they shrink automatically while repair or exact-item work
is active. Throughput defaults live in
Expand All @@ -626,18 +626,18 @@ is active. Throughput defaults live in
### Worker Budget

ClawSweeper has one main capacity knob:
`config/automation-limits.json` -> `workers.max`. The current value is `32`.
Lane limits are derived from that number: normal review defaults to 22 shards
for manual/backstop runs, scheduled normal review gets up to 12 after reserves,
hot intake up to 11 shards, commit review 1 commit per page, and existing
repair/issue implementation lanes use 40% of `workers.max`, currently 12 live
`config/automation-limits.json` -> `workers.max`. The current value is `48`.
Lane limits are derived from that number: normal review defaults to 28 shards
for manual/backstop runs, scheduled normal review gets up to 28 after reserves,
hot intake up to 16 shards, commit review 2 commits per page, and existing
repair/issue implementation lanes use 40% of `workers.max`, currently 19 live
workers. Imported gitcrawl cluster repair allows 2 live workers by default.
Exact-item review, repair, and issue implementation are priority work; normal
review, hot intake, and commit review are background work and automatically
yield when priority work is active. Exact-item runs use a durable Worker queue
that coalesces item deliveries, leases at most 4 concurrent reviews, and admits
at most one active exact review per target repository. Other lanes retain the
checked-in 32-worker scheduling model.
checked-in 48-worker scheduling model.
Use `workers.max` first when turning total Codex usage up or down; use
`lanes.repair.cluster_max_live_runs` to tune the imported legacy cluster-repair
lane separately, and individual environment overrides only for temporary
Expand Down
2 changes: 1 addition & 1 deletion config/automation-limits.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"workers": {
"max": 32,
"max": 48,
"reserve_for_interactive": 8,
"expansion_reserve": 12,
"minimum_background": 4
Expand Down
5 changes: 3 additions & 2 deletions dashboard/worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ const CLOSED_STATS_PAGE_LIMIT = 10;
const DEFAULT_CLAWSWEEPER_BOT_LOGINS = ["clawsweeper[bot]", "openclaw-clawsweeper[bot]"];
const GITHUB_TIMEOUT_MS = 4500;
const DEFAULT_STALE_QUEUED_WORKFLOW_MS = 6 * 60 * 60 * 1000;
const DEFAULT_WORKER_BUDGET = 48;
const DEFAULT_EXACT_REVIEW_QUEUE_MAX_CONCURRENT = 4;
const DEFAULT_EXACT_REVIEW_TARGET_MAX_CONCURRENT = 1;
const DEFAULT_EXACT_REVIEW_DISPATCH_LEASE_MS = 10 * 60 * 1000;
Expand Down Expand Up @@ -1240,7 +1241,7 @@ export function exactReviewQueueCapacity(env) {
return Math.max(
1,
Math.min(
32,
DEFAULT_WORKER_BUDGET,
numberFrom(env.EXACT_REVIEW_QUEUE_MAX_CONCURRENT, DEFAULT_EXACT_REVIEW_QUEUE_MAX_CONCURRENT),
),
);
Expand Down Expand Up @@ -1633,7 +1634,7 @@ async function statusSnapshot(env) {
.split(",")
.map((value) => value.trim())
.filter(Boolean);
const budget = numberFrom(env.WORKER_BUDGET, 32);
const budget = numberFrom(env.WORKER_BUDGET, DEFAULT_WORKER_BUDGET);
const [runs, completedRuns, filteredActiveRuns] = await Promise.all([
githubJson(env, `/repos/${repo}/actions/runs?per_page=100`).catch((error) => {
errors.push(`workflow runs: ${error.message}`);
Expand Down
2 changes: 1 addition & 1 deletion dashboard/wrangler.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ CLAWSWEEPER_REPO = "openclaw/clawsweeper"
CLAWSWEEPER_APP_CLIENT_ID = "Iv23liOECG0slfuhz093"
CLAWSWEEPER_CRABFLEET_URL = "https://crabfleet.openclaw.ai"
TARGET_REPOS = "openclaw/openclaw,openclaw/clawhub,openclaw/clawsweeper,openclaw/fs-safe"
WORKER_BUDGET = "32"
WORKER_BUDGET = "48"
WORKER_DETAIL_RUN_LIMIT = "128"
WORKER_JOB_FETCH_CONCURRENCY = "12"
WORKER_JOB_CACHE_TTL_SECONDS = "60"
Expand Down
6 changes: 3 additions & 3 deletions docs/commit-dispatcher.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ CLAWSWEEPER_COMMIT_REVIEW_SETTLE_SECONDS=60
Use `0` for settled manual backfills or a larger value during GitHub event
lag incidents.

Commit review is a background lane. It defaults to 1 commit per workflow page
Commit review is a background lane. It defaults to 2 commits per workflow page
when the system is quiet, but the receiver asks the central worker scheduler for
capacity before each page. Active repair, exact-item review, and sweep work can
lower the page size so commit review does not consume capacity needed by
Expand All @@ -116,10 +116,10 @@ maintainer-visible work. The checked-in default comes from
`openclaw/clawsweeper` only when the org has enough rate-limit headroom:

```text
CLAWSWEEPER_COMMIT_REVIEW_PAGE_SIZE=1
CLAWSWEEPER_COMMIT_REVIEW_PAGE_SIZE=2
```

The receiver clamps this between 1 and 32. Setting the variable bypasses the
The receiver clamps this between 1 and 48. Setting the variable bypasses the
dynamic default for that run; leave it unset when the central scheduler should
decide. Large push ranges continue in later workflow pages.

Expand Down
4 changes: 2 additions & 2 deletions docs/commit-sweeper.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,13 +89,13 @@ by SHA/range rather than detaching the whole target repository at the commit.

## Scaling

Commit Sweeper is background work. It defaults to 1 commit per workflow page
Commit Sweeper is background work. It defaults to 2 commits per workflow page
when the system is quiet, but the receiver asks the central worker scheduler for
the effective page size before dispatching the matrix. Active repair,
exact-item review, and sweep work can lower commit review to keep capacity
available for maintainer-visible work. The checked-in default lives in
`config/automation-limits.json`. The receiver clamps
`CLAWSWEEPER_COMMIT_REVIEW_PAGE_SIZE` between 1 and 32, then pages large ranges:
`CLAWSWEEPER_COMMIT_REVIEW_PAGE_SIZE` between 1 and 48, then pages large ranges:

- select up to the configured page size
- classify them cheaply
Expand Down
48 changes: 25 additions & 23 deletions docs/limits.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ The mental model:

| Name | Current | Meaning |
| --- | ---: | --- |
| `workers.max` | 32 | Maximum global Codex worker budget used to derive lane limits. |
| `workers.max` | 48 | Maximum global Codex worker budget used to derive lane limits. |
| `workers.reserve_for_interactive` | 8 | Worker slots background lanes leave open for exact/manual/urgent work. |
| `workers.expansion_reserve` | 12 | Extra slots background lanes leave open for independently planned matrix expansion. |
| `workers.minimum_background` | 4 | Target floor for background progress when enough global capacity is available. |
Expand All @@ -43,34 +43,36 @@ The mental model:

## Derived Limits

Review, commit, and existing repair limits are intentionally percentages of
Review, commit, and existing repair limits are intentionally derived from
`workers.max`; imported cluster repair has its own lane knob. With
`workers.max = 32`, normal review can use 22 workers, hot intake can use 11,
commit review can use 1 commit per page, existing repair lanes dispatch 12
`workers.max = 48`, normal review can use 28 workers after the interactive and
expansion reserves, hot intake can use 16,
commit review can use 2 commits per page, existing repair lanes dispatch 19
live workers by default, and imported cluster repair dispatches two live workers
by default.

| Name | Current | Meaning |
| --- | ---: | --- |
| `exact_review.concurrent_max` | 4 | Exact-item review admission cap, clamped to `workers.max`. |
| `assist.default` | 10 | Maintainer assist job cap. |
| `review_shards.normal_default` | 22 | Quiet-system normal review shard ceiling. |
| `review_shards.normal_active_floor` | 9 | Minimum active normal review shards to keep queued for `openclaw/openclaw`. |
| `review_shards.hot_intake_default` | 11 | Quiet-system broad hot-intake review shard ceiling. |
| `review_shards.normal_default` | 28 | Quiet-system normal review shard ceiling. |
| `review_shards.normal_active_floor` | 14 | Minimum active normal review shards to keep queued for `openclaw/openclaw`. |
| `review_shards.hot_intake_default` | 16 | Quiet-system broad hot-intake review shard ceiling. |
| `review_shards.exact_item_default` | 1 | Exact-item hot-intake shard count. |
| `review_shards.hard_cap` | 32 | Maximum accepted review shard count. |
| `commit_review.page_size_default` | 1 | Commits selected per commit-review page. |
| `commit_review.page_size_hard_cap` | 32 | Maximum commit-review page size. |
| `repair_live_runs.default` | 12 | Default live repair workflow run cap for manual dispatch/requeue/self-heal. |
| `repair_live_runs.hard_cap` | 32 | Absolute live repair run cap accepted by explicit CLI/env overrides with this config. |
| `repair_live_runs.automerge_default` | 12 | Live repair run cap for automerge comment-router dispatches. |
| `repair_live_runs.issue_implementation_default` | 12 | Live repair run cap for issue-to-PR implementation intake. |
| `review_shards.hard_cap` | 48 | Maximum accepted review shard count. |
| `commit_review.page_size_default` | 2 | Commits selected per commit-review page. |
| `commit_review.page_size_hard_cap` | 48 | Maximum commit-review page size. |
| `repair_live_runs.default` | 19 | Default live repair workflow run cap for manual dispatch/requeue/self-heal. |
| `repair_live_runs.hard_cap` | 48 | Absolute live repair run cap accepted by explicit CLI/env overrides with this config. |
| `repair_live_runs.automerge_default` | 19 | Live repair run cap for automerge comment-router dispatches. |
| `repair_live_runs.issue_implementation_default` | 19 | Live repair run cap for issue-to-PR implementation intake. |
| `repair_live_runs.cluster_default` | 2 | Live repair run cap for imported gitcrawl cluster dispatches. |
| `issue_implementation.dispatches_per_sweep_default` | 1 | Maximum implementation intake jobs queued from one review publish run. |

Formula summary:

- normal review: 70% of `workers.max`
- normal review: the smaller of 70% of `workers.max` and the capacity left
after interactive and expansion reserves
- normal active floor: 30% of `workers.max`
- hot intake: 35% of `workers.max`
- commit review page size: 5% of `workers.max`
Expand Down Expand Up @@ -124,13 +126,13 @@ keeps capacity waiting and retry state out of GitHub Actions runners.

Examples with the current config:

- Quiet system: manual normal review can request 22 shards; scheduled normal
review gets 12 after reserving 8 slots for exact/manual/urgent work and 12
- Quiet system: manual normal review can request 28 shards; scheduled normal
review gets 28 after reserving 8 slots for exact/manual/urgent work and 12
slots for in-flight matrix expansion.
- 4 active repair workers and 8 active background workers: normal review gets
1 because `32 - 8 interactive reserve - 12 expansion reserve - 4 priority
- 8 background = 0`, and enabled background lanes keep one slow-progress worker.
- 24 active priority workers: commit review gets 1, so commit review yields but
16 because `48 - 8 interactive reserve - 12 expansion reserve - 4 priority
- 8 background = 16`.
- 46 active priority workers: commit review gets 1, so commit review yields but
does not fully stall.

Use these commands to inspect the effective values from a checkout:
Expand All @@ -143,9 +145,9 @@ pnpm run --silent workflow -- worker-limit commit_review --active-critical 88
```

Change `workers.max` first when tuning review-side rate-limit pressure. For
example, setting `workers.max` to `40` automatically makes normal review `28`,
hot intake `14`, and commit review `2`. Existing repair lanes keep their
40% derived caps, while imported cluster repair remains separately bounded until
example, setting `workers.max` to `48` makes normal review `28`, hot intake
`16`, and commit review `2`. Existing repair lanes use their 40% derived caps,
while imported cluster repair remains separately bounded until
`lanes.repair.cluster_max_live_runs` is raised.

## Runtime Overrides
Expand Down
4 changes: 2 additions & 2 deletions docs/live-dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,8 +130,8 @@ The Worker fetches job details only for the bounded active-run set, limits that
GitHub fanout to 12 concurrent requests, and caches each run's jobs for 60
seconds. It separately samples 20 recent completed worker runs with ten-way
fanout and caches error/recovery telemetry for 120 seconds. This bounds
telemetry pressure without exceeding the 32-worker fleet budget. Worker details
paginate up to 300 jobs per workflow run so 22-shard runs remain fully visible,
telemetry pressure without exceeding the 48-worker fleet budget. Worker details
paginate up to 300 jobs per workflow run so 28-shard runs remain fully visible,
then finish before optional pipeline CI and historical
enrichment begin, so those secondary lookups do not compete with active worker
telemetry. If GitHub job telemetry is unavailable, the API and UI retain the
Expand Down
4 changes: 2 additions & 2 deletions docs/repair/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,12 +274,12 @@ pnpm run repair:import-gitcrawl -- --from-gitcrawl --limit 40 --mode autonomous

# Dispatch reviewed jobs. Dispatch derives its default live-worker cap from the
# job's job_intent and config/automation-limits.json. Existing repair lanes
# keep the normal 40%-of-workers.max cap, currently 12; imported gitcrawl
# keep the normal 40%-of-workers.max cap, currently 19; imported gitcrawl
# cluster jobs default to lanes.repair.cluster_max_live_runs, currently 2.
# Use CLAWSWEEPER_MAX_LIVE_WORKERS/--max-live-workers for a one-lane override.
# With --wait-for-capacity, dispatch can drain a larger file
# list in capacity-sized waves instead of refusing the whole batch.
CLAWSWEEPER_MAX_LIVE_WORKERS=12 pnpm run repair:dispatch -- jobs/openclaw/inbox/ordinary-example.md \
CLAWSWEEPER_MAX_LIVE_WORKERS=19 pnpm run repair:dispatch -- jobs/openclaw/inbox/ordinary-example.md \
--mode autonomous \
--runner blacksmith-4vcpu-ubuntu-2404 \
--execution-runner blacksmith-16vcpu-ubuntu-2404
Expand Down
Loading