Skip to content

fix: rotate apply queue batches#375

Merged
RomneyDa merged 3 commits into
openclaw:mainfrom
brokemac79:codex/apply-queue-rotation
Jul 1, 2026
Merged

fix: rotate apply queue batches#375
RomneyDa merged 3 commits into
openclaw:mainfrom
brokemac79:codex/apply-queue-rotation

Conversation

@brokemac79

@brokemac79 brokemac79 commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

Scheduled apply-existing can spend its processed-record budget on the same front of the proposed-close queue when many candidates are skipped, policy-blocked, or otherwise unchanged. That can make later stale OpenClaw issues/PRs wait indefinitely even though the workflow is running.

This PR makes automatic apply runs rotate through a bounded cursor window instead of always scanning the same candidates first.

Why This Change Was Made

  • Auto-select proposed apply candidates with a 300-record window keyed by apply_checked_at and item number.
  • Persist results/apply-cursors/<target>.json after automatic apply checkpoints.
  • Stamp apply_checked_at on processed apply outcomes, including comment sync, label sync, policy skips, and closed records.
  • Clear auto-selected item numbers before continuation dispatches so the next run reselects from the persisted cursor.
  • Initialize apply-selection flags inside the apply shell step that uses them, avoiding set -u unbound variables.
  • Persist cursor progress even if an auto-selected window produces an empty report, using the selected item list as the cursor fallback.
  • Avoid immediate zero-close auto-continuation churn; the next scheduled apply run advances the next cursor window.
  • Exclude proposed records without item_snapshot_hash from apply preselection, because apply-decisions ignores them before producing report entries.

User Impact

ClawSweeper should move through stale apply candidates more predictably instead of repeatedly spending the same apply budget on a skip-heavy prefix. This does not run the #371 manual back-sweep; that remains an explicit maintainer-triggered apply run.

Evidence

Open PR overlap check

Checked the current open openclaw/clawsweeper PRs on 2026-06-27: #374, #370, #369, #358, #350, #342, and #341. The nearest overlaps are #350 (worker budget/caps), #341 (repair checkpoint contract), and #358 (decision packets), but none implement apply queue rotation, apply_checked_at coverage, or apply cursor persistence.

ClawSweeper feedback addressed

  • P1 unbound workflow variables: moved explicit_item_numbers and auto_selected_apply_batch initialization into the apply step that uses them.
  • P2 zero-close continuation churn: automatic full windows with zero fresh closes now persist the cursor and wait for the next scheduled apply run instead of self-dispatching immediately.
  • P2 empty-report cursor stall: write-apply-cursor now accepts --item-numbers and falls back to the selected window when the apply report is empty.
  • P2 zero-result preselection: proposed records without item_snapshot_hash are not selected for apply windows.

Tests and checks

  • pnpm run build:all
  • node --test test/repair/workflow-utils.test.ts
  • node --test --test-name-pattern "(apply workflow bounds checkpoints and requeues with a fresh token|apply-decisions counts advisory label-only syncs against the processed limit|apply-decisions syncs labels when first review placeholder advanced issue updated_at)" test/sweep-workflow.test.ts test/clawsweeper.test.ts
  • pnpm exec oxfmt --check .github/workflows/sweep.yml src/clawsweeper.ts src/repair/workflow-utils.ts test/clawsweeper.test.ts test/repair/workflow-utils.test.ts test/sweep-workflow.test.ts
  • pnpm run lint:repair
  • pnpm run lint:src
  • pnpm run lint:scripts
  • pnpm run codex:local:check
  • codex review -c 'service_tier="fast"' --base origin/main -> no actionable correctness issues identified.
  • pnpm run review -- --local-only --target-repo openclaw/clawsweeper --item-number 375 --target-dir C:\oc-work\clawsweeper-apply-queue-rotation --artifact-dir artifacts/local-review-375 --codex-model gpt-5.5 --codex-reasoning-effort high --codex-timeout-ms 600000 -> patch correct, no review findings; remaining local review rank-up was PR-body proof, provided below.

Known local limitation: full node --test test/sweep-workflow.test.ts still has an existing Windows checkout file-mode assertion failure in read-only checkout mode restores file modes and leaves git metadata writable (292 !== 365). The apply workflow assertions in that file pass.

Temp-state apply proof

This proof used a temp record tree and the built CLI, not live GitHub mutation. It exercised cursor-based preselection, real apply-decisions processing with --processed-limit 300, cursor persistence, next-window selection, and the empty-report cursor fallback.

proof_root=C:\Users\marti\AppData\Local\Temp\clawsweeper-apply-cursor-proof-HxgqH9
initial_batch_count=300
initial_batch_first=1
initial_batch_last=300
apply_report_count=300
apply_report_first=1:kept_open
apply_report_last=300:kept_open
apply_checked_at_count=300
cursor_write=processed_count=300; selected_count=300; next_cursor_number=300; next_cursor_apply_checked_at=2026-06-27T14:03:06.038Z
cursor_next_after_number=300
cursor_processed_count=300
cursor_selected_count=300
next_batch_count=300
next_batch_first=301
next_batch_last=600
apply_exit_code=0
apply_stderr_tail=[apply] 2026-06-27T14:03:06.039Z finished apply closed=0/5 processed=300/300 counts={"kept_open":300}
empty_proof_root=C:\Users\marti\AppData\Local\Temp\clawsweeper-empty-cursor-proof-azvvio
empty_cursor_write=processed_count=0; selected_count=3; next_cursor_number=3; next_cursor_apply_checked_at=
empty_cursor_next_after_number=3
empty_cursor_processed_count=0
empty_cursor_selected_count=3
empty_next_batch=4,5,1

@clawsweeper

clawsweeper Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 27, 2026, 10:33 AM ET / 14:33 UTC.

Summary
This PR adds cursor-window selection for scheduled apply-existing runs, persists apply cursor state, stamps apply_checked_at on processed outcomes, adjusts continuation handling, and adds focused tests.

Reproducibility: yes. source-reproducible. Current main preselects proposed close records by item number without a bounded apply cursor, so a skip-heavy prefix can consume the processed-record budget before later records are reached.

Review metrics: 3 noteworthy metrics.

  • Workflow surface: 1 workflow changed. The changed workflow owns live apply checkpointing, cursor publication, and continuation dispatch.
  • Patch size: 6 files, +480/-70. The diff spans workflow shell, runtime apply logic, workflow utilities, and tests.
  • Temp-state proof: 300 processed; next batch 301-600; empty fallback 3 selected. The proof directly exercises the queue rotation behavior under review.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] This changes live scheduled apply cursor state, state publishing, and continuation behavior; a bad cursor path could stall apply progress or repeatedly revisit candidate windows.
  • [P1] The proof is a temp-state built-CLI run rather than a live GitHub-mutating scheduled run, so maintainers still own rollout observation for the production state repo path.

Maintainer options:

  1. Land after reviewing the cursor proof (recommended)
    Maintainers can accept the remaining rollout risk because the PR demonstrates cursor persistence, next-window selection, and empty-report fallback in a built-CLI temp-state run.
  2. Request one scheduled-run observation
    Before merging, maintainers can ask for a redacted workflow-shaped log or status sample showing the published apply cursor in the state repo path.
  3. Pause for a smaller rollout
    If cursor state changes feel too broad for the current apply lane, keep the PR open until a narrower operator rollout plan is agreed.

Next step before merge

  • No automated repair lane is needed; the remaining action is maintainer judgment on live apply rollout risk after green checks and sufficient temp-state proof.

Security
Cleared: No credential, permission, dependency, third-party execution, or supply-chain expansion was found beyond the existing workflow/apply automation path.

Review details

Best possible solution:

Land the cursor-based apply rotation after maintainers accept the live automation rollout risk and keep any extra production observation in workflow logs/status rather than adding branch churn.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible. Current main preselects proposed close records by item number without a bounded apply cursor, so a skip-heavy prefix can consume the processed-record budget before later records are reached.

Is this the best way to solve the issue?

Yes. The apply_checked_at cursor, processed-outcome stamping, and continuation changes are a narrow maintainable fix for the queue starvation path, with focused tests and temp-state proof covering the main edge cases.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against ae63b16d6c74.

Label changes

Label justifications:

  • P1: The PR changes live scheduled apply automation that determines whether high-confidence close proposals keep processing instead of stalling.
  • merge-risk: 🚨 automation: The diff changes workflow cursor selection, persisted apply state, and continuation mechanics that green CI cannot fully prove in production.
  • merge-risk: 🚨 availability: A bad cursor or continuation path could leave scheduled apply runs stuck or repeatedly processing the same candidate window.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes copied terminal output from a temp-state built-CLI run showing after-fix cursor persistence, next-window selection, and empty-report fallback.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied terminal output from a temp-state built-CLI run showing after-fix cursor persistence, next-window selection, and empty-report fallback.
Evidence reviewed

What I checked:

  • Repository policy applied: AGENTS.md says the apply lane mutates GitHub and that generated state in openclaw/clawsweeper-state is the live status surface, so this workflow/state change is automation-sensitive review material. (AGENTS.md:26, ae63b16d6c74)
  • Current main lacks apply cursor preselection: Current main auto-selects proposed close records with proposed-item-numbers but does not pass a batch size or apply cursor path, so the central rotation behavior is not already implemented. (.github/workflows/sweep.yml:2360, ae63b16d6c74)
  • Current main selection is number-sorted: Current main returns eligible proposed item numbers sorted only by item number, which supports the reported skip-heavy-prefix starvation path. (src/repair/workflow-utils.ts:321, ae63b16d6c74)
  • PR head adds cursor-ordered bounded selection: The PR head filters snapshot-backed candidates, sorts by apply_checked_at and item number, reads the persisted cursor, and slices a bounded batch. (src/repair/workflow-utils.ts:337, 10ad04bcc879)
  • PR head persists apply cursor progress: writeApplyCursor records the last processed item number, falls back to the selected item list when the report is empty, and writes selected and processed counts. (src/repair/workflow-utils.ts:595, 10ad04bcc879)
  • PR head publishes apply cursor state: The workflow writes results/apply-cursors, includes it in publish paths for auto-selected batches, and clears auto-selected item numbers before continuation dispatch. (.github/workflows/sweep.yml:2488, 10ad04bcc879)

Likely related people:

  • Dallin Romney: Blame ties the core apply queue iteration, applyDecisionsCommand, and proposed-item selection scaffolding to the initial ClawSweeper source import. (role: introduced behavior; confidence: high; commits: 4e5c4d47c83f; files: src/clawsweeper.ts, src/repair/workflow-utils.ts, .github/workflows/sweep.yml)
  • Vincent Koc: Recent main history changed apply checkpoint token renewal, close_processed_limit, and continuation gating in the workflow area this PR extends. (role: recent area contributor; confidence: high; commits: f94588d89c2f, f69d9debf487; files: .github/workflows/sweep.yml)
  • brokemac79: Beyond this branch, current-main history shows recent merged adjacent work on PR autoclose and local review paths in the same workflow/source area. (role: recent adjacent contributor; confidence: medium; commits: 84796f0c9b0c, 79e768844291; files: src/clawsweeper.ts, .github/workflows/sweep.yml)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 Urgent regression or broken agent/channel workflow affecting real users now. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. labels Jun 27, 2026
@brokemac79

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@brokemac79 brokemac79 marked this pull request as ready for review June 27, 2026 14:26
@brokemac79 brokemac79 requested a review from a team as a code owner June 27, 2026 14:26
@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jun 27, 2026
@RomneyDa RomneyDa force-pushed the codex/apply-queue-rotation branch from 10ad04b to 9b07ea3 Compare July 1, 2026 09:10
@RomneyDa RomneyDa merged commit 5bdf6f3 into openclaw:main Jul 1, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. P1 Urgent regression or broken agent/channel workflow affecting real users now. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants