Skip to content

feat: adapt apply scan window#401

Closed
brokemac79 wants to merge 1 commit into
openclaw:mainfrom
brokemac79:codex/adaptive-apply-batch
Closed

feat: adapt apply scan window#401
brokemac79 wants to merge 1 commit into
openclaw:mainfrom
brokemac79:codex/adaptive-apply-batch

Conversation

@brokemac79

@brokemac79 brokemac79 commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

The apply lane can spend an entire automatic cursor window on old, skip-heavy records and close nothing. After #375, the cursor advances, but a fixed 300-record scan window can still make rotation through the open issue/PR universe slower than necessary when the previous window was entirely skipped.

Why This Change Was Made

This adds a bounded adaptive scan-window selector for automatic close-mode apply runs:

  • keeps the normal scan window at 300 records;
  • expands the next automatic scan to 600, then up to a hard cap of 900, only after the previous close-mode cursor window was full, zero-close, and at least 80% skipped;
  • does not change explicit item-number applies, comment-sync-only runs, close limits, checkpoint size, live-state checks, policy gates, or mutation gates;
  • preserves last_close_apply_health so comment-sync/status updates do not hide the previous close-mode signal that drives the next scan-window choice;
  • preserves legacy close health from existing status files whose previous close signal is still only in apply_health.

User Impact

Maintainers get faster rotation through skip-heavy apply windows without increasing how many closes can happen in one run. The workflow status now reports the selected scan window and reason, for example:

Scan window: 600 records (previous_full_zero_close_skip_window)

Evidence

Focused local proof after rebasing onto current origin/main including #398:

pnpm run build:all
node --test test\repair\workflow-utils.test.ts test\sweep-workflow.test.ts test\clawsweeper.test.ts
pnpm run format:check

Result: passed. The focused node test run reported 130 tests, 129 passing, 1 skipped.

Adaptive utility runtime proof on the final build, with current status in a non-close state but last_close_apply_health preserving the previous close signal:

close_processed_limit=600
base_close_processed_limit=300
max_close_processed_limit=900
adaptive_apply_scan=true
adaptive_apply_scan_reason=previous_full_zero_close_skip_window
previous_apply_processed=300
previous_apply_processed_limit=300
previous_apply_closed=0
previous_apply_skipped=280

Full local check:

$env:PNPM_BIN='C:\Users\marti\AppData\Local\Corepack\shims\pnpm.cmd'
pnpm run check

Result: passed. The full check reported 1150 tests, 1149 passing, 1 skipped, coverage gates at 100%, and format clean.

Codex review loop:

codex review -c service_tier='fast' --base origin/main

Result: no actionable correctness issues found. The review noted the adaptive scan window logic, status preservation, workflow wiring, and tests are consistent with the intended behavior.

Screenshots / Visual Proof

No UI surface changes. The operator-visible surface is workflow/status text, shown above as Scan window: 600 records (previous_full_zero_close_skip_window).

Dependency / Stack Note

This is based on current main after #391, #397, #395, and #398. It is independent from #400 and #402.

Risk

The risk is limited to automatic apply scan-window selection. The PR only changes how many candidate records are scanned before selecting close candidates. It does not relax close gates, policy checks, live GitHub-state checks, checkpoint limits, or write permissions.

@brokemac79 brokemac79 requested a review from a team as a code owner July 4, 2026 02:44
@clawsweeper

clawsweeper Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed July 4, 2026, 12:15 AM ET / 04:15 UTC.

Summary
The PR adds a bounded adaptive scheduled close-apply scan window that can expand from 300 to 600 or 900 records after a full, zero-close, skip-heavy cursor window.

Reproducibility: not applicable. this is a feature PR rather than a current-main bug report. The changed behavior is supported by source inspection and PR-body live output instead of a failing reproduction.

Review metrics: 2 noteworthy metrics.

  • Diff size: 7 files changed; +321/-8. The patch is focused but spans workflow, source, docs, and tests, so the apply path should be reviewed end to end.
  • Workflow surface: 1 GitHub Actions workflow changed. The changed behavior affects scheduled apply automation that unit tests cannot fully simulate at production queue volume.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] Merging changes scheduled apply automation so a skip-heavy zero-close close window can cause the next untargeted close apply to inspect up to 900 proposed records instead of the current 300-record window.
  • [P1] The first live scheduled apply after merge should be watched for the selected scan window and reason because the selector depends on newly preserved last close-mode health.

Maintainer options:

  1. Merge with bounded apply monitoring (recommended)
    Accept the capped 900-record scan behavior once ordinary checks are green and watch the next scheduled apply status for the selected window and reason.
  2. Hold the fixed scan window
    Pause or close if maintainers want the 300-record scan to remain a hard default until production queue data justifies adaptive selection.

Next step before merge

  • [P2] No repair lane is needed; maintainers should decide whether to accept the bounded scheduled-apply automation change.

Security
Cleared: The diff changes first-party workflow, TypeScript, docs, and tests without new dependencies, broader permissions, secret handling changes, or third-party code execution.

Review details

Best possible solution:

Land the bounded adaptive selector only if maintainers accept the scheduled-apply throughput tradeoff, while keeping close gates, explicit item applies, checkpoint limits, and policy checks unchanged.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this is a feature PR rather than a current-main bug report. The changed behavior is supported by source inspection and PR-body live output instead of a failing reproduction.

Is this the best way to solve the issue?

Yes, subject to maintainer acceptance; putting the selector in workflow-utils and only feeding untargeted close preselection is a narrow maintainable path that preserves existing close gates.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against d20cf36edb6b.

Label changes

Label justifications:

  • P2: This is a bounded automation throughput improvement with limited blast radius.
  • merge-risk: 🚨 automation: The diff changes scheduled apply scan-window selection and sweep-status fields used by apply automation.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body includes copied live output from the adaptive utility plus build/test/format results for the changed behavior.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes copied live output from the adaptive utility plus build/test/format results for the changed behavior.
Evidence reviewed

What I checked:

  • Repository policy read: AGENTS.md was present and read fully; its automation-safety and throughput/default guidance applies because this PR changes both src/clawsweeper.ts and .github/workflows/sweep.yml. (AGENTS.md:1, d20cf36edb6b)
  • Current main fixed window: Current main still has close_processed_limit=300 and no adaptive apply scan selector or last_close_apply_health field, so the central change is not implemented on main. (.github/workflows/sweep.yml:2266, d20cf36edb6b)
  • PR adaptive selector: The diff adds adaptiveApplyBatchSize, which reads prior close-mode health, requires a cursor-backed full zero-close skip-heavy window, excludes unsafe attention reasons, and caps growth at maxSize. (src/repair/workflow-utils.ts:346, 1d10be8ea9ba)
  • PR workflow scope: The workflow calls adaptive-apply-batch-size only for untargeted non-comment-sync apply runs, then feeds the selected close_processed_limit into proposed close preselection and status text. (.github/workflows/sweep.yml:2314, 1d10be8ea9ba)
  • Status compatibility: The diff preserves last_close_apply_health and falls back from legacy close-mode apply_health so comment-sync or status updates do not hide the previous close-mode signal. (src/clawsweeper.ts:1924, 1d10be8ea9ba)
  • Regression coverage: The PR adds tests for skip-heavy expansion, preserved close health after comment-sync status, cap behavior, productive reset, unsafe attention reset, and missing status fallback. (test/repair/workflow-utils.test.ts:640, 1d10be8ea9ba)

Likely related people:

  • brokemac79: Recent merged work introduced apply cursor rotation, apply health, lane reporting, skip-action classification, and cycle reporting that this PR extends. (role: recent area contributor; confidence: high; commits: 5bdf6f31c904, ead7d6461a6c, 5f4af592da57; files: .github/workflows/sweep.yml, src/clawsweeper.ts, src/repair/workflow-utils.ts)
  • Peter Steinberger: Blame shows the core apply workflow and sweep status writer originated in earlier workflow/status work, with recent scheduler adjacency. (role: original workflow/status contributor; confidence: medium; commits: be44378572ac, 1343d8ec0cef; files: .github/workflows/sweep.yml, src/clawsweeper.ts, src/repair/workflow-utils.ts)
  • Vincent Koc: Blame and history show recent close checkpointing and fixed 300-record processed-limit work in the apply workflow. (role: adjacent apply workflow contributor; confidence: medium; commits: f94588d89c2f; files: .github/workflows/sweep.yml)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@brokemac79 brokemac79 force-pushed the codex/adaptive-apply-batch branch 2 times, most recently from c3218fe to 85911f2 Compare July 4, 2026 02:56
@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. labels Jul 4, 2026
@brokemac79 brokemac79 force-pushed the codex/adaptive-apply-batch branch from 85911f2 to df5c22a Compare July 4, 2026 03:06
@brokemac79 brokemac79 force-pushed the codex/adaptive-apply-batch branch from df5c22a to 1d10be8 Compare July 4, 2026 04:08
@brokemac79

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

@steipete

steipete commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Thanks @brokemac79 — the adaptive scan idea is preserved in #406, with current-main integration, close-health persistence, regression coverage, and the workflow-size repair needed after #402 pushed the inline expression over GitHub's limit.

GitHub rejected the maintainer update to this forked branch because the active OAuth token lacks workflow scope, so I recreated the exact reviewed tree on a maintainer branch and kept your co-author credit. Closing this branch in favor of #406.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants