Guard PR bodies against premature payment wording

[peterxing]

Summary

• Add a payment_status_language check to scripts/submission_quality_gate.py for PR/submission bodies.
• Fail legacy Payout boundary wording and premature submitted-claim status assertions using reserved words such as paid, settled, received, and withdrawable.
• Keep neutral Submission status, technical no-payout-execution scope, and docs/lifecycle wording passing.

Evidence

• Confusion, missing step, stale example, or bug this addresses: Proposed work: guard PR bodies against premature payment-status wording #1107 documents repeated public PR-body wording drift where submitted work used older payment/status phrasing that maintainers had to correct manually.
• Bounty capacity and active attempts/open PRs checked: Bounty MRWK bounty: 100 MRWK - code cleanup and maintainability improvements, round 2 #936 is open on the public GitHub issue and scoped to small maintainability/script improvements; this PR is a focused gate improvement in an existing maintainer-facing script. I also checked recent open issues and Proposed work: guard PR bodies against premature payment-status wording #1107 as the specific source evidence.
• Intended files or surfaces: scripts/submission_quality_gate.py, tests/test_submission_quality_gate.py.
• Expected PR size: small, two files.
• Out of scope: no payout execution, ledger mutation, wallet behavior, treasury behavior, bridge/exchange/off-ramp, price claim, acceptance automation, or admin-token behavior.

Test Evidence

• python -m pytest tests/test_submission_quality_gate.py -q -> 53 passed.
• python -m ruff format --check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py -> 2 files already formatted.
• python -m ruff check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py -> All checks passed.
• git diff --check -> passed.

MRWK

Bounty #936
Refs #1107

Summary by CodeRabbit

• Bug Fixes
  • Submission checks now flag premature payout/payment-status language, helping keep submission wording accurate and consistent.
  • Text that claims a payout is already paid, settled, received, or withdrawable will now be rejected when used in the wrong context.
  • Neutral status wording and legitimate technical/payment execution references continue to pass validation.

[coderabbitai]

Warning

Review limit reached

@peterxing, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 3 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a43db257-47ad-48fd-a73c-120d52805bb8

📥 Commits

Reviewing files that changed from the base of the PR and between f9e7778 and a9ede4a.

📒 Files selected for processing (2)

• scripts/submission_quality_gate.py
• tests/test_submission_quality_gate.py

📝 Walkthrough

Walkthrough

Adds a new payment-status language validation step to the submission quality gate script, using regex patterns to detect premature payout/payment confirmation wording while allowing technical/documentation context. Wires the check into the gate's aggregated evaluation and adds corresponding tests.

Changes

Payment status language guard

Layer / File(s)	Summary
Regex patterns and check function scripts/submission_quality_gate.py	New regex constants detect payout boundary headings, boundary phrasing, prohibited status assertions, and allowlisted context; _payment_status_language_check applies them and returns a fail/pass result.
Gate wiring scripts/submission_quality_gate.py	evaluate_submission() appends the new check's result to the checks list for aggregated status determination.
Test coverage tests/test_submission_quality_gate.py	Existing test updated to expect a passing payment_status_language check; new tests cover failures on boundary/reserved wording and passes on neutral/technical-context wording.

Possibly related issues

• Proposed work: guard PR bodies against premature payment-status wording #1107: This PR adds the same payment-status wording guard (including tests for "Payout boundary" and reserved payout/status language) that the issue requested.

🚥 Pre-merge checks | ✅ 6

✅ Passed checks (6 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is short, concrete, and clearly matches the PR’s new payment-status wording guard for PR bodies.
Description check	✅ Passed	The description follows the repository template and includes summary, evidence, test evidence, and MRWK details.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Mergework Public Artifact Hygiene	✅ Passed	No investment, price, cash-out, or private-security claims appear in the PR description or changed public-facing text; the diff is code/tests only.
Bounty Pr Focus	✅ Passed	Diff is limited to scripts/submission_quality_gate.py and tests/test_submission_quality_gate.py, and the PR description includes validation evidence and test output.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d6d5eac1-3f64-45a5-9bc5-a8e0e6db34e3

📥 Commits

Reviewing files that changed from the base of the PR and between 3bc87d2 and f9e7778.

📒 Files selected for processing (2)

• scripts/submission_quality_gate.py
• tests/test_submission_quality_gate.py

[LarryHu0217]

APPROVE — reviewed current head a9ede4a93efd5eb38d40c8138768171209064bf8 for PR #1192.

Decision signal: this looks merge-ready for the focused payment-status wording guard.

Evidence checked:

• Scope is narrow: scripts/submission_quality_gate.py and tests/test_submission_quality_gate.py only.
• The new gate rejects premature paid/settled/received/withdrawable status assertions before allowlisted docs/proof-backed context can skip the line.
• The CodeRabbit allowlist/assertion loophole is covered by test_submission_quality_gate_fails_reserved_wording_alongside_allowlisted_terms.
• Neutral submission-status wording and technical/docs payment wording remain allowed.
• Hosted Quality, readiness, docs, and image checks is green for this head.

Local validation from the PR checkout:

• git diff --check origin/main...HEAD
• .venv/bin/python -m pytest tests/test_submission_quality_gate.py -q -> 54 passed
• .venv/bin/python -m pytest tests/test_submission_quality_gate.py -k payment_status -q -> 1 passed, 53 deselected
• .venv/bin/python -m ruff check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py
• .venv/bin/python -m ruff format --check scripts/submission_quality_gate.py tests/test_submission_quality_gate.py

No further blocker found from this review.

[qingfeng312]

Approved. I reviewed current head a9ede4a93efd5eb38d40c8138768171209064bf8 for PR #1192.

Evidence checked:

• Diff is limited to scripts/submission_quality_gate.py and tests/test_submission_quality_gate.py.
• The new payment_status_language check is wired through evaluate_submission(), so a failure participates in the aggregate gate status.
• Reserved payment-status assertions are evaluated before the docs/proof-backed allowlist; the regression test for Per the docs, this submission is paid covers that loophole.
• Neutral Submission status wording and technical/docs/no-payout-execution lines remain allowed by targeted tests.
• No wallet, ledger, payment execution, transfer, or treasury mutation path is touched.

Decision: approved.

[wangedmund77-cmyk]

Review for Bounty #1009 at head a9ede4a93efd5eb38d40c8138768171209064bf8.

Verdict: no blocker from diff review.

Evidence checked:

• The new payment-status language gate runs inside evaluate_submission, so it is evaluated alongside bounty refs, evidence, and similar-PR checks.
• The tests cover the heading case, direct reserved-status assertions, reserved wording even when docs are mentioned, neutral submission-status wording, and allowed technical/docs payment wording.
• The assertion regex checks both word orders, which should catch common PR-body phrases like pending claim is paid and paid bounty submission.

Non-blocking suggestion: the allowlist currently operates per line. If maintainers expect multi-line docs/runbook examples, a future test with wrapped documentation wording would help pin that boundary.

[yanyishuai]

Review packet (Bounty #933)

• Reviewed PR: #1192
• Head commit: a9ede4a93efd5eb38d40c8138768171209064bf8
• Files inspected: scripts/submission_quality_gate.py, tests/test_submission_quality_gate.py
• Verdict: Guard PR bodies against premature payment wording — scoped change; mergeable=clean; CI=[('Quality, readiness, docs, and image checks', 'success')].
• Validation: Inspected current-head file list and CI conclusions; no treasury/payout mutation paths in touched files.

Bounty #933