Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
2055eb6
Harden review evidence and adjudication
morgaesis Aug 13, 2026
829f138
Harden bounded review adjudication
morgaesis Aug 13, 2026
8020a4e
Bind refutations to candidate evidence
morgaesis Aug 13, 2026
bd94f7a
Fix qualified identifier repository search
morgaesis Aug 13, 2026
fd2dfd5
Merge qualified identifier search
morgaesis Aug 13, 2026
d537da9
Keep oversized line evidence adjudicable
morgaesis Aug 13, 2026
530c24d
Merge oversized line evidence fix
morgaesis Aug 13, 2026
02a048f
Fix review baseline publication lifecycle
morgaesis Aug 13, 2026
20e51ca
Merge review lifecycle fixes
morgaesis Aug 13, 2026
7e328fb
Correct identifier receipt aggregation test
morgaesis Aug 13, 2026
a133f5a
Reject review-boundary finding language
morgaesis Aug 13, 2026
dd862e3
Preserve unreviewed baseline evidence
morgaesis Aug 13, 2026
d96b130
Preserve pre-provider review errors
morgaesis Aug 13, 2026
55965d2
Merge baseline evidence coverage fix
morgaesis Aug 13, 2026
1686b80
Merge pre-provider error contract fix
morgaesis Aug 13, 2026
58a0f07
Preserve grounded large-diff findings
morgaesis Aug 13, 2026
49374ef
Merge grounded large-diff finding fix
morgaesis Aug 13, 2026
6806a39
Scope adjudication completeness by candidate
morgaesis Aug 13, 2026
7204302
Merge candidate evidence completeness fix
morgaesis Aug 13, 2026
4aeb7b7
Resolve deleted baseline citations
morgaesis Aug 13, 2026
69db5c5
Merge deleted baseline citation fix
morgaesis Aug 13, 2026
a1d02c7
Reserve hosted review time for adjudication
morgaesis Aug 13, 2026
d45ceeb
Merge hosted adjudication timeout budget
morgaesis Aug 13, 2026
65053f8
fix: cover all Rust sources in qualification contract
morgaesis Aug 13, 2026
634311b
Merge complete qualification source contract
morgaesis Aug 13, 2026
a868ff8
Bound local repository object search
morgaesis Aug 13, 2026
77185c7
Merge bounded repository object search
morgaesis Aug 13, 2026
5522f5f
Preserve unresolved review findings
morgaesis Aug 13, 2026
a2eedd2
Merge unresolved finding lifecycle fix
morgaesis Aug 13, 2026
6dc7e6c
Harden review evidence lifecycle
morgaesis Aug 13, 2026
e066b25
Align hosted plan budget regression
morgaesis Aug 13, 2026
afdabbc
Keep adjudicated findings authoritative
morgaesis Aug 13, 2026
82018a9
Bind rewritten evidence to exact lines
morgaesis Aug 13, 2026
9e74ede
Bind evidence to candidate windows
morgaesis Aug 13, 2026
3d55035
Reject meaningless adjudication evidence
morgaesis Aug 13, 2026
3e05c17
Bind adjudication evidence to current coordinates
morgaesis Aug 13, 2026
0e1e977
Harden evidence receipts and release provenance
morgaesis Aug 13, 2026
4071e58
Preserve reviewed citation provenance
morgaesis Aug 13, 2026
8c46e63
Keep grounded findings beyond query metadata limits
morgaesis Aug 13, 2026
6159c9e
Bind repository evidence to immutable objects
morgaesis Aug 13, 2026
6a86fad
Cover embedded policy in review qualification
morgaesis Aug 13, 2026
5e7f9ce
Verify repository evidence object chains
morgaesis Aug 13, 2026
e1112da
Align qualification contract consumers
morgaesis Aug 13, 2026
83c145d
Preserve findings with bounded citations
morgaesis Aug 13, 2026
107d448
Document unresolved finding preservation
morgaesis Aug 13, 2026
e649740
Fail closed on adjudication identity loss
morgaesis Aug 13, 2026
cf79155
Assert preserved prompt-injection blockers
morgaesis Aug 13, 2026
43aa402
Preserve blockers through adjudication failure
morgaesis Aug 13, 2026
65126c7
Keep adjudication regressions independent
morgaesis Aug 13, 2026
80c3933
Keep incomplete adjudication fail closed
morgaesis Aug 13, 2026
67a0d2a
Honor advisory provider failures during adjudication
morgaesis Aug 13, 2026
ab1bd6a
Distinguish adjudication outages from invalid output
morgaesis Aug 13, 2026
fa812f9
Accept fenced adjudication arrays
morgaesis Aug 13, 2026
720dc80
Preserve findings when confirmation rewrites are invalid
morgaesis Aug 13, 2026
0eb8caf
Assert fail-safe confirmation demotion
morgaesis Aug 13, 2026
2dea384
Validate duplicate identities before demotion
morgaesis Aug 13, 2026
1343e46
Record adjudication-aware review baseline
morgaesis Aug 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ jobs:
- run: cargo fmt --check
- run: cargo clippy --quiet --all-targets -- -D warnings
- run: cargo test --quiet
- run: cargo build --quiet --features qualification-candidate
- run: cargo test --quiet --features qualification-candidate
- run: cargo build --quiet --release

musl:
Expand Down
42 changes: 37 additions & 5 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ jobs:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
fetch-depth: 0
- name: Require release commit on main
run: |
git fetch --no-tags origin main
git merge-base --is-ancestor "$GITHUB_SHA" origin/main || {
echo "::error::Release tag commit is not reachable from main."
exit 1
}
- uses: dtolnay/rust-toolchain@4be7066ada62dd38de10e7b70166bc74ed198c30 # stable
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
Expand All @@ -36,7 +43,7 @@ jobs:
bench-live:
needs: validate-tag
runs-on: ubuntu-latest
timeout-minutes: 30
timeout-minutes: 90
permissions:
contents: read
steps:
Expand Down Expand Up @@ -68,17 +75,31 @@ jobs:
exit 1
}
# A key with a spent limit authenticates and then fails every call.
remaining="$(jq -r '.data.limit_remaining // "unlimited"' /tmp/openrouter-key.json)"
if [[ "${remaining}" != "unlimited" ]] && \
awk "BEGIN { exit !(${remaining} < 1) }"; then
if ! jq -e \
'.data.limit_remaining == null or (.data.limit_remaining | type == "number")' \
/tmp/openrouter-key.json >/dev/null; then
echo "::error::OpenRouter returned an invalid credit limit." >&2
exit 1
fi
if jq -e '.data.limit_remaining != null and .data.limit_remaining < 1' \
/tmp/openrouter-key.json >/dev/null; then
remaining="$(jq -r '.data.limit_remaining' /tmp/openrouter-key.json)"
echo "::error::OPENROUTER_API_KEY has ${remaining} credit remaining, below the cost of one benchmark run."
exit 1
fi
remaining="$(jq -r '.data.limit_remaining // "unlimited"' /tmp/openrouter-key.json)"
echo "OpenRouter credential accepted (remaining: ${remaining})."

# Plain release profile: the same build the release job ships, so the
# gate measures the binary users will actually get.
- run: cargo build --quiet --release
- name: Preserve benchmarked Linux binary
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: benchmarked-x86_64-unknown-linux-gnu
path: target/release/postil
if-no-files-found: error
retention-days: 1
- name: Install benchmark dependencies
working-directory: bench
run: bun install --frozen-lockfile
Expand Down Expand Up @@ -143,7 +164,18 @@ jobs:
- run: cargo test --quiet
# SHA-pinned: this action handles the OIDC token.
- uses: sigstore/cosign-installer@398d4b0eeef1380460a10c8013a76f728fb906ac # v3
- run: cargo build --quiet --release --target ${{ matrix.target }}
- name: Download benchmarked Linux binary
if: matrix.target == 'x86_64-unknown-linux-gnu'
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: benchmarked-x86_64-unknown-linux-gnu
path: target/${{ matrix.target }}/release
- name: Restore benchmarked binary mode
if: matrix.target == 'x86_64-unknown-linux-gnu'
run: chmod 0755 target/${{ matrix.target }}/release/postil
- name: Build release binary
if: matrix.target != 'x86_64-unknown-linux-gnu'
run: cargo build --quiet --release --target ${{ matrix.target }}
- name: Package, checksum, and sign
env:
COSIGN_YES: "true"
Expand Down
107 changes: 77 additions & 30 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@ behavior; `src/envelope.rs` is the envelope contract shared with the hosted work
```
acquire diff --> parse supported lockfiles --> parse + index --> bounded evidence batches
|
envelope <-- gate <-- reconcile <-- filter + aggregate <-- model + final synthesis
(cascade/consensus)
envelope <-- gate <-- reconcile <-- score <-- adjudicate <-- filter + aggregate <-- model
| (cascade/consensus)
repository search
│ (baseline) (ground, policy)
├─ stdout JSON (--output-json)
├─ terminal (stderr)
Expand All @@ -22,9 +23,10 @@ acquire diff --> parse supported lockfiles --> parse + index --> bounded evidenc
the model is allowed to cite. Cargo, npm package-lock/shrinkwrap, Yarn v1/Berry,
pnpm, and Go checksum lockfiles become bounded,
format-specific added-package-version and removed-package-version metadata;
malformed, unsupported, or over-budget lockfile sections fail closed. Every other
path remains untrusted reviewable source, including generated,
distribution, vendor, snapshot, and dependency-directory names. Source evidence
malformed, unsupported, or over-budget lockfile sections fail closed. Ignore patterns
remove matching paths before grounding or review planning. Every other path remains
untrusted reviewable source, including generated, distribution, vendor, snapshot, and
dependency-directory names. Source evidence
splits at file and hunk boundaries with overlap, repeats a bounded changed-file
manifest, segments oversized lines without hiding the tail, and records deletion,
binary, rename, mode, and dependency evidence under `.postil/change-metadata`.
Expand All @@ -42,28 +44,36 @@ acquire diff --> parse supported lockfiles --> parse + index --> bounded evidenc
secret-redaction semantics remain shared. Optional private-endpoint authentication is
a separate header whose name cannot collide with provider-managed headers. A shared
admission ledger reserves each HTTP attempt, exact serialized JSON request bytes,
maximum output,
and worst-case token spend before sending any initial, retry, repair, planner,
scorer, or mention-response call. Hosted preflight covers every active cascade or
consensus model, schema and semantic repair, transport retry, output bound, and
admitted-model price bound before the first provider request. Exact serialization
includes JSON expansion for quotes, backslashes, and control characters.
maximum output, and worst-case token spend before sending it. Hosted preflight
covers every active cascade or consensus model, the largest initial or correction
request shape, the logical-call ceiling, output bounds, and admitted-model prices
before the first provider request. Preflight reserves maximum bounded uncertainty
resolution and finding-compression request shapes. Transport retries reserve their
exact exposure atomically at runtime. Exact
serialization includes JSON expansion for quotes, backslashes, and control characters.
Every provider HTTP call has a model-usage record with a product role, logical phase,
operation-wide call ordinal, phase-local attempt, token counts, and exact-cost source.
OpenRouter's response `usage.cost` is preserved as canonical decimal dollars without
floating-point conversion; rounded micro-dollars remain a display/index field.
Endpoints that omit cost retain token-only accounting with unavailable provenance.
- `review.rs`: orchestration; enforces acquisition, model-aware context, request,
provider-attempt, output-token, and worst-case token-exposure budgets before calls;
one UTF-8 byte counts as one projected token rather than using an optimistic ratio;
reviews every evidence batch outside hosted inference. A diff that materializes more
than 24 source batches enters a deterministic large-review route on every surface,
including `--diff-file`: at most 24 requests, at most four concurrent requests, and
an exact hunk receipt committed by SHA-256 before provider contact. Security,
one UTF-8 byte counts as one projected token rather than using an optimistic ratio.
A diff that exceeds its selected-request capacity enters a deterministic large-review
route on every surface, including `--diff-file`. Non-hosted execution selects at most
24 requests. Hosted execution lowers that ceiling when the configured generator,
consensus, scorer, uncertainty-resolution, and finding-compression fan-out needs
fewer requests to stay inside the 64-call watchdog plan. The route uses at most four
concurrent provider calls; consensus reduces batch concurrency so combined model
fan-out stays within that limit. It commits an exact hunk receipt by SHA-256 before provider
contact. Security,
authorization, configuration, policy, billing, migration, release-control, and
executable vendor hunks require direct source evidence. Exact-evidence summaries are
limited to supported dependency metadata, provenance-bound generated output, and
low-risk non-security churn. Missing or invalid receipt coverage fails closed. When
executable vendor hunks require direct source evidence. Low-risk hunks receive
semantic credit only from selected proof batches that retain the exact repository
path, hunk identity, every changed line, and a substantive added line in the final
model request. Missing
direct capacity, proof evidence, or complete receipt coverage fails before plan
registration or provider contact. When
`POSTIL_LARGE_REVIEW_PLAN_ENDPOINT` and `POSTIL_LARGE_REVIEW_PLAN_TOKEN` are set, the
CLI registers a versioned deterministic request plan with the authenticated loopback
endpoint before any provider call. A missing, rejected, or unreachable registration
Expand All @@ -82,6 +92,20 @@ acquire diff --> parse supported lockfiles --> parse + index --> bounded evidenc
the envelope; and owns check-run lifecycle ordering (checks are created before the model
runs so a crash can still be reported against them). It persists safe structured model
incidents for monitoring without raw provider or model text.
Repository-wide absence and mismatch claims declare bounded typed queries for named
resources, values, versions, paths, and identifiers. One receipt binds those queries to
the immutable reviewed head and reports `complete`, `unavailable`, or `exhausted`.
Only a complete receipt with no positive counterexample supports a universal claim.
Incomplete repository evidence cannot confirm it, so the original finding remains open.
Every surviving generated candidate, plus applicable baseline candidates during a full
rereview, enters one bounded adjudication operation before scoring and publication. The
operation admits the complete candidate set or fails closed before provider contact. Its
direct-source receipt hashes and scans the complete diff, records deterministic citation
occurrence counts, and carries only bounded evidence windows to the model. Adjudication
validates exact candidate identities, result completeness, evidence, publication text,
and duplicate primaries. Later and cross-file evidence can refute stale claims, while
unresolved claims remain open. Semantic duplicates collapse across files and kinds
only when one established defect remains; distinct defects sharing a line remain separate.
- `forge/`: trait + GitHub, GitLab, Bitbucket Cloud, and Azure DevOps implementations,
with self-managed base URLs where the same API contract applies. Paginated forge
metadata has aggregate byte and changed-file bounds. Source responses stream to
Expand All @@ -100,6 +124,14 @@ acquire diff --> parse supported lockfiles --> parse + index --> bounded evidenc
GitLab full reviews require a collected diff version whose `real_size` matches the
exhausted paginated file count, then reconstruct source from base/head content;
incremental compares reject `compare_timeout`.
GitHub repository evidence resolves the reviewed commit to its exact tree and streams blobs
within one aggregate budget for requests, objects, bytes, and elapsed time. Local base reviews read tree and blob
objects from the exact committed head. Staged reviews bind the diff and repository search to
one immutable index tree created by `git write-tree`; arbitrary diff files have no proven
repository snapshot and cannot support repository-dependent findings. Snapshot digests include
object mode, path, blob object ID, and gitlink path and object ID. Symlink blobs are searched as
link text and are never followed. A snapshot containing a gitlink remains incomplete because
content inside the referenced repository is outside the snapshot.
- `respond.rs`: interactive bot (`postil respond`): answers an @postil mention on a PR
or issue, grounded in the diff/issue, and posts one reply. Review-and-answer only; it
never opens PRs or pushes commits.
Expand Down Expand Up @@ -199,10 +231,11 @@ additions (violations are `kind: contentPolicy`). Content policy is on by defaul
8. `humanEscalation` blocks by kind at confidence 0.30 or above. It represents an
irreducible owner decision, not uncertainty about a concrete defect. Admin overrides
apply to that kind-only decision rather than ordinary risk findings.
9. Review resource or request-budget exhaustion cannot produce a clean verdict. It
emits the generic internal `Review incomplete` operational finding before any model
call, preserves the hosted global deadline, and keeps full-review reconciliation
untrustworthy.
9. Review resource or request-budget exhaustion cannot produce a clean verdict. An
incomplete deterministic receipt fails before durable plan registration or model
contact. Other preflight exhaustion emits the generic internal `Review incomplete`
operational finding without model contact. Both preserve the hosted global deadline
and keep full-review reconciliation untrustworthy.
10. Bounded JSON metadata pages use a 32 MiB per-page cap and a 64 MiB aggregate
metadata cap. Source files and reconstructed diffs stream beyond that page limit
into one 512 MiB operation workspace shared by acquired source snapshots,
Expand All @@ -211,21 +244,35 @@ additions (violations are `kind: contentPolicy`). Content policy is on by defaul
Supported lockfile sections compact independently and have a 16 MiB per-section cap.
11. Every configured chain is planned against the smallest conservative model context.
A review admits at most three models per logical request and hard-caps logical
requests, HTTP attempts including repair/retry paths, per-response output tokens,
planner and scorer input, worst-case token exposure, and projected cost across
cascade or consensus before provider contact.
requests including repair and bounded post-processing paths, per-response output
tokens, planner and scorer input, worst-case token exposure, and projected cost
across cascade or consensus before provider contact. Each transport retry reserves
its exact attempt, input, output, and cost exposure against the same hard limits.
12. Every completed review envelope records source-batch coverage when batching runs.
Deterministic large reviews also record a plan hash and direct, semantic, and
unreviewed hunk counts. Every normalized hunk has exactly one disposition; evidence
identifiers bind the exact hunk digest, and any unreviewed hunk fails the gate.
Semantic coverage cannot resolve baseline findings. Bounded reviews expose selected
and total source-batch counts in compact output. Planner fallback remains audit
metadata and does not expose provider failure details to a PR.
identifiers bind the exact hunk digest, and any unreviewed hunk rejects the plan
before registration or provider contact. Semantic coverage cannot resolve baseline
findings. Bounded reviews expose selected and total source-batch counts in compact
output. Planner fallback remains audit metadata and does not expose provider failure
details to a PR.
13. Operational and provider virtual anchors expire after each run. Reviewable
PR-description and change-metadata anchors carry across unrelated incremental
reviews, and a same-head rerun with either anchor falls back to a full review.
Change-metadata supersession requires an exact stable semantic ID; synthetic line
reuse alone cannot clear a baseline finding.
14. Repository-dependent findings never derive universal absence or mismatch from incomplete
context. Every model finding explicitly declares its repository context. Mismatch refutation
requires the named target and compared value in one searched evidence unit. Query, request,
object, tree-depth, byte, deadline, and detailed-match bounds are explicit in the receipt
outcome. Public findings state the repository construct and correction without describing
evidence retrieval boundaries or delegating evidence collection to the author.
15. Finding adjudication performs exactly one logical provider operation over every admitted
candidate. Candidate identities bind the exact snapshot and semantic finding fields.
Adjudication input, output, attempts, deadline, and projected cost are bounded, and results
cannot expand or enter schema repair. Confirmation and refutation require exact supplied
evidence; repository-wide conclusions additionally require a complete receipt for the exact
snapshot. Public rewrites describe only the defect, impact, and correction.

## Residual prompt-injection surface

Expand Down
15 changes: 14 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 6 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "postil-cli"
version = "0.8.12"
version = "0.8.13"
edition = "2024"
description = "Postil: a low-noise AI review gate. Silent on clean PRs, hard gate on real risk."
license = "Apache-2.0"
Expand All @@ -21,6 +21,7 @@ default = []
qualification-candidate = []

[dependencies]
aho-corasick = "1.1.4"
anyhow = "1.0.102"
clap = { version = "4.6.1", features = ["derive"] }
dirs = "6.0.0"
Expand All @@ -33,6 +34,7 @@ owo-colors = "4.3.0"
reqwest = { version = "0.13.4", default-features = false, features = ["json", "rustls", "charset"] }
serde = { version = "1.0.228", features = ["derive"] }
serde_json = { version = "1.0.150", features = ["raw_value"] }
sha1 = "0.11.0"
sha2 = "0.11.0"
similar = "3.1.1"
tempfile = "3.27.0"
Expand All @@ -41,6 +43,9 @@ tokio = { version = "1.52.3", features = ["rt-multi-thread", "macros", "process"
toml = "1.1.2"
yaml_serde = "0.10.4"

[build-dependencies]
serde_json = "1.0.150"

[dev-dependencies]
assert_cmd = "2.2.2"
csv = "1.4.0"
Expand Down
Loading
Loading