feat: structural parity guard on every write path (#119, #65) - #128
Merged
Conversation
The silent-corruption class (#118, #119, #65) is structural MyST mutated by a path that should have passed it through verbatim, with the run reporting success every time. This adds the cheap deterministic guard #119 proposed: extract the structural tokens — fenced-directive openings and (label)= target anchors — from source and output, and fail the file loudly on divergence. Wired into all three write paths: sync's markdown handler, sync's renamed-file handler, and forward's post-finalization output (checking exactly the bytes that would be written). A violation throws in sync (existing failure handling opens an issue) and errors the file in forward. Comparison rules, calibrated against the real corpus rather than reasoned from the spec: - Directive name sequence must match; #118's fence-wrap collapses the whole sequence and fails on count. - Structural arguments (raw format, include/figure paths, solution's exercise label) must match byte-for-byte — the #119 defect. - Flexible arguments only need presence to match: admonition/prf:* titles and {contents}/{index} display text are legitimately translated, and {code-cell}'s kernel tag legitimately differs between editions (target frontmatter is ground truth by design, #105/#108). A dropped argument still fails. - Anchor sequence must match exactly — labels are cross-reference targets, never translated (#65). The calibration was empirical: a first byte-equal-everything draft was dry-run over all 211 source/target lecture pairs across five editions and produced 362 false positives in exactly the three flexible-arg classes above. After calibration, 200 of 211 pairs pass; the remainder are pending-drift artifacts (current-vs-current comparison, which the guard never performs in situ — it compares one translation run's input with its own output) plus one genuine production find: sympy.fr had dropped its (sympy)= anchor, restored in lecture-python-programming.fr#16. Scanning is a stateful fence walk, top-level only: nested directives and directive syntax quoted in documentation examples are invisible, deliberately — both sides use the same walker, so blind spots cancel, and all three confirmed defect shapes are top-level tokens. Lives in src/structural-parity.ts per the Stage-2 rule that guards go in modules Jest can load; 18 tests cover each confirmed defect shape by issue number and the legitimate-translation cases that must not trip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a structural-parity guard to prevent “success-shaped” silent corruption by asserting that key MyST structural tokens (top-level fenced-directive openings and (label)= target anchors) survive translation/resync unchanged, failing loudly when they diverge.
Changes:
- Introduces
src/structural-parity.tsto extract/compare structural tokens and format actionable failure messages. - Wires the guard into sync write paths (normal markdown + renamed-file handling) and the forward resync write path (post-finalization).
- Adds Jest coverage for confirmed defect shapes (#118/#119/#65) and updates
dist-action/output.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/sync-orchestrator.ts | Runs structural parity check after validateMyST() in both standard and renamed-file sync processing paths. |
| src/structural-parity.ts | Implements token extraction (stateful fence walk), parity comparison, and formatted violations for logs/errors. |
| src/cli/commands/forward.ts | Runs parity check after finalizeResyncContent() so it validates exact output bytes before writing/PR creation. |
| src/tests/structural-parity.test.ts | Adds targeted unit tests for known corruption shapes and allowed “flexible arg” cases. |
| dist-action/index.js | Bundled build output reflecting the new parity guard and call sites. |
Copilot caught that the missing/invented computation ignored multiplicity: a duplicated label with one copy dropped reported as "same labels, different order" — misleading exactly when someone is diagnosing a parity failure. Detection was unaffected (the sequence compare fires either way); only the message was wrong. Now a per-label count diff: dropped copies report as missing (with a xN count when more than one), surplus copies as not-in-source, and the different-order message is reserved for the case where the multisets genuinely match. Duplicate labels are themselves a source defect, but the guard's diagnosis must not depend on the source being clean. Two tests pin the shapes includes() got wrong: a dropped duplicate must name the label rather than claim reordering, and output-side surplus must carry its count. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mmcky
added a commit
that referenced
this pull request
Jul 21, 2026
Promote [Unreleased] to [0.20.0]: #128 closing #119 and #65 — the structural parity guard on every write path, calibrated against all 211 real source/target lecture pairs, plus the #127 docs wording fix. Minor bump: new behaviour (files can now fail that previously passed), though only when the output genuinely diverges structurally from the source — which was always a defect, just a silent one. Bump package.json/package-lock, copilot-instructions version + counts (50 suites), record the release in .dev/STATE with the operational note that the first wave may loudly fail files carrying pre-existing damage. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jul 21, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #119. Closes #65. First item of the P0 corruption-class plan in #120.
What it does
Extracts the structural tokens — fenced-directive openings and
(label)=target anchors — from a translation's source and output, and fails the file loudly on divergence. Wired into all three write paths: sync's markdown handler, sync's renamed-file handler, and forward's post-finalization output (so it checks exactly the bytes that would be written).Every confirmed defect in this class shipped as a success and surfaced weeks later on a downstream strict build. Each one is now a named test:
{raw} jupyter→{raw}(label)=anchors droppedThe rules were calibrated empirically, not reasoned from the spec
A first draft required byte-equality everywhere. Dry-running it over all 211 source/target lecture pairs across five editions produced 362 false positives in exactly three classes, now admitted as presence-only ("flexible") arguments:
{admonition}/{prf:*}titles and{contents}/{index}display text — legitimately translated (Contents→目录corpus-wide);{code-cell}'s kernel tag — editions legitimately pin different kernels (python3upstream vsipython3across lecture-python.zh-cn), and target frontmatter is ground truth by design (forward --github: resync PRs omit state files and heading maps, and corrupt frontmatter-less files #105/fix(forward): complete + reviewable --github PRs, flag-based discovery, localisation as ground truth #108).A dropped argument still fails even in the flexible classes. Deliberately byte-equal:
solution(its arg is a cross-reference label),raw,include/figure/imagepaths. After calibration: 200 of 211 pairs pass; the remaining 11 are pending-drift artifacts of the current-vs-current dry-run — a comparison the guard never performs in situ, since it checks one run's input against that run's own output.It found real production damage during calibration
sympy.frhad dropped its(sympy)=anchor — the only clean #65-class hit in the corpus, invisible to-Wbecause nothing references the anchor yet. Restored in lecture-python-programming.fr#16. One guard, one dry-run, one live defect — on the class where the previous eight instances were all found weeks late, downstream.Scope boundaries, stated
{tab-set}etc.) and directive syntax quoted in documentation examples are invisible. Both sides use the same walker so blind spots cancel, and all three confirmed defect shapes are top-level. Nested coverage belongs to Work plan: suggested order for the post-review backlog #94 Phase 2's round-trip invariant, not this guard.{raw} jupyter->{raw}(sir_model, v0.17.0) #119: "fail the file loudly"), and resync is the repair tool — but the first sync wave after this ships may fail files that were already broken, which is the guard doing its job on backlog.src/index.tsuntouched — the guard lives insrc/structural-parity.tsper the Stage-2 rule that guards go in modules Jest can load.Validation
18 new tests (each defect shape by issue number + the legitimate-translation cases that must not trip), 50 suites / 1,171 passing, all five gates clean, corpus dry-run as above. A harness end-to-end can't easily manufacture a violating model response on demand; the corpus dry-run is the field validation for this one, and the first real sync/resync wave is the live test.
🤖 Generated with Claude Code