Skip to content

feat: structural parity guard on every write path (#119, #65) - #128

Merged
mmcky merged 2 commits into
mainfrom
feat-119-structural-parity
Jul 21, 2026
Merged

feat: structural parity guard on every write path (#119, #65)#128
mmcky merged 2 commits into
mainfrom
feat-119-structural-parity

Conversation

@mmcky

@mmcky mmcky commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Closes #119. Closes #65. First item of the P0 corruption-class plan in #120.

What it does

Extracts the structural tokens — fenced-directive openings and (label)= target anchors — from a translation's source and output, and fails the file loudly on divergence. Wired into all three write paths: sync's markdown handler, sync's renamed-file handler, and forward's post-finalization output (so it checks exactly the bytes that would be written).

Every confirmed defect in this class shipped as a success and surfaced weeks later on a downstream strict build. Each one is now a named test:

Defect Shape Caught by
#119 {raw} jupyter{raw} structural args byte-equal
#65 (label)= anchors dropped anchor sequence exact
#118 whole document wrapped in a fence directive count/sequence collapse

The rules were calibrated empirically, not reasoned from the spec

A first draft required byte-equality everywhere. Dry-running it over all 211 source/target lecture pairs across five editions produced 362 false positives in exactly three classes, now admitted as presence-only ("flexible") arguments:

A dropped argument still fails even in the flexible classes. Deliberately byte-equal: solution (its arg is a cross-reference label), raw, include/figure/image paths. After calibration: 200 of 211 pairs pass; the remaining 11 are pending-drift artifacts of the current-vs-current dry-run — a comparison the guard never performs in situ, since it checks one run's input against that run's own output.

It found real production damage during calibration

sympy.fr had dropped its (sympy)= anchor — the only clean #65-class hit in the corpus, invisible to -W because nothing references the anchor yet. Restored in lecture-python-programming.fr#16. One guard, one dry-run, one live defect — on the class where the previous eight instances were all found weeks late, downstream.

Scope boundaries, stated

  • Top-level tokens only. The scanner is a stateful fence walk: nested directives (inside {tab-set} etc.) and directive syntax quoted in documentation examples are invisible. Both sides use the same walker so blind spots cancel, and all three confirmed defect shapes are top-level. Nested coverage belongs to Work plan: suggested order for the post-review backlog #94 Phase 2's round-trip invariant, not this guard.
  • Latent target damage surfaces on next sync. Section-based sync carries unchanged target sections forward; if a file's historical translation already dropped anchors, its next sync fails the guard until repaired. That is loud-by-design (forward resync stripped a MyST directive argument: {raw} jupyter -> {raw} (sir_model, v0.17.0) #119: "fail the file loudly"), and resync is the repair tool — but the first sync wave after this ships may fail files that were already broken, which is the guard doing its job on backlog.
  • src/index.ts untouched — the guard lives in src/structural-parity.ts per the Stage-2 rule that guards go in modules Jest can load.

Validation

18 new tests (each defect shape by issue number + the legitimate-translation cases that must not trip), 50 suites / 1,171 passing, all five gates clean, corpus dry-run as above. A harness end-to-end can't easily manufacture a violating model response on demand; the corpus dry-run is the field validation for this one, and the first real sync/resync wave is the live test.

🤖 Generated with Claude Code

The silent-corruption class (#118, #119, #65) is structural MyST mutated
by a path that should have passed it through verbatim, with the run
reporting success every time. This adds the cheap deterministic guard
#119 proposed: extract the structural tokens — fenced-directive
openings and (label)= target anchors — from source and output, and fail
the file loudly on divergence.

Wired into all three write paths: sync's markdown handler, sync's
renamed-file handler, and forward's post-finalization output (checking
exactly the bytes that would be written). A violation throws in sync
(existing failure handling opens an issue) and errors the file in
forward.

Comparison rules, calibrated against the real corpus rather than
reasoned from the spec:

- Directive name sequence must match; #118's fence-wrap collapses the
  whole sequence and fails on count.
- Structural arguments (raw format, include/figure paths, solution's
  exercise label) must match byte-for-byte — the #119 defect.
- Flexible arguments only need presence to match: admonition/prf:*
  titles and {contents}/{index} display text are legitimately
  translated, and {code-cell}'s kernel tag legitimately differs between
  editions (target frontmatter is ground truth by design, #105/#108).
  A dropped argument still fails.
- Anchor sequence must match exactly — labels are cross-reference
  targets, never translated (#65).

The calibration was empirical: a first byte-equal-everything draft was
dry-run over all 211 source/target lecture pairs across five editions
and produced 362 false positives in exactly the three flexible-arg
classes above. After calibration, 200 of 211 pairs pass; the remainder
are pending-drift artifacts (current-vs-current comparison, which the
guard never performs in situ — it compares one translation run's input
with its own output) plus one genuine production find: sympy.fr had
dropped its (sympy)= anchor, restored in
lecture-python-programming.fr#16.

Scanning is a stateful fence walk, top-level only: nested directives
and directive syntax quoted in documentation examples are invisible,
deliberately — both sides use the same walker, so blind spots cancel,
and all three confirmed defect shapes are top-level tokens.

Lives in src/structural-parity.ts per the Stage-2 rule that guards go
in modules Jest can load; 18 tests cover each confirmed defect shape by
issue number and the legitimate-translation cases that must not trip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 21, 2026 04:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a structural-parity guard to prevent “success-shaped” silent corruption by asserting that key MyST structural tokens (top-level fenced-directive openings and (label)= target anchors) survive translation/resync unchanged, failing loudly when they diverge.

Changes:

  • Introduces src/structural-parity.ts to extract/compare structural tokens and format actionable failure messages.
  • Wires the guard into sync write paths (normal markdown + renamed-file handling) and the forward resync write path (post-finalization).
  • Adds Jest coverage for confirmed defect shapes (#118/#119/#65) and updates dist-action/ output.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/sync-orchestrator.ts Runs structural parity check after validateMyST() in both standard and renamed-file sync processing paths.
src/structural-parity.ts Implements token extraction (stateful fence walk), parity comparison, and formatted violations for logs/errors.
src/cli/commands/forward.ts Runs parity check after finalizeResyncContent() so it validates exact output bytes before writing/PR creation.
src/tests/structural-parity.test.ts Adds targeted unit tests for known corruption shapes and allowed “flexible arg” cases.
dist-action/index.js Bundled build output reflecting the new parity guard and call sites.

Comment thread src/structural-parity.ts
Copilot caught that the missing/invented computation ignored
multiplicity: a duplicated label with one copy dropped reported as
"same labels, different order" — misleading exactly when someone is
diagnosing a parity failure. Detection was unaffected (the sequence
compare fires either way); only the message was wrong.

Now a per-label count diff: dropped copies report as missing (with a
xN count when more than one), surplus copies as not-in-source, and the
different-order message is reserved for the case where the multisets
genuinely match. Duplicate labels are themselves a source defect, but
the guard's diagnosis must not depend on the source being clean.

Two tests pin the shapes includes() got wrong: a dropped duplicate must
name the label rather than claim reordering, and output-side surplus
must carry its count.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mmcky
mmcky merged commit 9916272 into main Jul 21, 2026
1 check passed
@mmcky mmcky mentioned this pull request Jul 21, 2026
5 tasks
mmcky added a commit that referenced this pull request Jul 21, 2026
Promote [Unreleased] to [0.20.0]: #128 closing #119 and #65 — the
structural parity guard on every write path, calibrated against all 211
real source/target lecture pairs, plus the #127 docs wording fix.

Minor bump: new behaviour (files can now fail that previously passed),
though only when the output genuinely diverges structurally from the
source — which was always a defect, just a silent one.

Bump package.json/package-lock, copilot-instructions version + counts
(50 suites), record the release in .dev/STATE with the operational note
that the first wave may loudly fail files carrying pre-existing damage.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants