feat(schema): add optional crossImplementation.suiteScenarioIds corpus field#526
feat(schema): add optional crossImplementation.suiteScenarioIds corpus field#526stevenobiajulu wants to merge 2 commits into
Conversation
…s field Corpus entry ids are file/line-derived and unusable as cross-repo join keys, so a tests renderer cannot place per-test "Other implementations" rows next to a safe-docx scenario. Add an optional, additive crossImplementation.suiteScenarioIds field to the corpus contract, authored via a new @suiteScenarioIds JSDoc tag (a comma/whitespace list of join keys, parsed statically), and emitted only when present. The keys are not prose, so they live outside the word-count tagDefinitions and outside the entry narrative object. Ref: #391
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Review: approve-with-nitsAdversarial pass done. The change is additive, optional, well-scoped to #391, and self-consistent. I verified the claimed gates rather than trusting the PR body. Verified locally
No blocking findings. Nits (non-blocking)
|
LLM-Based Quality GateOverall:
Full checklist questions
Estimated cost (this run): $0.0087 — 26,265 input + 291 output tokens (≈4 chars/token) on |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
extractNarrative and extractSuiteScenarioIds each re-implemented the same JSDoc comment loop (split lines, strip the `* ` gutter, match @tag/continuation lines). The sibling derivation could silently drift from the canonical narrative parse. Consolidate both onto a single parseJsDocTags helper so tag recognition has one source of truth, with extractNarrative last-winning per tag and extractSuiteScenarioIds accumulating across repeats. Behavior and schema are unchanged; existing tests pass. Addresses the LLM-gate "re-derived facts vs canonical sources" finding. Ref: #391
|
Addressed the blocking LLM-gate finding ("Re-derived facts vs canonical sources"). |
LLM-Based Quality GateOverall: ✅ PASS (5 pass · 0 warn · 12 skipped · 17 total)
Full checklist questions
Estimated cost (this run): $0.0092 — 27,967 input + 264 output tokens (≈4 chars/token) on |
1 participant
What
Adds an optional, additive
crossImplementation: { suiteScenarioIds: string[] }field to each corpus entry intests-corpus.schema.json, so a downstream tests renderer can join a safe-docx corpus entry to the cross-implementation suite repo's published results JSON.Corpus entry ids are file/line-derived (
scripts/build_tests_corpus.mjs) and unusable as cross-repo join keys; this field supplies stable suite scenario ids for "Other implementations" rows.Changes
packages/test-narrative/src/tagSchema.ts: newSUITE_SCENARIO_IDS_TAG+suiteScenarioIdsSchema(non-empty, unique, trimmed id list). Kept out of the prosetagDefinitions(no word counts) since join keys are not prose.packages/test-narrative/src/astExtractor.ts: parses a new@suiteScenarioIdsJSDoc tag (comma/whitespace-separated, multi-line) intoScenarioEvidence.suiteScenarioIds, separate from thenarrativeobject so it never leaks into the prose schema.scripts/generate_tests_corpus_schema.mjs: adds the optionalcrossImplementationfield +CrossImplementation$defto the generated schema.scripts/build_tests_corpus.mjs: emitscrossImplementationonly when the tag is present, validating ids via the Zod schema with afile:lineerror on failure.tests-corpus.schema.json: regenerated.add-corpus-cross-implementation-join(validated--strict), adding an ADDED requirement under thetest-corpus-narrativecapability.Additive and optional: entries without the field stay valid.
OpenSpec
New capability/behavior on the corpus contract, so a change proposal was scaffolded and validated before coding.
Gates run (all green by exit code)
npm run build -w @usejunior/test-narrative— 0npm run test:run -w @usejunior/test-narrative— 0 (43 tests; new AST + schema cases)npm run lint:workspaces— 0 (warnings pre-existing)npm run check:spec-coverage— 0npm run check:conformance-citations— 0npm run check:conformance-doc— 0node scripts/check_tests_corpus_schema.mjs(drift gate) — 0Reviewer notes
check:tests-corpus-schema) runs only in the release workflow, not on PR CI; verified locally instead. The regeneratedtests-corpus.schema.jsonis committed.build:tests-corpus) needsallure-results/and was not run; population is covered by the in-build Zod validation + ajv spot-check. No test in-repo yet authors@suiteScenarioIds(none correspond to suite scenarios today) — the field stays absent until a test opts in.Ref: #391 (tracking: #283)