Skip to content

docs(spec-loop): add spec-gap staleness verification spec - #926

Open
justinmclean wants to merge 1 commit into
apache:mainfrom
justinmclean:spec-gap-staleness
Open

docs(spec-loop): add spec-gap staleness verification spec#926
justinmclean wants to merge 1 commit into
apache:mainfrom
justinmclean:spec-gap-staleness

Conversation

@justinmclean

Copy link
Copy Markdown
Member

Summary

Adds tools/spec-loop/specs/spec-gap-staleness.md, a proposed spec for
stopping a ## Known gaps bullet from outliving the gap it describes.

Spec-only. Nothing is implemented by this PR.

Why now

Nine Known-gap bullets across eight specs currently describe capabilities
that have already shipped. Examples verified against main:

  • adapters.md says adapter authoring smoke validation is missing;
    validate_adapter_authoring is called at __init__.py:3710.
  • privacy-llm-gate.md says branch-name confidentiality validation is
    missing; validate_branch_name_confidentiality is called twice.
  • issue-management-family.md and reviewer-routing.md both say the
    reviewer-routing row is missing from docs/modes.md; it is at
    docs/modes.md:85.
  • release-management-lifecycle.md says the audit record schema is
    prose-only; skills/release-audit-report/audit-record-schema.md exists.

This matters beyond tidiness: the plan beat reads Known gaps to select
work, so a stale bullet sends the loop at finished work. There is also a
straight contradiction between triage-mode.md (repo-health is "a
five-skill family") and repo-health-family.md ("six skills deep").

What the spec proposes

  1. An evidence rule on the plan and update beats. A Known-gap bullet
    may only become a work item if the beat cites a call site, current
    file content, or a counted value. The presence of a name is
    explicitly not admissible, since a function defined but never called
    satisfies a name match while the gap stays open.
  2. Optional gap-closes-when: predicates on gap bullets, with a
    three-verb vocabulary (symbol-called, path-exists,
    skill-count) that covers all nine observed cases. A satisfied or
    malformed predicate is HARD; an unannotated bullet is SOFT, so
    nothing breaks on the day this lands.
  3. validate_cross_spec_counts, HARD. Counted claims checked
    against live family: frontmatter, with cross-spec disagreement a
    violation regardless of which side is right.

Bullets whose gap is inherently narrative declare
gap-closes-when: none (narrative). That is the correct annotation for
the eleven "experimental, no adopter pilot has run" bullets, whose truth
depends on facts outside the repository and which should never be
machine-checked.

Reviewer questions

  • validate_cross_spec_counts is proposed HARD and the unannotated-bullet
    advisory SOFT. SOFT is what allowed nine bullets to accumulate, so
    there is a case for making it HARD, at the cost of a blocking backfill
    across all 25 specs.

Generated-by: Claude (claude-opus-5)

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • prek run --all-files passes
  • For Python packages touched: uv run pytest / ruff check / mypy passes
  • For Groovy bridges touched: command-line invocation tested end-to-end
  • For skill changes: eval suite passes for the affected skill
    (PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)
  • For skill behaviour changes: a new or updated eval fixture is included in this PR
    (a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
  • Other:

@justinmclean justinmclean self-assigned this Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant