Skip to content

feat: migrate the backlog into the per-item store, and gate it against the table (Q889) - #1596

Merged
karlkfi merged 10 commits into
mainfrom
claude/q889-store-migrate
Aug 17, 2026
Merged

karlkfi merged 10 commits into
mainfrom
claude/q889-store-migrate

Conversation

@karlkfi

@karlkfi karlkfi commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Phase 2 of Q889: the backlog becomes 173 files under docs/queue/, one per item, so priority is the rank key rather than a line's position in a shared table. Two sessions taking the top two items now touch two different files.

Follows #1595, which landed phase 1's tooling.

What changed

62 titles were rewritten by hand. The store caps a title at 72 characters and the old table capped only the Notes cell, so 62 of 173 were over, the longest at 130. This was the work; the migration itself was one command. Rewriting is judgement rather than truncation, so each was done individually: where a test identifier was the item's only handle it stayed (Q549 sits at exactly the cap), and where the plan doc or the trigger already named it, it moved there.

The migration is reconciled, order included. The ID sets match both ways, the Queue's 106 rows are the store's first 106 in order, and the remaining 67 are Deferred then Flake watch in order. Order is the claim worth checking, because the table's order is the priority and a count cannot see it scrambled. All 29 flake-watch rows arrive deferred carrying flake, which is the plan's decision 4 satisfied by migrate itself, with no extra work.

queue-drift-check is new, and it is why this could land separately at all. The plan made a drift check the condition for splitting phases 1 and 2 out of the atomic switch. Until phase 3 moves the consumers, the backlog is written down twice and an edit to one side alone is silent in the direction that matters. The check re-runs migrate into a throwaway store and compares the loaded items, so what is compared is queue.py's own reading of the table rather than a second parser free to drift from it. It retires itself once docs/STATUS.md is deleted in phase 6.

Two things that were nearly missed

mdreflow folded the store README's Conventions block into one paragraph, and rule 11 anchors the label vocabulary to a line starting **Labels:**. STATUS.md survives the same treatment only because its equivalent block carries trailing two-space hard breaks. The checker exiting 2 on an unreadable vocabulary is the sole reason this was caught: had it treated that as nothing to check, rule 11 would have passed green while enforcing nothing, for as long as the store exists.

The rules suite only ever filed inline labels: [a, b], which is the one form migrate never writes — every real item carries a YAML block list. The checker parses both, so nothing was broken, but nothing held it to that either. Found by a probe of my own using an inline-only parser, visible only because the reconciliation printed both directions rather than the empty one. A one-sided read would have said "no undeclared labels" and been believed.

Testing

  • make check green.
  • queue.py lint: 173 item(s) OK. check-queue-rules.sh: 173 items, rules 8, 9 and 11.
  • check-queue-drift-test.sh: 12 checks, every field in both directions, plus the two controls they pull against (a rewrapped note is not drift; a README is not an item).
  • The new block-form label case was falsified by deleting the mechanism: with the block branch removed it goes red and both inline cases stay green, so it is genuinely the only coverage for the shape the store is written in.
  • The reflow's round-trip claim was verified by rendering the whole store before and after and diffing, with a control asserting the diff could see a change, rather than by reading read_item.

What is not in this PR

Phases 3 to 6: switching the 53 consumers, rewriting 218 anchors, the /dev/queue/ render, and deleting docs/STATUS.md. The drift gate holds the interim.

@karlkfi
karlkfi added this pull request to the merge queue Aug 17, 2026
@karlkfi
karlkfi removed this pull request from the merge queue due to a manual request Aug 17, 2026
karlkfi added 10 commits August 16, 2026 23:42
The per-item store caps a title at 72 characters, because a title renders
whole in every index row, in the kickoff prompt, and in any session named
after the item. The single table capped only the Notes cell, so 62 of 173
titles are over: the longest at 130, and `queue.py lint` fails on each.

Rewriting is judgement rather than truncation. Where a title carried a
test identifier it could not afford, the identifier moves to the linked
plan doc or the trigger, which already name it -- except where the
identifier is the item's only handle, which is why Q549 keeps its spec
name at exactly the cap and Q809, its mode B, now reads as prose.
One file per item under docs/queue/, so priority is the rank key rather
than a line's position in a shared table. Two sessions taking the top two
items now touch two different files.

Reconciled against the table it came from, order included, because the
table's order IS the priority and a count cannot see it scrambled: the ID
sets match both ways, the Queue's 106 rows are the store's first 106 in
order, and the remaining 67 are Deferred then Flake watch in order. All 29
flake-watch rows arrive deferred carrying `flake`, which is decision 4
satisfied by migrate itself.

The store's README declares the closed label vocabulary rule 11 reads.
Thirteen labels are in use across 338 assignments; the shipped 1.0 through
1.5 release gates are retired rather than carried, since no open item can
hold one. docs/queue/** joins both path filter lists now that it exists.
The rules suite only ever filed inline `labels: [a, b]`, which is the one
form `queue.py migrate` never produces: every real item carries a YAML
block list. The checker parses both, but nothing held it to that, so a
regression in the block branch would have passed a green suite while
letting an undeclared label into every migrated item.

Falsified by deleting the mechanism: with the block branch removed the new
case goes red and both inline cases stay green, so it is the only coverage
for the shape the store is actually written in.
The migrated bodies arrive as single-line paragraphs, which md-reflow-check
rejects across all 152 of them. Reflowing is safe because read_item joins
every non-empty body line with a space, so the note it reconstructs is
byte-identical either way; verified by rendering the whole store before and
after and diffing, with a control asserting the diff can see a change.

The README's Conventions block needed the trailing two-space hard breaks
STATUS.md already uses. Without them mdreflow folds the four lines into one
paragraph and rule 11 loses the `**Labels:**` anchor it reads the
vocabulary from -- caught only because the checker exits unmeasurable there
rather than treating an unreadable vocabulary as nothing to check.
The store is 173 items, reconciled against the table it came from by ID
set and by order. The count moved from 61 titles to 62 because the first
reading was taken after Q490 had been fixed in a store that was then
discarded, so it measured the artifact rather than the table.

Two findings worth keeping: the block-list label form migrate writes was
untested by the rules suite, and mdreflow folding the README's Conventions
block silently removes rule 11's anchor.
The plan made a drift check the condition for landing phases 1 and 2 as
their own PR rather than one atomic switch, and until phase 3 moves the
consumers the backlog is written down twice. An edit landing on one side
alone is silent in the direction that matters: a session grooming the table
leaves a store that still reads as authoritative.

The comparison is semantic. It re-runs migrate into a throwaway store and
compares the loaded items, so what gets compared is the tool's own reading
of the table rather than a second parser free to drift from it, and the
reflowed store is invisible to it because read_item joins a body back to one
note first. Rank values are excluded and only the order they produce is
compared: a re-rank changes no priority, and failing it would fire on the
one operation the store exists to allow.

It retires itself, passing and saying so once the table is gone. Twelve
checks cover every field a row carries, in both directions, plus the two
controls those pull against.
173 new pages under docs/queue/ reddened both mkdocs builds: every scope
builds --strict, and validation.nav.omitted_files fails on a published page
that sits in no nav section. Neither gate is in make check, so the branch
was locally green and remotely red.

/queue/ now sits exactly where /STATUS.md and /plan/ sit: excluded from the
stable versions, published on dev, and declared in not_in_nav so the
omitted-files report keeps meaning something.

Verified against the local build rather than the exit code, since excluding
it from every scope would go equally green: stable builds 0 queue pages, dev
builds 176, and /dev/queue/ serves the store's README until phase 5.
The interim window's cost, paid for the first time. #1597 added the row to
docs/STATUS.md while this branch holds the store, so the rebase brought a
table with 174 rows and a store with 173, which queue-drift-check named
precisely rather than leaving to be discovered later.

Ranked awi, between Q780's aw and Q846's ax, so the store's order still
matches the table's. Adding the one file rather than re-running migrate
keeps the diff to it: a regeneration would re-mint all 174 ranks.
@karlkfi
karlkfi force-pushed the claude/q889-store-migrate branch from 9426194 to 8f3c0b0 Compare August 17, 2026 06:45
@karlkfi
karlkfi added this pull request to the merge queue Aug 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 17, 2026
@karlkfi
karlkfi added this pull request to the merge queue Aug 17, 2026
Merged via the queue into main with commit b9a8376 Aug 17, 2026
66 checks passed
@karlkfi
karlkfi deleted the claude/q889-store-migrate branch August 17, 2026 12:56
karlkfi added a commit that referenced this pull request Aug 17, 2026
maintaining-backlog.md still told a contributor "the layout is the single
table", and warned against describing the store before a cutover landed on
main. The store landed on main in phase 2, so the page was stale in exactly
the direction its own rule warns about, and a contributor following it would
edit the table alone and hit queue-drift-check with no explanation of why.

That is not hypothetical: it happened twice in one session, filing Q890 and
Q891, and once more as a DIRTY rebase on #1596.

The section now says both layouts are live, that every filing, closing or
re-rank lands in both places, that queue-drift-check names precisely what is
missing, and that `queue.py migrate` regenerates the store wholesale when that
beats hand-editing. It also says when the double-write ends, which is phase 6.

The caps bullet gains the store's title cap, 72 characters, which the table
never had and which cost 62 rewritten titles to adopt.

Deliberately not rewritten: the merge driver, escaped-pipe, isolated-commit and
Progress sections all describe machinery that is still live and still gated.
Rewording them now to describe a post-table world would make the page wrong
today and would be rewritten again in phase 6, where most of it is deleted
rather than reworded.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant