feat: migrate the backlog into the per-item store, and gate it against the table (Q889) - #1596
Merged
Merged
Conversation
The per-item store caps a title at 72 characters, because a title renders whole in every index row, in the kickoff prompt, and in any session named after the item. The single table capped only the Notes cell, so 62 of 173 titles are over: the longest at 130, and `queue.py lint` fails on each. Rewriting is judgement rather than truncation. Where a title carried a test identifier it could not afford, the identifier moves to the linked plan doc or the trigger, which already name it -- except where the identifier is the item's only handle, which is why Q549 keeps its spec name at exactly the cap and Q809, its mode B, now reads as prose.
One file per item under docs/queue/, so priority is the rank key rather than a line's position in a shared table. Two sessions taking the top two items now touch two different files. Reconciled against the table it came from, order included, because the table's order IS the priority and a count cannot see it scrambled: the ID sets match both ways, the Queue's 106 rows are the store's first 106 in order, and the remaining 67 are Deferred then Flake watch in order. All 29 flake-watch rows arrive deferred carrying `flake`, which is decision 4 satisfied by migrate itself. The store's README declares the closed label vocabulary rule 11 reads. Thirteen labels are in use across 338 assignments; the shipped 1.0 through 1.5 release gates are retired rather than carried, since no open item can hold one. docs/queue/** joins both path filter lists now that it exists.
The rules suite only ever filed inline `labels: [a, b]`, which is the one form `queue.py migrate` never produces: every real item carries a YAML block list. The checker parses both, but nothing held it to that, so a regression in the block branch would have passed a green suite while letting an undeclared label into every migrated item. Falsified by deleting the mechanism: with the block branch removed the new case goes red and both inline cases stay green, so it is the only coverage for the shape the store is actually written in.
The migrated bodies arrive as single-line paragraphs, which md-reflow-check rejects across all 152 of them. Reflowing is safe because read_item joins every non-empty body line with a space, so the note it reconstructs is byte-identical either way; verified by rendering the whole store before and after and diffing, with a control asserting the diff can see a change. The README's Conventions block needed the trailing two-space hard breaks STATUS.md already uses. Without them mdreflow folds the four lines into one paragraph and rule 11 loses the `**Labels:**` anchor it reads the vocabulary from -- caught only because the checker exits unmeasurable there rather than treating an unreadable vocabulary as nothing to check.
The store is 173 items, reconciled against the table it came from by ID set and by order. The count moved from 61 titles to 62 because the first reading was taken after Q490 had been fixed in a store that was then discarded, so it measured the artifact rather than the table. Two findings worth keeping: the block-list label form migrate writes was untested by the rules suite, and mdreflow folding the README's Conventions block silently removes rule 11's anchor.
The plan made a drift check the condition for landing phases 1 and 2 as their own PR rather than one atomic switch, and until phase 3 moves the consumers the backlog is written down twice. An edit landing on one side alone is silent in the direction that matters: a session grooming the table leaves a store that still reads as authoritative. The comparison is semantic. It re-runs migrate into a throwaway store and compares the loaded items, so what gets compared is the tool's own reading of the table rather than a second parser free to drift from it, and the reflowed store is invisible to it because read_item joins a body back to one note first. Rank values are excluded and only the order they produce is compared: a re-rank changes no priority, and failing it would fire on the one operation the store exists to allow. It retires itself, passing and saying so once the table is gone. Twelve checks cover every field a row carries, in both directions, plus the two controls those pull against.
173 new pages under docs/queue/ reddened both mkdocs builds: every scope builds --strict, and validation.nav.omitted_files fails on a published page that sits in no nav section. Neither gate is in make check, so the branch was locally green and remotely red. /queue/ now sits exactly where /STATUS.md and /plan/ sit: excluded from the stable versions, published on dev, and declared in not_in_nav so the omitted-files report keeps meaning something. Verified against the local build rather than the exit code, since excluding it from every scope would go equally green: stable builds 0 queue pages, dev builds 176, and /dev/queue/ serves the store's README until phase 5.
The interim window's cost, paid for the first time. #1597 added the row to docs/STATUS.md while this branch holds the store, so the rebase brought a table with 174 rows and a store with 173, which queue-drift-check named precisely rather than leaving to be discovered later. Ranked awi, between Q780's aw and Q846's ax, so the store's order still matches the table's. Adding the one file rather than re-running migrate keeps the diff to it: a regeneration would re-mint all 174 ranks.
karlkfi
force-pushed
the
claude/q889-store-migrate
branch
from
August 17, 2026 06:45
9426194 to
8f3c0b0
Compare
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Aug 17, 2026
karlkfi
added a commit
that referenced
this pull request
Aug 17, 2026
maintaining-backlog.md still told a contributor "the layout is the single table", and warned against describing the store before a cutover landed on main. The store landed on main in phase 2, so the page was stale in exactly the direction its own rule warns about, and a contributor following it would edit the table alone and hit queue-drift-check with no explanation of why. That is not hypothetical: it happened twice in one session, filing Q890 and Q891, and once more as a DIRTY rebase on #1596. The section now says both layouts are live, that every filing, closing or re-rank lands in both places, that queue-drift-check names precisely what is missing, and that `queue.py migrate` regenerates the store wholesale when that beats hand-editing. It also says when the double-write ends, which is phase 6. The caps bullet gains the store's title cap, 72 characters, which the table never had and which cost 62 rewritten titles to adopt. Deliberately not rewritten: the merge driver, escaped-pipe, isolated-commit and Progress sections all describe machinery that is still live and still gated. Rewording them now to describe a post-table world would make the page wrong today and would be rewritten again in phase 6, where most of it is deleted rather than reworded.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 2 of Q889: the backlog becomes 173 files under
docs/queue/, one per item, so priority is therankkey rather than a line's position in a shared table. Two sessions taking the top two items now touch two different files.Follows #1595, which landed phase 1's tooling.
What changed
62 titles were rewritten by hand. The store caps a title at 72 characters and the old table capped only the Notes cell, so 62 of 173 were over, the longest at 130. This was the work; the migration itself was one command. Rewriting is judgement rather than truncation, so each was done individually: where a test identifier was the item's only handle it stayed (
Q549sits at exactly the cap), and where the plan doc or the trigger already named it, it moved there.The migration is reconciled, order included. The ID sets match both ways, the Queue's 106 rows are the store's first 106 in order, and the remaining 67 are Deferred then Flake watch in order. Order is the claim worth checking, because the table's order is the priority and a count cannot see it scrambled. All 29 flake-watch rows arrive
deferredcarryingflake, which is the plan's decision 4 satisfied bymigrateitself, with no extra work.queue-drift-checkis new, and it is why this could land separately at all. The plan made a drift check the condition for splitting phases 1 and 2 out of the atomic switch. Until phase 3 moves the consumers, the backlog is written down twice and an edit to one side alone is silent in the direction that matters. The check re-runsmigrateinto a throwaway store and compares the loaded items, so what is compared isqueue.py's own reading of the table rather than a second parser free to drift from it. It retires itself oncedocs/STATUS.mdis deleted in phase 6.Two things that were nearly missed
mdreflow folded the store README's Conventions block into one paragraph, and rule 11 anchors the label vocabulary to a line starting
**Labels:**.STATUS.mdsurvives the same treatment only because its equivalent block carries trailing two-space hard breaks. The checker exiting 2 on an unreadable vocabulary is the sole reason this was caught: had it treated that as nothing to check, rule 11 would have passed green while enforcing nothing, for as long as the store exists.The rules suite only ever filed inline
labels: [a, b], which is the one formmigratenever writes — every real item carries a YAML block list. The checker parses both, so nothing was broken, but nothing held it to that either. Found by a probe of my own using an inline-only parser, visible only because the reconciliation printed both directions rather than the empty one. A one-sided read would have said "no undeclared labels" and been believed.Testing
make checkgreen.queue.py lint: 173 item(s) OK.check-queue-rules.sh: 173 items, rules 8, 9 and 11.check-queue-drift-test.sh: 12 checks, every field in both directions, plus the two controls they pull against (a rewrapped note is not drift; a README is not an item).read_item.What is not in this PR
Phases 3 to 6: switching the 53 consumers, rewriting 218 anchors, the
/dev/queue/render, and deletingdocs/STATUS.md. The drift gate holds the interim.