Repository navigation
Conversation
…store and before a rebuild drops the idle one
braedonsaunders
approved these changes
Nov 20, 2026
braedonsaunders
left a comment
Owner
There was a problem hiding this comment.
Thanks @oleedv, this is an excellent catch, and the write-up made it quick to verify: the restore copies base tables but not the view, so it ends up reading the retired copy about half the time, and PostgreSQL then refuses the DROP (2BP01) on every rebuild after. Both halves of the fix are right:
- The restore hook points the view at
rank_sideinside the restore transaction. - The rebuild repairs a database that was restored before this fix, before it drops the idle copy. PostgreSQL goes through
swapRankViewPG; SQLite drops and recreates the view.
I merged it into the 0.3.17 branch, and the catalog suite passes. It ships with v0.3.17; this PR will show as merged once that reaches main.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The popularity rank has two copies,
catalog_rank_aandcatalog_rank_b.catalog_meta.rank_sidenames the live one (the one the triggers maintain), and the viewcatalog_rankreads it. Each daily rebuild fills the idle copy, swaps the view andrank_sidetogether, and empties the retired copy.A logical restore (
store.ImportLogical: a backup restore, orether cluster initcopying a SQLite server into PostgreSQL) copies base tables only. The target keeps the view its migrations created, oncatalog_rank_a, while the restoredcatalog_metamay sayrank_side=b. That happens whenever the source's last rebuild leftblive, so about half the time.catalog_rank(popularity pages, search walks, topic rows) reads the retired copy, which is empty.aas idle and runsDROP TABLE catalog_rank_a. PostgreSQL refuses:cannot drop table catalog_rank_a because other objects depend on it (SQLSTATE 2BP01). Every later rebuild fails the same way, so the rank never recovers. SQLite allows the drop, so there the next rebuild repairs it, a day later.Seen on a server moved from SQLite to PostgreSQL with
ether cluster init: the copy ran 32 minutes after the last SQLite rebuild had madeblive. From the next daily rebuild on, the PostgreSQL log shows theDROPfailure every hour.catalog_rank_aheld 0 rows,catalog_rank_bheld 154,082, and the view readcatalog_rank_a.Fix
catalog_rank_view) points the view at the copy the restoredrank_sidenames, inside the restore transaction.DROP VIEW IF EXISTSplusCREATE VIEWworks on both dialects.rebuildRankchecks the view before it touches the idle copy. If the view reads the idle copy, it points the view back at the live one (swapRankViewPGon PostgreSQL) and logs a warning. Servers that were already restored into this state recover at their next rebuild without manual SQL.Tests
Both tests fail without the change, on SQLite and on PostgreSQL (
ETHER_TEST_POSTGRES):TestLogicalRestoreKeepsTheRankViewOnTheLiveCopy: a catalogue whose live copy isb, exported and imported. Before the change,catalog_rankread 0 of 25 rows after the restore. The next rebuild must also succeed.TestRankRebuildRepairsAViewOnTheIdleCopy: a view left on the retired copy. Before the change, the PostgreSQL rebuild failed with the same2BP01error as the field server.gofmt,go vetandgo test ./internal/catalog/ ./internal/store/ ./internal/app/ ./internal/backup/ ./internal/migrate/ ./internal/clusterconfig/pass on SQLite and PostgreSQL 18, Go 1.27.4.