feat: batch catalog reads for schema migration planning

[arreyder]

Summary

• Adds CatalogSnapshot type that pre-fetches all catalog metadata (columns, indexes, statistics, storage params) in 4 bulk queries instead of 4 queries per table
• Adds MigrationsWithCatalog() that uses the snapshot for migration planning, functionally identical to Migrations()
• Switches from slow information_schema.columns view to direct pg_attribute queries
• Existing Migrations() API preserved for backward compatibility

Problem

With 200+ models, EnsureSchema runs ~800 sequential catalog queries (4 per table). On a loaded Aurora PG17 instance these take 20-30 minutes total — the dominant cost of schema migration, even when most tables have zero actual DDL to run.

Caller migration

Callers switch from:

for _, model := range models {
    lines, _ := pgdb_v1.Migrations(ctx, conn, model, dialect)
    // execute lines...
}

To:

snap, _ := pgdb_v1.ReadCatalogSnapshot(ctx, conn)
for _, model := range models {
    lines, _ := pgdb_v1.MigrationsWithCatalog(snap, model, dialect)
    // execute lines...
}

Safety audit

Concern	Status
Logic parity with Migrations()	MigrationsWithCatalog is line-for-line identical logic, reads from snapshot instead of DB
New table detection (len(haveCols) == 0)	Works — nil map from snapshot has len() == 0 in Go, same as original empty map
Schema filtering	Improvement — bulk queries all filter on public schema. Original readColumns and readIndexes didn't filter schema
IF NOT EXISTS / IF EXISTS safety	All DDL uses these clauses, so stale snapshot can't cause duplicate creation errors
rows.Err() checking	Improvement — bulk queries check it, original doesn't
Backward compatibility	Migrations() untouched, new API is additive only

Test plan

• Existing unit tests pass
• New unit tests for CatalogSnapshot accessors
• Integration test with real PG (covered by existing example/models tests in CI)
• Validate in c1 repo after vendoring update

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR introduces a bulk-read catalog snapshot mechanism to speed up schema migration planning by prefetching PostgreSQL catalog metadata once and reusing it across many models/tables.

Changes:

• Added CatalogSnapshot + ReadCatalogSnapshot() to fetch columns, indexes, statistics, and storage params in 4 bulk queries.
• Added MigrationsWithCatalog() that computes per-table migration DDL using the snapshot (keeping existing Migrations() API intact).
• Added unit tests for CatalogSnapshot accessor behavior (though they currently don’t exercise MigrationsWithCatalog() itself).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.

File	Description
pgdb/v1/schema_bulk.go	Implements snapshot bulk queries and a snapshot-backed migrations planner API.
pgdb/v1/schema_bulk_test.go	Adds tests around snapshot accessors / missing-table behaviors.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

This test name suggests it covers MigrationsWithCatalog, but it only validates CatalogSnapshot accessor behavior. Renaming it to reflect what it asserts (or adding assertions against MigrationsWithCatalog output) would make the intent accurate.

[Copilot]

MigrationsWithCatalog will panic if the caller passes a nil CatalogSnapshot (snap.columnsForTable dereferences snap). Since this is a new exported API, return an error when snap is nil rather than crashing.

Suggested change

	func MigrationsWithCatalog(snap *CatalogSnapshot, msg DBReflectMessage, dialect Dialect) ([]string, error) {
	func MigrationsWithCatalog(snap *CatalogSnapshot, msg DBReflectMessage, dialect Dialect) ([]string, error) {
	if snap == nil {
	return nil, fmt.Errorf("nil CatalogSnapshot")
	}

[Copilot]

indexesForTable allocates a new empty map on every call for a missing table. Nil maps are safe for len() and lookups, so consider returning nil (or a shared empty map) to avoid per-table allocations during migration planning; similar pattern exists in statsForTable/storageParamsForTable.

Suggested change

	// indexesForTable returns the indexes for a table, or an empty map if none exist.
	func (s *CatalogSnapshot) indexesForTable(tableName string) map[string]struct{} {
	if m := s.indexes[tableName]; m != nil {
	return m
	}
	return make(map[string]struct{})
	}
	// statsForTable returns the statistics for a table, or an empty map if none exist.
	func (s *CatalogSnapshot) statsForTable(tableName string) map[string]struct{} {
	if m := s.stats[tableName]; m != nil {
	return m
	}
	return make(map[string]struct{})
	}
	// storageParamsForTable returns storage parameters for a table, or an empty map if none exist.
	func (s *CatalogSnapshot) storageParamsForTable(tableName string) map[string]string {
	if m := s.storageParams[tableName]; m != nil {
	return m
	}
	return make(map[string]string)
	// indexesForTable returns the indexes for a table, or nil if none exist.
	func (s *CatalogSnapshot) indexesForTable(tableName string) map[string]struct{} {
	return s.indexes[tableName]
	}
	// statsForTable returns the statistics for a table, or nil if none exist.
	func (s *CatalogSnapshot) statsForTable(tableName string) map[string]struct{} {
	return s.stats[tableName]
	}
	// storageParamsForTable returns storage parameters for a table, or nil if none exist.
	func (s *CatalogSnapshot) storageParamsForTable(tableName string) map[string]string {
	return s.storageParams[tableName]

[Copilot]

readAllStorageParams reads reloptions for every relation in the public schema (tables, indexes, sequences, etc.) as long as reloptions is not NULL. Since only table storage params are used later, add a relkind filter (e.g., regular + partitioned tables) to reduce unnecessary rows and memory in the snapshot.

Suggested change

	qb = qb.Where(goqu.L("pg_namespace.nspname = ?", "public"))
	qb = qb.Where(goqu.L("pg_namespace.nspname = ?", "public"))
	qb = qb.Where(goqu.L("pg_class.relkind IN (?, ?)", "r", "p"))

[Copilot]

This test is named as if it validates MigrationsWithCatalog behavior, but it only checks columnsForTable and never calls MigrationsWithCatalog/CreateSchema. Either rename it to match what it asserts, or extend it to call MigrationsWithCatalog and verify it returns CreateSchema DDL for a missing table.

[Copilot]

statsForTable allocates a new empty map on every call when the table is missing from the snapshot. Since the returned map is only used for read-only lookups, returning nil (or a shared empty map) would avoid per-call allocations.

Suggested change

	// statsForTable returns the statistics for a table, or an empty map if none exist.
	func (s *CatalogSnapshot) statsForTable(tableName string) map[string]struct{} {
	if m := s.stats[tableName]; m != nil {
	return m
	}
	return make(map[string]struct{})
	// statsForTable returns the statistics for a table, or nil if none exist.
	func (s *CatalogSnapshot) statsForTable(tableName string) map[string]struct{} {
	return s.stats[tableName]

[Copilot]

storageParamsForTable allocates a new empty map on every call for missing tables. Because nil maps work fine for len() and reads, consider returning nil (or a shared empty map) to reduce allocations when planning migrations for many models.

Suggested change

	// storageParamsForTable returns storage parameters for a table, or an empty map if none exist.
	func (s *CatalogSnapshot) storageParamsForTable(tableName string) map[string]string {
	if m := s.storageParams[tableName]; m != nil {
	return m
	}
	return make(map[string]string)
	// storageParamsForTable returns storage parameters for a table, or nil if none exist.
	func (s *CatalogSnapshot) storageParamsForTable(tableName string) map[string]string {
	return s.storageParams[tableName]