Skip to content

Support optional schema creation in database migrators#4911

Merged
mauriceyap merged 5 commits into
masterfrom
schema-creator
May 18, 2026
Merged

Support optional schema creation in database migrators#4911
mauriceyap merged 5 commits into
masterfrom
schema-creator

Conversation

@mauriceyap
Copy link
Copy Markdown
Collaborator

Add an opt-in MigrationConfig to the scheduler and lookout migrators that, when configured, ensures the target schema exists and runs migrations inside it. Supports a two-role pattern where a separate "schema-creator" role runs CREATE SCHEMA and GRANT, while a least-privileged migrator role runs the DDL. Identifier interpolation goes through pgx.Identifier.Sanitize() with a non-empty + 63-byte length check.

Default behaviour is unchanged when MigrationConfig is unset.

mauriceyap and others added 2 commits May 14, 2026 16:25
Add an opt-in MigrationConfig to the scheduler and lookout migrators
that, when configured, ensures the target schema exists and runs
migrations inside it. Supports a two-role pattern where a separate
"schema-creator" role runs CREATE SCHEMA and GRANT, while a
least-privileged migrator role runs the DDL. Identifier interpolation
goes through pgx.Identifier.Sanitize() with a non-empty + 63-byte
length check.

Default behaviour is unchanged when MigrationConfig is unset.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Maurice Yap <mauriceyap@hotmail.co.uk>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 14, 2026

Greptile Summary

This PR introduces optional schema-scoped migrations for the scheduler and lookout migrators via a new MigrationConfig type and PrepareSchema helper. Default behaviour is fully preserved when MigrationConfig is unset.

  • Adds internal/common/database/schema.go with PrepareSchema / MigrationConfig, identifier validation, and a two-role pattern (bootstrap connection creates the schema and grants privileges; the migrator connection runs DDL inside it). Identifier sanitization uses pgx.Identifier.Sanitize() with non-empty and 63-byte length checks.
  • Fixes the pre-existing lookout migrator to use OpenPgxConn (single connection) instead of OpenPgxPool, so SET search_path is session-persistent across all subsequent migration queries and ApplyPartitioner calls.
  • Widens ApplyPartitioner to accept a TxBeginner interface so a *pgx.Conn (or *pgxpool.Pool) can be passed without a pool wrapper.

Confidence Score: 5/5

This PR is safe to merge. The previously flagged pool-vs-single-connection bug in the lookout migrator is correctly resolved, and the new schema creation logic is well-guarded with identifier sanitization and idempotent SQL.

The core concern from the previous review round — lookout's migrate using OpenPgxPool so SET search_path would silently not apply to subsequent migration queries — is properly fixed by switching to OpenPgxConn with a defer db.Close. The new PrepareSchema helper uses pgx.Identifier.Sanitize() for all identifier interpolation, validates both schema and role names against Postgres's 63-byte limit, and the two-role GRANT path is idempotent. The TxBeginner abstraction in ApplyPartitioner is structurally compatible with pgx.TxStarter so it compiles and runs correctly with a *pgx.Conn.

No files require special attention.

Important Files Changed

Filename Overview
internal/common/database/schema.go New file implementing PrepareSchema, createSchema (single- and two-role patterns), and validateIdentifier. Logic is correct: identifiers are sanitized via pgx.Identifier.Sanitize(), the two-role GRANT is idempotent, and the zero-value MigrationConfig is a provably safe no-op.
cmd/lookout/main.go Correctly switched from OpenPgxPool to OpenPgxConn, added defer db.Close, and calls PrepareSchema before UpdateDatabase. search_path is now session-persistent across all migration queries including ApplyPartitioner.
internal/scheduler/database/util.go Migrate now takes *pgx.Conn and MigrationConfig; PrepareSchema is called first. Type narrowing from database.Querier to *pgx.Conn is intentional and correct given the search_path session requirement.
internal/lookouthc/schema/schema.go ApplyPartitioner broadened from *pgxpool.Pool to TxBeginner interface, allowing *pgx.Conn to be passed directly. TxBeginner is structurally identical to pgx.TxStarter so BeginTxFunc accepts it without a cast.
internal/common/database/schema_test.go Comprehensive integration tests covering validation, single-role, two-role, quoted identifiers, idempotency, and no-op paths. acquireConn hijacks a pool connection so SET search_path behaves identically to production.
cmd/scheduler/cmd/migrate_database.go Only the Migrate call signature changed to pass config.Migration. Pre-existing OpenPgxConn usage and the absence of defer db.Close are unchanged from before this PR.
internal/lookout/configuration/types.go Adds Migration database.MigrationConfig field to LookoutConfig. Zero value is safe; no breaking config changes.
internal/scheduler/configuration/configuration.go Adds Migration database.MigrationConfig field to scheduler Configuration. Zero value is safe; no breaking config changes.

Reviews (3): Last reviewed commit: "comments" | Re-trigger Greptile

Comment thread internal/common/database/schema_test.go
Signed-off-by: Maurice Yap <mauriceyap@hotmail.co.uk>
Comment thread internal/common/database/schema.go Outdated
Comment thread internal/common/database/schema.go Outdated
Signed-off-by: Maurice Yap <mauriceyap@hotmail.co.uk>
@mauriceyap mauriceyap merged commit b346a32 into master May 18, 2026
18 checks passed
@mauriceyap mauriceyap deleted the schema-creator branch May 18, 2026 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants