feat(module-postgres): configurable snapshot socket timeout - #739
feat(module-postgres): configurable snapshot socket timeout#739henriquekraemer wants to merge 2 commits into
Conversation
Adds a snapshot_socket_timeout connection option (seconds) for the idle timeout on snapshot connection sockets, defaulting to the previous fixed 30 seconds. When the storage cannot keep up with the snapshot, a storage flush can stall the snapshot loop for longer than the timeout, and the idle timeout kills the source connection mid-snapshot. The snapshot resumes, but each kill costs a retry delay plus re-reading the current chunk. This follows the same pattern as the replication_socket_timeout option proposed in powersync-ja#715, for the snapshot connection.
🦋 Changeset detectedLatest commit: 8d30fca The changes in this PR will be included in the next version bump. This PR includes changesets to release 12 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
# Conflicts: # modules/module-postgres/src/types/types.ts # modules/module-postgres/test/src/config.test.ts
|
Small update from staging: we found a second way to hit this timeout that has nothing to do with storage backpressure. Running with initial snapshot filters (a Postgres port of #502) on a large sparse table without a supporting index, a single chunk query ( The real fix in that case was a partial index matching the filter predicate (2.6s afterwards), but the configurable timeout is what turned an endless retry loop into something we could see and diagnose. We're running with |
1 participant
Adds a
snapshot_socket_timeoutconnection option (in seconds) for the idle timeout on snapshot connection sockets, defaulting to the current fixed 30 seconds. Same pattern as thereplication_socket_timeoutoption proposed in #715, applied to the snapshot connection instead.Background
The snapshot socket timeout is an idle timeout: it fires when the snapshot loop stops consuming the source stream for 30 seconds. The main way that happens is storage backpressure, since a storage flush blocks the chunk loop. We hit this reliably while re-replicating a deployment where the new sync rules instance snapshots while the previous instance keeps streaming on the same storage: on wide tables, a flush under that combined load can take longer than 30s, and the idle timeout kills the source connection mid-snapshot. It shows up either as a socket timeout inside the chunk query or as
postgres query failedon the next chunk, when the timeout fired between queries and destroyed the connection.With resumable snapshots the table resumes from the last key, so no work is lost, but each kill costs the retry delay plus re-reading the current chunk. Snapshot concurrency (#731) makes the window easier to hit, but the exposure exists in the sequential path as well, since flushes have always run between chunk reads.
The default stays at 30 seconds, so behavior is unchanged unless the option is set.
Tests
Added config normalization tests (seconds to milliseconds, unset by default, invalid values ignored), mirroring the tests in #715.