Skip to content

perf(channels): shard outbound dispatch per conversation#1464

Open
connermo wants to merge 1 commit into
nextlevelbuilder:devfrom
connermo:feat/outbound-dispatch-sharding
Open

perf(channels): shard outbound dispatch per conversation#1464
connermo wants to merge 1 commit into
nextlevelbuilder:devfrom
connermo:feat/outbound-dispatch-sharding

Conversation

@connermo

Copy link
Copy Markdown

Problem

Manager.dispatchOutbound ran as a single goroutine calling channel.Send() synchronously. Every reply in the process — every channel, every user — queued behind the slowest send.

Two consequences:

  • Blast radius. One media upload taking a few seconds froze all outbound traffic for that long, across unrelated channels.
  • Throughput ceiling. Sustained delivery was capped at one Send round-trip (~100-200ms for a text send), regardless of how much LaneMain capacity was configured.

Why serialization was not required

The ordering that loop actually protected is per-conversation: a run emits block replies, a possible Provider busy, retrying... notice, and the final answer to one ChatID, and those must arrive in that order. There are 41 PublishOutbound call sites, several of which target the same chat within one run.

A global order was never needed. Ordering across different conversations carries no meaning.

Change

Dispatch now shards on channel + ChatID:

  • A conversation always maps to the same shard and is delivered serially there → ordering guarantee unchanged.
  • Unrelated conversations proceed in parallel.
  • GOCLAW_OUTBOUND_SHARDS tunes worker count (default 8).

The channel name is part of the shard key because ChatIDs are only unique within a channel.

Temp media needed a real fix alongside it

Serial dispatch deduped temp media implicitly: the first delivery removed the file, and a later message carrying the same path found os.Stat failing and skipped it. The existing comment ("already sent by another dispatch") states the assumption.

Across concurrent shards that check alone is a TOCTOU race — two shards can both stat a live file and upload it twice. Claims are now taken atomically and released after the send. Non-temp (workspace) media passes through unclaimed, since it is never removed after delivery.

Also included

Pool limits that bound this same path, made configurable with defaults unchanged:

Env Default
GOCLAW_PG_MAX_OPEN_CONNS 25
GOCLAW_PG_MAX_IDLE_CONNS 10
GOCLAW_HTTP_MAX_IDLE_CONNS 100
GOCLAW_HTTP_MAX_IDLE_CONNS_PER_HOST 10

The per-host limit binds when one provider takes nearly all traffic: every concurrent request past it pays a fresh TCP+TLS handshake instead of reusing a pooled connection. A deployment raising LaneMain needs to be able to lift it. Self-hosted / in-network providers have no reason to be throttled at 10.

Tests

internal/channels/dispatch_shard_test.go:

  • shard mapping is stable and bounded; channel name participates in the key
  • 50 messages to one chat arrive strictly in order
  • a stalled conversation does not block an unrelated one — the regression this change exists for; one chat's Send is held hostage and the other must still be delivered
  • 16 goroutines racing to claim the same temp file → exactly one wins
  • non-temp media passes through unclaimed; missing files are skipped
  • releasing drops both the file and the claim, so the claim set stays bounded

Verification

  • go build ./..., go build -tags sqliteonly ./..., go vet ./... — clean
  • go test -race green for channels, scheduler, store/pg, providers

Note: internal/channels/telegram TestDisplayWidth fails on this branch, but it fails identically on a pristine origin/dev worktree — pre-existing and unrelated.

A single dispatch goroutine delivered every reply in the process — every
channel, every user — each Send blocking the next. One media upload of a
few seconds froze all outbound traffic for that long, and sustained
throughput was capped at one Send round-trip regardless of lane capacity.

The ordering that loop actually protected is per-conversation: a run emits
block replies, retry notices and a final answer to one ChatID, and those
must arrive in that order. A global order was never required.

Dispatch now shards on channel+ChatID. A conversation always maps to the
same shard and is delivered serially there, so the ordering guarantee is
unchanged, while unrelated conversations proceed in parallel.
GOCLAW_OUTBOUND_SHARDS tunes the worker count (default 8).

Temp media needed a real fix to go with it. Serial dispatch deduped it
implicitly — the first delivery removed the file, and a later message
carrying the same path found os.Stat failing and skipped it. Across
concurrent shards that check alone is a TOCTOU race, so claims are now
taken atomically and released after the send.

Also makes the pool limits that bound this path configurable, defaults
unchanged: GOCLAW_PG_MAX_OPEN_CONNS / GOCLAW_PG_MAX_IDLE_CONNS, and
GOCLAW_HTTP_MAX_IDLE_CONNS / GOCLAW_HTTP_MAX_IDLE_CONNS_PER_HOST. The
per-host limit binds when one provider takes nearly all traffic: every
request past it pays a fresh TCP+TLS handshake, which a deployment
raising LaneMain needs to be able to lift.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant