Skip to content

pool/multiplexed: synchronize initial connection setup - #252

Merged
liuzengh merged 1 commit into
trpc-group:mainfrom
qihai-coding:fix/multiplexed-initialization-race
Oct 10, 2026
Merged

liuzengh merged 1 commit into
trpc-group:mainfrom
qihai-coding:fix/multiplexed-initialization-race

Conversation

@qihai-coding

Copy link
Copy Markdown
Contributor

A failed dial during connection initialization can modify the connection slice concurrently with initialization, or remove the node before the connection set is published. Construct the set first, then initialize and publish it under the existing connection-set mutex. Dialing remains asynchronous.

Expand the existing dial-failure test to cover TCP and UDP with the default, one, and four initial connections. Each case makes 100 attempts in the same pool and waits for cleanup before the next initialization.

Validation:

  • The expanded test detects the race on the original implementation.
  • Targeted race tests pass for 20 repetitions each with Go 1.19.13 and 1.27.2 on Linux.
  • Go 1.19.13 connection-pool tests, build, and full main-module test suite pass.
  • Formatting, diff checks, and incremental golangci-lint 1.64.8 pass. The linter uses Go 1.24.1 to load packages.

Fixes #251

RELEASE NOTES: Fix a data race when initial multiplexed connections fail to dial, and ensure failed connection sets are removed from the pool.

原有初始化未持有连接集合互斥锁,并在连接集合写入池前启动异步拨号。快速拨号失败会在后续连接追加期间修改同一切片,还可能先删除节点再发布已经失效的连接集合。

将连接集合构造与初始化分离,在节点初始化入口持有集合锁,完成初始连接创建及池内发布后释放。拨号继续异步执行,正常连接选择与扩容沿用原有锁,公开接口、默认连接数量、重试策略和依赖版本保持不变。

扩展现有拨号失败测试,覆盖流式连接和数据报连接的默认数量、单连接及四连接配置。每组在同一池中重复一百次,关闭成功返回的虚拟连接并取消上下文,以条件等待确认失败节点被清理及后续请求能够重新初始化。

新增测试在原始实现上触发初始化与失败清理的数据竞争;修复后,Go 1.19.13 和 1.27.2 下目标竞态测试各连续通过二十轮。Go 1.19.13 下原有连接池测试、主模块全库构建和测试通过;格式、差异及静态检查通过。静态检查使用 1.64.8 版本工具和匹配的 Go 1.24.1 工具链。

Fixes trpc-group#251

RELEASE NOTES: 修复初始连接拨号失败时的连接集合竞态,保证失效连接集合在发布后能够正确清理。
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@qihai-coding

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

@qihai-coding qihai-coding reopened this Oct 9, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 9, 2026
@trpc-group trpc-group unlocked this conversation Oct 10, 2026
@liuzengh
liuzengh self-requested a review October 10, 2026 03:10

@liuzengh liuzengh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 119a78a2de19953dbfb02174d94397a01914a7d4. This fixes #251: holding the connection-set mutex across both initialization and publication prevents the append/expel race and ensures failed-dial cleanup cannot run before Store. Dialing remains asynchronous, and the existing connection-reuse and expansion paths gain no additional lock operations. I found no new lock-order cycle.

Independent validation on Go 1.24.11, Linux/amd64:

  • The expanded regression test passed with -race -count=20.
  • The issue-style default-pool failed-dial reproducer passed 30,000 iterations with -race.
  • Applying the expanded regression test to the pre-fix implementation reproduced the original data race.
  • go test ./pool/... passed.

The broader go test -race ./pool/... run still reports races in the test TCP server's connection list and TestGetDialCtx's deep comparison of contexts. I reproduced both on base commit 56cbf53; they are pre-existing test issues, not regressions introduced here.

For performance, the additional mutex acquisition is per connection-set initialization, not per request. Local warm-path microbenchmarks showed unchanged allocation counts and no consistent slowdown. Longer cold-start benchmarks encountered loopback TCP dial timeouts, so they do not support a reliable cold-start latency or throughput claim.

No blocking findings. Approve.


已评审 119a78a2de19953dbfb02174d94397a01914a7d4。该修改解决了 #251:连接集合互斥锁同时覆盖初始化与发布,既避免 append 与 expel 并发修改切片,也保证拨号失败后的清理不会早于 Store。拨号仍然异步执行,已有连接复用和扩容路径没有新增锁操作;未发现新的锁顺序循环。

在 Go 1.24.11、Linux/amd64 上独立验证:

  • 扩展后的回归测试以 -race -count=20 连续通过。
  • 按 issue 方式构造的默认连接池拨号失败测试,在 -race 下累计 30,000 次循环通过。
  • 将扩展后的回归测试用于修复前实现,可以复现原始数据竞争。
  • go test ./pool/... 通过。

更大范围的 go test -race ./pool/... 仍报告测试 TCP server 连接列表、以及 TestGetDialCtx 深比较 context 时的竞态。这两项均已在基线 56cbf53 上复现,属于既有测试问题,并非本 PR 引入的回归。

性能方面,新增加锁发生在连接集合初始化时,而非每次请求。本地已有连接复用基准的分配次数不变,未观察到一致的性能下降。较长时间的冷启动基准遇到了 loopback TCP 拨号超时,因此不能据此给出可靠的冷启动延迟或吞吐变化结论。

未发现阻塞合并的问题,同意合入。

@liuzengh

Copy link
Copy Markdown
Contributor

流水线 typo 问题已经在 #254 中修复,与本PR变更无关,同意合入

@liuzengh
liuzengh merged commit 6cb412c into trpc-group:main Oct 10, 2026
7 of 10 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 10, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pool/multiplexed: default initialization races with failed-dial eviction

2 participants