Skip to content

Verify LaneProposal signer and parent hash chain in PushBlock - #2925

Merged
wen-coding merged 6 commits into
mainfrom
wen/fix_laneproposal_verify
Feb 20, 2026
Merged

wen-coding merged 6 commits into
mainfrom
wen/fix_laneproposal_verify

Conversation

@wen-coding

@wen-coding wen-coding commented Feb 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Signer == lane check: PushBlock did not verify that the signer of a LaneProposal matches the block's lane. A malicious validator could craft blocks on another validator's lane, bypassing VerifySig (which only checks the signer is a committee replica, not the lane producer). Added a cheap signer-lane comparison before expensive crypto verification.
  • Parent hash chain verification: PushBlock did not verify that the incoming block's parentHash matches the previous block's header hash. A malicious lane proposer could send a block with a fake parentHash, which honest validators would accept and vote on. When headers() later tries to reconstruct the block chain by walking parentHash links backwards, it cannot find a matching vote and waits forever — deadlocking the node. Added parent hash verification when the previous block is available (after pruning, the check is safely skipped since headers() never follows the first block's parentHash in a LaneRange). A mismatch silently drops the block rather than returning an error, since equivocation (same key producing different chains) is a legitimate scenario.

@github-actions

github-actions Bot commented Feb 18, 2026 •

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedFeb 20, 2026, 2:27 PM

@github-actions

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedFeb 18, 2026, 8:42 PM

@codecov

codecov Bot commented Feb 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.42%. Comparing base (d327efb) to head (0ca0a40).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2925   +/-   ##
=======================================
  Coverage   68.42%   68.42%           
=======================================
  Files           5        5           
  Lines         456      456           
=======================================
  Hits          312      312           
  Misses        114      114           
  Partials       30       30           
Flag Coverage Δ
sei-db 68.42% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pompon0
pompon0 self-requested a review February 19, 2026 10:07
// Verify parent hash chain to prevent a malicious proposer from
// breaking the block chain, which would deadlock header reconstruction.
// A mismatch is not an error: it can happen legitimately when a
// producer equivocates (restarts and produces a different chain).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

equivocation of producer is as malicious behavior as equivocation of proposer. It does not endanger the security, but imo it would be worth escalating as an error, so that it gets logged. It will help debugging stalled lanes, if nothing more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, if we return an error, and the producer keep pushing the same equivocating block, then every receiver will repeatedly reconnect?
I've changed it to log an error but return nil for now

@wen-coding
wen-coding requested a review from arajasek February 19, 2026 19:19
wen-coding and others added 4 commits February 19, 2026 18:40
PushBlock accepted blocks without checking that the signer matches the
lane or that the block's parentHash chains to the previous block. A
malicious proposer could impersonate another lane or break the parent
hash chain, which would deadlock header reconstruction in headers().

Added signer == lane check (before expensive crypto verification) and
parent hash chain verification when the previous block is available.

Co-authored-by: Cursor <cursoragent@cursor.com>
A parentHash mismatch can happen legitimately when a producer
equivocates (e.g. restarts and produces a different chain). The block
is still prevented from entering the queue, but the caller is not
treated as malicious.

Co-authored-by: Cursor <cursoragent@cursor.com>
Parent hash mismatch indicates a producer equivocated, which is
malicious behavior worth logging to aid debugging stalled lanes.

Co-authored-by: Cursor <cursoragent@cursor.com>
Parent hash mismatch is logged at error level to aid debugging
stalled lanes, but not returned as an error to avoid tearing down
the peer connection in the p2p layer.

Co-authored-by: Cursor <cursoragent@cursor.com>
@wen-coding
wen-coding force-pushed the wen/fix_laneproposal_verify branch from 8836922 to 8df3583 Compare February 20, 2026 02:43
@wen-coding
wen-coding enabled auto-merge (squash) February 20, 2026 14:26
@wen-coding
wen-coding merged commit aa58397 into main Feb 20, 2026
37 of 38 checks passed
@wen-coding
wen-coding deleted the wen/fix_laneproposal_verify branch February 20, 2026 14:41
qezz pushed a commit to ChorusOne/sei-chain that referenced this pull request Aug 6, 2026
## Summary

- Fixes flaky `TestAvailClientServer`
(`sei-tendermint/internal/p2p/giga`), which has been hanging Race
Detection CI (including on unrelated PRs) with `parent hash mismatch`
logs and a stuck `GlobalBlock` wait.
- Replaces the broken “fake node0” setup on **shared** `nodes[0].data`
with a same-key equivocating peer that has its **own** store and feeds
corrupt blocks into node2 via `PushBlock` (admission API), after an
honest tip exists.

## History

**sei-v3** (`TestClientServer` in
`stream/consensus/avail/server_test.go`) intentionally stressed the mesh
by spawning a task that wrote **corrupt node0-lane blocks directly into
node2’s avail** (`ProduceBlock` on node2’s state for lane 0)
**concurrently** with honest production. There was no second node0, no
shared data dual-`Run`, and no wait for an honest tip. That worked
because sei-v3 `PushBlock` did **not** verify parent hashes—later honest
blocks could still append at `next` even after a bad tip.

**sei-chain migration (sei-protocol#2791)** rewrote this into giga
`TestAvailClientServer` as a “fake unconnected node0” that:
1. Built a second `consensus.State` on **shared** `nodes[0].data`
2. Called `fakeNode0.Run` while `env.Run` already ran
`nodes[0].consensus.Run`
3. Seeded node2 from that fake producer **before** honest production

That mistranslated the sei-v3 inject and introduced a data race under
`-race`.

**sei-protocol#2925** then added parent-hash verification in
`PushBlock`/`PushHeader`: on mismatch it logs and returns `nil` (drop,
no rewind). A divergent tip on node2 can no longer be extended by honest
gossip, so `GlobalBlock` can wait forever. The test was already probed
as flaky on clean main in Feb (`2fbb0c6e3`); under Race Detection the
hang shows up as a ~30m package timeout.

## What we test now

1. **3-of-4 committee mesh** via `env.Run` (giga avail client/server) —
same as before.
2. **Equivocating peer**: same key as node0, **own** `data`/avail
(`newTestNode`), **not** joined to the mesh (`testEnv` keys nodes by
`PublicKey`, so a second node0 cannot be `AddNode`’d). The test calls
`PushBlock` on node2 directly—same admission path gossip would use,
without a real peer connection. The peer is never `Run`, so inject is
bounded to `avail.BlocksPerLane` (initial lane capacity);
`ProduceLocalBlock` fails if that bound is raised.
3. **Honest production** on each active node’s own lane, concurrent with
the corrupt sender (`SpawnBg`).
4. **Agreement**: all nodes see the same `GlobalBlock` sequence and
accept `PushAppHash`.

No second `consensus.Run` on shared `nodes[0].data`.

## Why we wait for a non-empty honest tip

If the corrupt peer’s genesis block lands on node2 while lane 0 is still
empty:

1. Corrupt block 0 becomes the tip
2. Honest block 0 is ignored as stale
3. Honest block 1 fails parent-hash check and is dropped
4. The lane never advances → `GlobalBlock` hangs

So the corrupt task blocks on `a2.Block(ctx, lane0, 0)` (tolerating
`ErrPruned`) until node2 has an honest tip, then pushes its alternate
chain. Those `PushBlock`s are then stale or parent-mismatch drops; the
honest tip is preserved. This wait did **not** exist in sei-v3; it is
required under today’s `PushBlock` semantics (sei-protocol#2925).

## Why main is flaky

On main, dual `Run` + shared data + early corrupt/fake seeding races
under `-race`. When node2 adopts a fork that honest gossip cannot
extend, parent-hash drops leave sequencing stuck until the test package
times out. Without `-race` / low `-count` it often still passes by luck
(historically known-flaky).

Verified locally:

```bash
GOWORK=off go test ./internal/p2p/giga/ -race -count=30 -timeout 300s -run '^TestAvailClientServer$'
```

(30/30 after this change.)

## Test plan

- [x] `GOWORK=off go test ./internal/p2p/giga/ -race -count=30 -timeout
300s -run '^TestAvailClientServer$'`
- [x] CI Race Detection on this PR

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants