Skip to content

feat: hole-punch nim-v2.2 peer - #78

Merged
seetadev merged 16 commits into
libp2p:masterfrom
rlve:hp-nim-libp2p
Jul 14, 2026
Merged

feat: hole-punch nim-v2.2 peer#78
seetadev merged 16 commits into
libp2p:masterfrom
rlve:hp-nim-libp2p

Conversation

@rlve

@rlve rlve commented Mar 31, 2026

Copy link
Copy Markdown
Contributor

Add nim-v1.15 to hole-punch interop tests as dialer/listener.

@rlve
rlve requested a review from dhuseby as a code owner March 31, 2026 12:45
@richard-ramos
richard-ramos self-requested a review March 31, 2026 13:09
@rlve

rlve commented Mar 31, 2026

Copy link
Copy Markdown
Contributor Author

Oops, was working locally 🤔

@rlve

rlve commented Apr 1, 2026

Copy link
Copy Markdown
Contributor Author

I made the workflow pass on my fork on a default github runner (with little changes to not use interface_name Docker Compose feature, as it's not available there): rlve#3

Which suggests that the issue might be with the connectivity of the self-hosted runner.

cc: @richard-ramos

@rlve rlve changed the title add: hole-punch nim-v1.15 peer feat: hole-punch nim-v1.15 peer May 4, 2026
@seetadev

Copy link
Copy Markdown
Member

Excellent contribution by @rlve on this PR.

Adding nim-v1.15 support to the hole-punch interoperability test suite as both dialer and listener is an important step toward improving cross-implementation compatibility and strengthening NAT traversal testing across the libp2p ecosystem. It is great to see continued work expanding interoperability coverage for the Nim implementation.

I also appreciate the persistence shown here in debugging the workflow issues, validating the setup independently on a fork, and identifying that the remaining failures may be related to self-hosted runner connectivity rather than the implementation itself. That kind of investigation adds a lot of confidence to the contribution and helps narrow down where the actual bottleneck exists.

This is clearly an important PR for improving hole-punch interoperability coverage, and the work put into iterating through multiple fixes and validating behavior is highly appreciated.

Since Hole Punch Interoperability Tests (PR) / run-tests is still failing, tagging @luca and @Sumanjeet here as well, could you both please help investigate the remaining CI/CD or runner-related issues and work toward resolving them at the earliest so we can move this forward for merge.

Great work overall, @rlve, this is a meaningful contribution toward improving libp2p interoperability and hole-punch testing reliability across implementations.

CCing @johannamoran

@sumanjeet0012

Copy link
Copy Markdown
Collaborator

Relay Connectivity Failure - RCA and Verified Fix

Failing Tests & Configurations

  1. rust-v0.56 x nim-v1.15 (tcp, noise, yamux)
  2. rust-v0.56 x nim-v1.15 (tcp, noise, mplex)
  3. nim-v1.15 x rust-v0.56 (tcp, noise, yamux)
  4. nim-v1.15 x rust-v0.56 (tcp, noise, mplex)
  5. nim-v1.15 x nim-v1.15 (tcp, noise, yamux)
  6. nim-v1.15 x nim-v1.15 (tcp, noise, mplex)

Error Status:

  • Nim nodes: Connection to relay timed out: Timeout exceeded! (after 30 seconds)
  • Rust nodes: Error: Failed to connect: Transport([... Other(Custom { kind: Other, error: Timeout })]) (after 30 seconds)

Root Cause Analysis

The failures occurred before relay reservation or hole-punching could begin.

The relay was sending TCP SYN-ACK responses correctly, but the NAT routers (dialer-router and listener-router) were dropping them. Due to Docker veth checksum offloading, conntrack marked incoming SYN-ACK packets as INVALID, preventing reverse NAT translation.

As a result:

  1. TCP handshakes never completed.
  2. Nodes could not connect to the relay.
  3. Relay reservations were never created.
  4. All hole-punching tests timed out after 30 seconds.

Resolution

  1. Modified images/linux/Dockerfile to install ethtool.
  2. Updated images/linux/run.sh to include iptables CHECKSUM fill targets for outgoing packets (iptables -t mangle -A POSTROUTING -p tcp -j CHECKSUM --checksum-fill).
  3. Re-enabled the INPUT DROP rule in images/linux/run.sh to properly simulate the NAT and drop un-tracked packets (to avoid TCP RSTs).
  4. Modified the Docker Compose generation in lib/run-single-test.sh to inject the net.netfilter.nf_conntrack_checksum=0 sysctl into the routers. This forces conntrack to ignore checksum verification on incoming packets and allows it to successfully track and translate the Relay's SYN-ACK packets.

Verification

The fixes were verified locally.

@seetadev
seetadev requested review from acul71 and seetadev July 7, 2026 21:19
@seetadev

seetadev commented Jul 7, 2026

Copy link
Copy Markdown
Member

@acul71 , @sumanjeet0012 : Lets discuss with @richard-ramos and @rlve and arrive at a good conclusion on the issue.

This is an important PR indeed.

@rlve rlve changed the title feat: hole-punch nim-v1.15 peer feat: hole-punch nim-v2.2 peer Jul 8, 2026
@rlve

rlve commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

@seetadev @acul71 @sumanjeet0012 @richard-ramos

I have updated nim version to the latest master.

I wanted to see if it's gonna work in CI, but the workflow is hanging every time and won't finish.

Locally it's working for me:

# Hole Punch Interoperability Test Results

## Test Pass: `hole-punch-70b6eb26-140209-08-07-2026`

**Summary:**
- **Total Tests:** 4
- **Passed:** ✅ 4
- **Failed:** ❌ 0
- **Pass Rate:** 100.0%

**Environment:**
- **Platform:** arm64
- **OS:** Darwin
- **Workers:** 12
- **Duration:** 183s

**Timestamps:**
- **Started:** 2026-07-08T13:02:09Z
- **Completed:** 2026-07-08T13:05:12Z

---

## Test Results

| Test | Dialer | Listener | Transport | Status | Duration |
|------|--------|----------|-----------|--------|----------|
| nim-v2.2 x rust-v0.56 (tcp, noise, yamux) [dr - linux, rly - rust-v0.56, lr - linux] | nim-v2.2 | rust-v0.56 | tcp | ✅ | 2s |
| rust-v0.56 x nim-v2.2 (tcp, noise, yamux) [dr - linux, rly - rust-v0.56, lr - linux] | rust-v0.56 | nim-v2.2 | tcp | ✅ | 4s |
| rust-v0.56 x nim-v2.2 (tcp, noise, mplex) [dr - linux, rly - rust-v0.56, lr - linux] | rust-v0.56 | nim-v2.2 | tcp | ✅ | 4s |
| nim-v2.2 x rust-v0.56 (tcp, noise, mplex) [dr - linux, rly - rust-v0.56, lr - linux] | nim-v2.2 | rust-v0.56 | tcp | ✅ | 7s |

---

*Generated: 2026-07-08T13:05:12Z*

@seetadev seetadev left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@acul71 , @sumanjeet0012 : Please help arrive at a good conclusion on the issue. Looks like an issue on the configuration front.

Works locally with all tests passing.

@sumanjeet0012 sumanjeet0012 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rlve The tests are now running, however they are taking longer time,
So lets remove the timeout and re run the tests.

needs: resolve-parameters
if: needs.resolve-parameters.outputs.should-run-tests == 'true'
runs-on: [self-hosted, linux, x64, ephemeral]
timeout-minutes: 31

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rlve Can you remove the timeout and re-run the tests.

@seetadev

Copy link
Copy Markdown
Member

@rlve : Great, thank you for making the requisite additions. Looking forward to investigating the 1 CI/CD issue, which is giving trouble.

@sumanjeet0012 , @acul71 : Wish to share the logs:

Artifact hole-punch-snapshot-hole-punch-f55a4511-131026-11-07-2026 has been successfully uploaded! Final size is 1308771 bytes. Artifact ID is 8249774949
Artifact download URL: https://github.com/libp2p/unified-testing/actions/runs/29148739054/artifacts/8249774949
Run if [ -f /tmp/test-exit-code.txt ]; then
Tests failed with exit code 1
Error: Process completed with exit code 1.

@rlve

rlve commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

@seetadev it looks to me the issues are not related to nim implementation, there is rather a problem with openbsd router. This is what ai suggests:

Summary

The 4 "failing" hole-punch tests (nim-v2.2 dialer, dr=linux, lr=openbsd,
tcp/noise, yamux+mplex) are not a nim-libp2p bug. They are a pre-existing
infrastructure bug in the unified-testing OpenBSD NAT router. nim hole-punching
works — the only tests that actually exercise it (no OpenBSD router) pass with
real measurements.

Root cause

unified-testing/hole-punch/images/openbsd/entrypoint.sh runs under
set -euo pipefail. The per-bridge netfilter loop writes to a read-only /sys
in the unprivileged router container:

[ -f "/sys/class/net/$br/bridge/$f" ] && echo 0 > "/sys/class/net/$br/bridge/$f"

The write fails (Read-only file system); under set -e this aborts the
entrypoint
before the OpenBSD VM boots, so the container's HEALTHCHECK
(test -f /tmp/healthy, 120s start-period) never passes.

Why the pass/fail is misleading

Every test that uses an OpenBSD router (dr or lr) crashes the same way — hole
punch never happens in any of them (all 18 have empty measurements). The score
is a docker compose up --exit-code-from dialer --abort-on-container-exit
artifact:

  • dr=openbsd → dialer never starts (dep failed) → compose returns 0 → false pass
  • dr=linux, lr=openbsd → dialer starts, then SIGTERM'd by abort → exit code decides:
    rust dialer exits 0 (false pass), nim dialer exits 143 → normalized to fail.

That is the only reason exactly the 4 nim-dialer + dr=linux + lr=openbsd cases
surfaced as failures.

@sumanjeet0012

Copy link
Copy Markdown
Collaborator

@rlve Spot on—your observations are completely correct.
The underlying issue was caused by OpenBSD, so I have removed it from the hole-punching interop tests to resolve this. I've already validated these changes and the overall test suite via a separate sandbox (PR #117).

Could you please rebase or merge the latest master changes into this PR? Once applied, everything should work as expected.

@rlve

rlve commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

@sumanjeet0012 the branch is updated and tests passed. Thanks!

@seetadev

Copy link
Copy Markdown
Member

@rlve , @sumanjeet0012 : Awesome, great to hear. Thank you so much for sharing the updates.

Ccing @johannamoran and @mishmosh

LGTM

@seetadev
seetadev merged commit f544e0a into libp2p:master Jul 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants