feat: benchmark stability improvements (CPU pinning, swap control, image pinning)#7
Merged
kamilchodola merged 5 commits intomainfrom Feb 27, 2026
Merged
Conversation
… for benchmark stability Add cpuset/infra_cpuset fields to pin execution client and infrastructure containers to dedicated CPU cores, eliminating scheduler migration noise. Add mem_swappiness control (default 0) to prevent swap-induced latency spikes. Pin K6 and Alloy Docker images to specific versions for reproducible runs. Add CPU governor preflight warning when not set to 'performance'.
mem_swappiness=0 aggressively evicts page cache to protect anonymous memory, which devastates I/O-heavy workloads like Nethermind that rely on page cache for state DB reads. Defaulting to None (host setting) preserves the previous behavior and avoids unexpected slowdowns.
Don't pass None values to Docker SDK containers.run() — some SDK versions may convert None to 0 for integer fields like mem_swappiness, silently changing container behavior. Build kwargs conditionally instead.
- Expand CPU governor check into broader preflight checks that also warn about Transparent Huge Pages (always mode) and active systemd timers known to cause I/O/CPU spikes during benchmarks - Update example cpuset values to show HT-aware pinning that avoids sharing physical cores between execution client and infra containers
Move the fix commands from structured log kwargs into the warning message text so they are always visible in log output.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cpuset/infra_cpuset): Optional Docker--cpuset-cpussupport to pin execution client and infrastructure containers to specific CPU cores. Useful on shared machines to isolate benchmark workloads. On dedicated machines, leaving these unset (letting the scheduler spread threads across all physical cores) typically gives better throughput.mem_swappiness): Optional Docker--memory-swappinesscontrol for the execution client container. Defaults to host setting (not passed to Docker) to avoid page cache eviction in I/O-heavy workloads.performancealwaysmode)1.6.1) and Alloy (v1.13.2) images pinned to specific versions instead of:latestfor reproducible benchmarks. Can be overridden in config.Important notes
cpuset,infra_cpuset,mem_swappiness) are optional with safe defaults. Existing configs work without changes.lscpu -e=CPU,CORE,SOCKET).mem_swappinessshould generally be left unset. Setting it to 0 aggressively evicts page cache, which devastates I/O-heavy workloads like Ethereum execution clients.Test plan
cpuset/infra_cpuset/mem_swappinessset — should matchmainbranch performancecpuset/infra_cpusetset — verify containers are pinned viadocker inspectperformance, THP isalways, or noisy timers are active