Skip to content

deps: upgrade wide/thiserror/criterion + once_cell→std; fast_max/min recovers ~2.2x AVX2 - #56

Merged
isPANN merged 1 commit into
mainfrom
perf/wide-fast-max-min
May 30, 2026
Merged

deps: upgrade wide/thiserror/criterion + once_cell→std; fast_max/min recovers ~2.2x AVX2#56
isPANN merged 1 commit into
mainfrom
perf/wide-fast-max-min

Conversation

@isPANN

@isPANN isPANN commented May 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Two connected changes — a dependency refresh, and the SIMD fix that the refresh turned out to need.

1. Dependency upgrades

  • wide 0.7 → 1 (SIMD)
  • thiserror 1.0 → 2.0
  • criterion 0.5 → 0.8 (dev/bench only; criterion::black_boxstd::hint::black_box)
  • once_cell → std OnceLock (external dep removed from tropical-gemm-cuda)

2. fast_max/fast_min — recovering performance the wide bump silently cost

wide 1.x's f32xN::max/min are NaN-aware, and that handling is not free:

arch .max() / .min() (wide 1.x default) .fast_max() / .fast_min()
x86 AVX max_m256 + is_nan() + vblendvps (3 ops + a max→isnan→blend dependency chain in the hot loop) single max_m256
ARM NEON NaN-aware vmaxnmq_f32 / vminnmq_f32 plain vmaxq_f32 / vminq_f32

Tropical GEMM never produces NaN — MaxPlus inputs are finite or −∞, MinPlus finite or +∞, MaxMul finite (zero = 0.0) — so the only NaN source (+∞ + −∞) cannot occur and the NaN handling is dead work. Switching all 7 microkernels (avx2 ×4, neon ×3) to fast_max/fast_min lowers each to a single maxps/minps/vmaxq/vminq.

Without this, the wide 0.7→1.x bump alone silently ~halves AVX2 throughput.

Benchmarks (A/B: fast_* vs default max/min, same wide 1.4.0, criterion, p < 0.05)

x86 AVX2 (HKUST-GZ HPC, f32):

size default max/min fast_* speedup
128 ~600 µs ~291 µs 2.07×
256 ~4.59 ms ~2.09 ms 2.20×
512 ~36.5 ms ~16.4 ms 2.22×
1024 ~291 ms ~131 ms 2.23×

(MaxPlus / MinPlus / MaxMul all ≈ −54%.)

ARM NEON (Apple M-series, f32): −3% to −4% on MinPlus/MaxMul; MaxPlus ≈ flat.

Correctness

cargo test -p tropical-gemm with fast_* in place: 279 unit + 24 doctests, 0 failures on both x86 AVX2 (HPC) and ARM NEON (local). Includes the assertion-based AVX2 kernel unit tests. cargo audit clean. CUDA crate (thiserror 2.0 + once_cell→std + cudarc) builds and its 56 GPU tests pass on A40/CUDA 12.8.

🤖 Generated with Claude Code

…x on AVX2)

wide 1.x's `f32xN::max`/`min` are NaN-aware: on x86 AVX they compile to
`max_m256` + `is_nan()` + `vblendvps` (1 -> 3 ops plus a max->isnan->blend
dependency chain in the inner loop); on aarch64 NEON they emit the NaN-aware
`vmaxnmq_f32`/`vminnmq_f32` instead of plain `vmaxq`/`vminq`.

Tropical GEMM never produces NaN -- MaxPlus inputs are finite or -inf, MinPlus
finite or +inf, MaxMul finite (zero = 0.0) -- so the only NaN source
(+inf + -inf) never occurs and the NaN handling is dead work. Switch all 7
microkernels (avx2 x4, neon x3) to `.fast_max()`/`.fast_min()`, which lower to a
single `maxps`/`minps`/`vmaxq`/`vminq`.

Hardware-measured A/B (fast_* vs default max/min, same wide 1.4.0):
- x86 AVX2 (HKUST-GZ): -51% to -55%, i.e. ~2.2x faster across all sizes and
  semirings. (So the wide 0.7->1.x bump had silently ~halved AVX2 throughput.)
- ARM NEON (Apple M-series): -3% to -4% on MinPlus/MaxMul; MaxPlus ~flat.

Correctness verified on both paths: 279 unit + 24 doctests on x86 AVX2 and on
ARM NEON, 0 failures.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codecov

codecov Bot commented May 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.29%. Comparing base (35e589c) to head (4fca49c).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #56   +/-   ##
=======================================
  Coverage   96.29%   96.29%           
=======================================
  Files          19       19           
  Lines         918      918           
=======================================
  Hits          884      884           
  Misses         34       34           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@isPANN isPANN changed the title perf(simd): use wide fast_max/fast_min in tropical microkernels (~2.2x on AVX2) deps: upgrade wide/thiserror/criterion + once_cell→std; fast_max/min recovers ~2.2x AVX2 May 30, 2026
@isPANN
isPANN merged commit 9904988 into main May 30, 2026
10 checks passed
@isPANN
isPANN deleted the perf/wide-fast-max-min branch May 30, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant