[MarkRose]

Right now no job builds the hand-tuned SIMD asm under a sanitizer. Every sanitizer job is pinned to NOSIMD, because macOS cannot assemble the SIMD under -Og (invalid CFI advance_loc expression). That pinning is correct for macOS, but on Linux it gives up the configuration that has actually caught a SIMD memory bug in this codebase: issue #12, a SEGV in radix48_dit_pass1 during Mlucas -s m, was reported by AddressSanitizer on an ARM runner.

This PR now also covers the x86 side, which #221 proposed as a separate job. Following the review there, that job is dropped in favour of this change to the existing job, and #221 is closed.

What it does

The ASan-Linux job gets an OS axis, ubuntu-latest and ubuntu-26.04-arm, and builds the runner's native SIMD mode under the same -Og -fsanitize=address,undefined flags as before:

• x86 builds AVX2 or AVX-512, depending on the runner. GitHub's x86 runners include AVX-512 machines often enough that both get covered over time, without an emulator.
• ARM builds ASIMD.
• On ARM, config-fermat.sh runs from a NOSIMD build, the same substitution the Linux job makes: ASIMD's trial-factoring routines are stubs (No TF support on ARMv8! in twopmodq80.c). The Mfactor builds stay NOSIMD on both, as before.
• Same tiers as before: tt t s m plus PRP tt t s.
• Like the other jobs since CI: upload the logs, not the whole workspace #295, it uploads only the logs. For consistency, every ASan and TSan job now writes its self-test output to test.log and uploads the logs. The jobs that use GitHub's default shell gain -o pipefail, so that a sanitizer abort still fails the step through tee.

UBSan stays enabled alongside ASan.

Verified

x86, built from main with the job's flags (-std=gnu99 -Wall -Wextra -g -Og -fsanitize=address,undefined), gcc 16.2 and clang 23:

build	where it ran	-s tt	-s t	-prp -s tt
AVX2 (native), gcc and clang	Intel laptop	clean	clean	clean
AVX-512, gcc and clang	Zen 4 desktop, natively	clean	clean	clean

"Clean" means exit status 0, no Res64 errors and no ASan or UBSan reports; the AVX-512 runs confirm Build uses AVX512. Both compilers build both modes without errors, so the asm operand has impossible constraints failure that once hit gcc's ASan stack instrumentation in the SIMD macros, and the duplicate asm labels under UBSan, are not a problem at -Og on current main.

ARM: the gcc ASIMD build under -Og -fsanitize=address,undefined compiled, linked and self-tested clean on aarch64 hardware (Debian 12, gcc 12.2), and clang 14 assembled the SIMD-heaviest units (radix48_ditN_cy_dif1.c, the one from #12, and the radix-16/32 pass and wrapper files) without diagnostics. The runner's compilers are newer, so this PR's own CI run is the real check there.

On #12 itself

I tried to reproduce #12 on aarch64 hardware against main, and it does not reproduce: the reported configuration passes, including under ASan, and the -s m sweep reached M57984131 at 3072K with radices 48/32/32/32 and passed it. I would not call the issue fixed on that basis (different platform, Debian/gcc vs macOS/clang, and the report says "sometimes"), but this job is what would tell us on an ongoing basis.

Overlaps

• Make the self-test capable of failing, and cover the FFT lengths it never reached #216 rewrites the self-test lines in this job to add its exit-status gate, CI: Windows ASan in native SIMD mode, and no -s m in the macOS ASan job #367 moves this job's -s m to the monthly workflow, and Split Linux and ASan CI into build and test jobs #363 splits this job into build and test jobs. All three conflict with this PR; whichever lands second gets rebased.

CI only; no source changes.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Tests
  • Expanded sanitizer test coverage across Linux, ARM, macOS, and Windows, including native SIMD builds and additional Mlucas tests.
  • Test output is now captured in logs, and sanitizer jobs retain build and test artifacts for review.

[tdulcet]

Thanks for the PR.

Instead of adding a new CI job for this, could you instead update the matrix for the existing ASan Linux job to also run on a ARM runner. That would eliminate the duplication.

[MarkRose]

Done - the ARM run is now a cell of the existing ASan-Linux matrix rather than a separate job, and the new job is gone. Description rewritten to match.

The matrix is os: [ubuntu-latest, ubuntu-24.04-arm] × cc: [gcc, clang], with the build mode selected from $HOSTTYPE the way the Linux and Singlethread-Linux jobs already do it: x86 keeps NOSIMD and behaves exactly as before, ARM lets makemake.sh auto-detect, which gives ASIMD. On ARM, config-fermat.sh and the Mfactor self-tests run from the NOSIMD builds - the same substitution those two jobs make, since ASIMD's trial-factoring routines are stubs.

Two other things fell out of the merge:

• The -s m gating is gone. I had held the medium tier back for pushes and the schedule because I had no trustworthy cost estimate. Sharing the job body settles it: ASIMD is far faster per iteration than the NOSIMD build the x86 cells already run -s m on, so the ARM cells run the same tt t s m plus PRP tt t s.
• The WARNING: 0 of and runtime error: assertions are dropped from this PR. They would be new failure conditions on cells that are green today, which is a separate change and shouldn't ride along on "eliminate the duplication". I'd still like them; happy to open that as its own PR.

I kept the assertion that the binary really reports Build uses ARMv8 advanced-SIMD, since without it the ARM cells silently become duplicates of the x86 ones if auto-detection ever changes. Checked that the string is printed only under #ifdef USE_ARM_V8_SIMD - the near-miss line for a scalar build on ASIMD-capable hardware reads CPU supports ARMv8 advanced-SIMD ... but using scalar floating-point build and does not match.

Since the matrix now puts clang on ARM as well, I checked that too on aarch64 hardware: the SIMD-heaviest translation units - radix48_ditN_cy_dif1.c (the file from #12), radix32_wrapper_square.c, radix32_dif_dit_pass.c, radix16_dif_dit_pass.c - all assemble with zero diagnostics under -mcpu=native -Og -fsanitize=address,undefined, and the full gcc build compiles, links and self-tests clean. That box has clang 14 rather than the runner's much newer one, so this PR's own CI run is the real check, but the macOS assembler problem clearly does not follow us to ARM Linux.

[MarkRose]

Follow-up with the numbers I said I was missing when I gated the medium tier, now that the ARM build has finished here. Debian 12 / gcc 12.2, native ASIMD, -Og -fsanitize=address,undefined, 4 threads on a Raspberry Pi 4 - so a floor, not a prediction for the runners, which are considerably faster:

tier	elapsed	exit	UBSan runtime error:	ASan reports	WARNING: 0 of
-s t	29 s	0	0	0	0
-s s	364 s	0	0	0	0

And the ASIMD guard checked against a real binary rather than just against util.c: that build prints INFO: Build uses ARMv8 advanced-SIMD instruction set., so the job's grep matches.

Given -s s is six minutes on a Pi, -s m on an ubuntu-24.04-arm runner should sit comfortably inside what the x86 NOSIMD cells already spend on the same tier - ASIMD does far more work per iteration than the NOSIMD build those cells run. This PR's own CI run will give the real figure; if it turns out badly I will put the gate back rather than slow the matrix down.

---

Since the runners are backed up (this PR's run has been fully queued for three hours), I also verified the restructured job's shell logic directly, by extracting the Script step out of the workflow and running it against stub makemake.sh / Mlucas / Mfactor / config-fermat.sh with HOSTTYPE forced each way:

• HOSTTYPE=x86_64 - builds obj_nosimd only, no second build, no ASIMD assert, all tiers from obj_nosimd. Identical to the job's current behaviour.
• HOSTTYPE=aarch64 - builds obj (ASIMD) and obj_nosimd, plus the six Mfactor dirs; asserts on the ASIMD binary; runs tt t s m and PRP tt t s from obj; then cd ../obj_nosimd for config-fermat.sh and ../obj_mfac_*_nosimd for the Mfactor self-tests.
• HOSTTYPE=aarch64 with the banner absent - emits ::error::ARM cell is not an ASIMD build and exits 1 before running any tier, so a silently-degraded cell fails in seconds instead of after an hour.

That covers the part I would otherwise only find out about from a failed CI run: the MODE selection and the directory sequence.

[MarkRose]

The ASan-Linux cells on this PR are failing, and I want to get ahead of it: the failure is pre-existing on main and not caused by this change. AddressSanitizer Linux is currently red on main on every recent run - I checked the last four - for both the gcc and clang cells, and the two new ARM cells fail in exactly the same place for exactly the same reason.

What it is

config-fermat.sh, F24 at FFT length 1008K:

==8236==ERROR: AddressSanitizer: global-buffer-overflow
READ of size 8 at 0x... thread T7
    #0 cy1008_process_chunk ../src/radix1008_main_carry_loop.h:718
SUMMARY: AddressSanitizer: global-buffer-overflow in cy1008_process_chunk

That line is the scalar-double Fermat carry, fermat_carry_norm_errcheckB(..., icycle[ic_idx], ...) - an indexed read into the cycle-index array. Note this is the NOSIMD build (obj_nosimd/Mlucas), which the ARM cells build for config-fermat.sh exactly as the x86 cells do, so it has nothing to do with the ASIMD build this PR adds.

Minimal reproducer, no CI needed:

$ sed -i 's/^CFLAGS = .*/CFLAGS = -std=gnu99 -Wall -g -Og -fsanitize=address,undefined/' makemake.sh
$ bash makemake.sh use_hwloc nosimd
$ ./obj_nosimd/Mlucas -f 24 -fft 1008 -iters 100
==2608433==ERROR: AddressSanitizer: global-buffer-overflow
SUMMARY: ... radix1008_main_carry_loop.h:720 in cy1008_process_chunk
exit 1

#156 fixes it

Same command, same build settings, against #156:

build	exit	ASan	result
main	1	global-buffer-overflow at radix1008_main_carry_loop.h:720	abort
#156	0	none	INFO: 2 of 3 radix-sets at FFT length 1008 K passed - writing cfg-file entry

#156 is a one-word change - % nwt16 -> % nwt in the scalar-double branch of radix1008_ditN_cy_dif1.c, because icycle[] holds plain non-pointerized indices in [0,nwt) there while nwt16 is the SIMD/pointerized modulus. An index reduced by the larger modulus runs off the end of the array, which is precisely the read ASan is reporting.

So #156 is what is keeping AddressSanitizer Linux red, and landing it should green all four cells here.

(The remaining 2 of 3 is unrelated to ASan: the two radix-1008-leading sets agree on DB9F01963ED9DC8B, the radix-63-leading set gives D6AFB0809A81A431. This is a Pépin test of F24, so that looks like the radix-63 Pépin defect #232 addresses, not anything new.)

What this PR's own run did establish

• The ASIMD guard works on a real runner - Build uses ARMv8 advanced-SIMD is present in both ARM cells' mode.log, so the cells really are instrumenting the SIMD build rather than silently degrading.
• Both ARM cells got through every Mlucas self-test tier and both PRP tiers before config-fermat.sh, so clang and gcc both assemble and run the hand-tuned ASIMD asm under -Og -fsanitize=address,undefined. That was the open question when I posted the Pi results.
• One new ARM finding: the clang cell reports WARNING: 0 of 2 radix-sets at FFT length 10 K passed, where both sets produce the same residue 4428783BC62760F0 but MaxErr = 0.4375 exceeds the self-test's 0.40625 gate for 100-iteration runs. The gcc cell on the same runner has no such line, so this is clang-specific rather than an -Og artifact. Filing separately; it does not block this PR.

[MarkRose]

Rebased onto main now that #291 is in. The one conflict was the ASan job's build block: I kept this PR's ARM/x86 mode selection and took #291's {{1..4},n}word loop.

[coderabbitai]

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: 23f3b1d3-a1bf-4947-a2ea-5cf9722b3445

📥 Commits

Reviewing files that changed from the base of the PR and between 3cd54ed and 4fede2e.

📒 Files selected for processing: 1

• .github/workflows/ci.yml

+-----------------------------------------------------------------+
| This is Sparta! And I'm here to kick bugs into the pit of doom. |
+-----------------------------------------------------------------+
  \
   \     /)/)
        ( '')
        /  |
      o(___)

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: 4d3256db-ce39-478b-a4ec-3cb030627840

📥 Commits

Reviewing files that changed from the base of the PR and between 5c898e4 and 3cd54ed.

📒 Files selected for processing: 1

• .github/workflows/ci.yml

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

The CI workflow expands ASan-Linux coverage to an ARM runner and changes sanitizer build and test steps. Linux, macOS, and Windows sanitizer jobs capture test output in test.log and upload selected build and test files.

Changes: Sanitizer CI

Layer / File(s)	Summary
ASan-Linux matrix and builds .github/workflows/ci.yml	ASan-Linux adds an ARM runner and tests the native SIMD build under ASan/UBSan. On ARM, the job also builds NOSIMD for Fermat configuration. Mfactor variants remain NOSIMD builds, and their self-tests run from those builds. The job appends Mlucas test output to test.log and uploads selected build, test, and Fermat configuration files.
Linux TSan test capture .github/workflows/ci.yml	The Linux TSan script enables pipefail and appends the tiny Mlucas self-test output to test.log. The job uploads selected build, test, and Fermat configuration files.
macOS and Windows sanitizer test capture .github/workflows/ci.yml	The macOS ASan and TSan scripts enable pipefail and append Mlucas test output to test.log. The Windows ASan tests also append Mlucas output. These jobs upload selected build, test, and Fermat configuration files.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Merge Risk: ⚪ Minimal · up to 3cd54

The PR adds ARM sanitizer coverage and captures test artifacts. The reported Fermat overflow was fixed before this change, and no new failure is established.

Security architecture risk: 🔵 Low · up to 3cd54

The change broadens sanitizer coverage and publishes additional diagnostic files without adding credentials or deployment privileges. No introduced security defect was established. Access to those files and their availability after interrupted runs depend on settings and behavior not verified here.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The demonstrated expansion is additional sanitizer execution on hosted ARM runners and publication of CI-generated diagnostics. The changed jobs do not introduce production deployment or a privileged consumer of those files. Their effective readership and any external downstream consumption remain outside the available evidence.

Trust Boundaries and Controls

• inferred — Pull-request-controlled source can influence the uploaded diagnostics because the jobs build and execute checked-out code. These files therefore remain untrusted test output, not proof of successful validation. The inspected change adds publication but does not establish a new transition from those files into privileged execution.

Pre-merge checks: 5

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: extending the ASan Linux job to ARM and instrumenting the ASIMD build.
Docstring Coverage		No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[tdulcet]

Thanks, this looks much better. I just have a few minor nits.

[MarkRose]

Thanks. I've applied all three, and I've folded #221 into this PR as you suggested there: the job now builds the native SIMD mode on x86 as well (AVX2 or AVX-512, depending on the runner), with no new job and no SDE. UBSan stays on. Both compilers build AVX2 and AVX-512 under the job's flags and pass -s tt, -s t and -prp -s tt with no sanitizer reports; I ran the AVX-512 builds on a Zen 4 machine. I've updated the description.

[tdulcet]

Thanks, this looks good. Just one more change is needed.

[tdulcet]

Thanks.