Reject types with a niche in copy_struct/copy_vec - #129
Open
korniltsev-grafanista-yolo-vibecoder239 wants to merge 1 commit into
Open
Conversation
korniltsev-grafanista-yolo-vibecoder239
force-pushed
the
niche-guard
branch
4 times, most recently
from
September 7, 2026 08:48
53480a7 to
82635c3
Compare
`copy_struct<T: Copy>` materializes a `T` out of bytes read from another process, but `T: Copy` does not mean that every bit pattern is a valid `T`. When `T` has a niche, rustc may store the discriminant of the returned `Result<T, Error>` inside it - and if the niche leaves no room for a real tag, it will. A stale read whose bytes happen to hit the niche value then comes back as an `Err` whose payload is 32 bytes of target memory, and dropping that error frees a pointer that came from the profiled process. That is the root cause of a `free(): invalid pointer` crash in py-spy on Python 3.11, whose `_PyInterpreterFrame` binding declares `is_entry` as a `bool`: the struct is 80 bytes with 3 bytes of tail padding, so `Result<_PyInterpreterFrame, Error>` is 80 bytes too and the tag lives in `is_entry`. See grafana/pyroscope-python#146 for the disassembly and the decoded core dumps. Assert at compile time that copied types have no niche, using the fact that a niche-free type always makes `Option<T>` strictly larger than `T`. The bound stays `T: Copy`, so this is not a breaking change: consumers that copy niche-free types are unaffected, and the ones that aren't were already unsound. `copy_pointer` is covered because it delegates to `copy_struct`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
korniltsev-grafanista-yolo-vibecoder239
force-pushed
the
niche-guard
branch
from
September 7, 2026 08:53
82635c3 to
36602e7
Compare
This was referenced Sep 8, 2026
korniltsev-grafanista
added a commit
to grafana/pyroscope-py-spy
that referenced
this pull request
Sep 8, 2026
Carries benfred/remoteprocess#129, which rejects niche-having types in `copy_struct`/`copy_vec` at compile time so the previous commit's bug cannot come back. Pinned by `rev` to the tip of the fork's still-open PR branch, so the rev must move to the merge commit before this lands. The fork is at 0.5.3, which needs `proc-maps ^0.5`, so the lockfile also takes `libc 0.2.185 -> 0.2.189`, `anyhow 1.0.102 -> 1.0.104`, `read-process-memory 0.1.6 -> 0.2.0`, and a second `proc-maps` alongside py-spy's own 0.4.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
korniltsev-grafanista
added a commit
to grafana/pyroscope-py-spy
that referenced
this pull request
Sep 8, 2026
Carries benfred/remoteprocess#129, which rejects niche-having types in `copy_struct`/`copy_vec` at compile time so the previous commit's bug cannot come back. Pinned by `rev` to the tip of the fork's still-open PR branch, so the rev must move to the merge commit before this lands. The fork is at 0.5.3, which needs `proc-maps ^0.5`, so the lockfile also takes `libc 0.2.185 -> 0.2.189`, `anyhow 1.0.102 -> 1.0.104`, `read-process-memory 0.1.6 -> 0.2.0`, and a second `proc-maps` alongside py-spy's own 0.4.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
korniltsev-grafanista
added a commit
to grafana/pyroscope-py-spy
that referenced
this pull request
Sep 8, 2026
Carries benfred/remoteprocess#129, which rejects niche-having types in `copy_struct`/`copy_vec` at compile time so the previous commit's bug cannot come back. Pinned by `rev` to the tip of the fork's still-open PR branch, so the rev must move to the merge commit before this lands. The fork is at 0.5.3, which needs `proc-maps ^0.5`, so the lockfile also takes `libc 0.2.185 -> 0.2.189`, `anyhow 1.0.102 -> 1.0.104`, `read-process-memory 0.1.6 -> 0.2.0`, and a second `proc-maps` alongside py-spy's own 0.4.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
korniltsev-grafanista
added a commit
to grafana/pyroscope-py-spy
that referenced
this pull request
Sep 8, 2026
…V) (#3) * fix: don't read target-process bytes as `bool` `_PyInterpreterFrame.is_entry` (py 3.11) and the `numpy.bool` payload are read out of the profiled process, where the byte can hold any value. Only 0 and 1 are valid `bool` bit patterns, so materializing one is UB. It also gave `_PyInterpreterFrame` a niche: the struct was 80 bytes with `is_entry` its only niche, so rustc stored the `Err` tag of `Result<_PyInterpreterFrame, remoteprocess::Error>` inside that byte. A stale read whose byte was neither 0 nor 1 came back as an `Err` holding 32 bytes of target memory, and dropping it freed a pointer from the profiled process. That is the `free(): invalid pointer` crash root-caused in grafana/pyroscope-python#146. `numpy.bool` locals now render as 0/1 instead of true/false, so `test_local_vars` is updated to match. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * build: use grafana/pyroscope-remoteprocess with the niche guard Carries benfred/remoteprocess#129, which rejects niche-having types in `copy_struct`/`copy_vec` at compile time so the previous commit's bug cannot come back. Pinned by `rev` to the tip of the fork's still-open PR branch, so the rev must move to the merge commit before this lands. The fork is at 0.5.3, which needs `proc-maps ^0.5`, so the lockfile also takes `libc 0.2.185 -> 0.2.189`, `anyhow 1.0.102 -> 1.0.104`, `read-process-memory 0.1.6 -> 0.2.0`, and a second `proc-maps` alongside py-spy's own 0.4.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@benfred hi, I reported initial issue in Grafana slack channel. Happy to provide more information if needed as it's fairly easy to reproduce it on our end. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ProcessMemory::copy_struct<T: Copy>materializes aTout of bytes read fromanother process.
T: Copydoes not mean every bit pattern is a validT, andwhen
Thas a niche, rustc may put the discriminant of the returnedResult<T, Error>inside it — if the niche leaves no room for a real tag, itwill:
A stale read whose bytes happen to hit the niche value then comes back as an
Err(Error)whose payload is 32 bytes of target memory. The caller usuallycontexts it into an
anyhow::Error; when that is dropped,drop_gluereads thefirst word as an enum discriminant, selects a
String-owning variant, and freesthe next word as a heap pointer.
The real case
This is the root cause of a
free(): invalid pointer/ SIGSEGV in py-spy onPython 3.11, reported against the Pyroscope Python agent and root-caused in
grafana/pyroscope-python#146
(disassembly of the shipped wheel, four decoded core dumps, deterministic
reproducer). py-spy's 3.11
_PyInterpreterFramebinding declaresis_entryasa
boolat offset 68, which is the struct's only niche. The 32-byteErrorfits in the bytes below it, so rustc niche-fills the
Resultinto that byterather than adding a tag:
so the
Errtag is stored inis_entry, and the shipped code really doescmp $0x2, %alon that byte to decide whether the read failed.The change
Assert at compile time that copied types have no niche, using the fact that a
niche-free type always makes
Option<T>strictly larger thanT.copy_pointeris covered for free because it delegates to
copy_struct;copy_vecgets thesame line since it builds its
Vec<T>without going throughcopy_struct.Not a breaking change: the bound stays
T: Copy. Consumers that copyniche-free types — integers, floats, raw pointers,
Option<extern fn>, arraysand
#[repr(C)]structs of those — are unaffected. The ones that aren't werealready unsound.
I considered
bytemuck::AnyBitPatternas the bound instead, and it does not workhere: bytemuck only implements
Podfor*const T/*mut Tbehind itsunsound_ptr_pod_implfeature and never for C function pointers, so#[derive(AnyBitPattern)]rejects essentially every FFI struct a consumer ofthis crate would want to copy. It would also be a breaking API change.
Effect on py-spy
Building py-spy (master,
32080cc) against this branch fails with exactly tworejections, both genuine:
The first is the known crash. The second was previously unknown:
python_data_access.rsreads anumpy.bool_'s payload byte straight into a Rustbool(format_obval::<bool, P>), i.e. an arbitrary target byte materialized asa
bool. With those two fixed (is_entry→c_char,numpy.bool→u8),py-spy builds clean with default features,
--no-default-features,--features cliand--all-targets, and its unit tests pass — so the checkaccepts every other type py-spy copies across all 14 CPython binding modules,
including the function-pointer-heavy
PyTypeObject/PyHeapTypeObject.Tests
cargo testpasses unchanged.🤖 Generated with Claude Code