Skip to content

Reject types with a niche in copy_struct/copy_vec - #129

Open
korniltsev-grafanista-yolo-vibecoder239 wants to merge 1 commit into
benfred:masterfrom
korniltsev-grafanista-yolo-vibecoder239:niche-guard
Open

Reject types with a niche in copy_struct/copy_vec#129
korniltsev-grafanista-yolo-vibecoder239 wants to merge 1 commit into
benfred:masterfrom
korniltsev-grafanista-yolo-vibecoder239:niche-guard

Conversation

@korniltsev-grafanista-yolo-vibecoder239

@korniltsev-grafanista-yolo-vibecoder239 korniltsev-grafanista-yolo-vibecoder239 commented Sep 7, 2026

Copy link
Copy Markdown

ProcessMemory::copy_struct<T: Copy> materializes a T out of bytes read from
another process. T: Copy does not mean every bit pattern is a valid T, and
when T has a niche, rustc may put the discriminant of the returned
Result<T, Error> inside it — if the niche leaves no room for a real tag, it
will:

fn copy_struct<T: Copy>(&self, addr: usize) -> Result<T, Error> {
    let mut data = vec![0; std::mem::size_of::<T>()];
    self.read(addr, &mut data)?;
    Ok(unsafe { std::ptr::read(data.as_ptr() as *const _) })
}

A stale read whose bytes happen to hit the niche value then comes back as an
Err(Error) whose payload is 32 bytes of target memory. The caller usually
contexts it into an anyhow::Error; when that is dropped, drop_glue reads the
first word as an enum discriminant, selects a String-owning variant, and frees
the next word as a heap pointer.

The real case

This is the root cause of a free(): invalid pointer / SIGSEGV in py-spy on
Python 3.11, reported against the Pyroscope Python agent and root-caused in
grafana/pyroscope-python#146
(disassembly of the shipped wheel, four decoded core dumps, deterministic
reproducer). py-spy's 3.11 _PyInterpreterFrame binding declares is_entry as
a bool at offset 68, which is the struct's only niche. The 32-byte Error
fits in the bytes below it, so rustc niche-fills the Result into that byte
rather than adding a tag:

size_of::<v3_11_0::_PyInterpreterFrame>()               == 80
size_of::<Result<_PyInterpreterFrame, remoteprocess::Error>>() == 80

so the Err tag is stored in is_entry, and the shipped code really does
cmp $0x2, %al on that byte to decide whether the read failed.

The change

Assert at compile time that copied types have no niche, using the fact that a
niche-free type always makes Option<T> strictly larger than T. copy_pointer
is covered for free because it delegates to copy_struct; copy_vec gets the
same line since it builds its Vec<T> without going through copy_struct.

Not a breaking change: the bound stays T: Copy. Consumers that copy
niche-free types — integers, floats, raw pointers, Option<extern fn>, arrays
and #[repr(C)] structs of those — are unaffected. The ones that aren't were
already unsound.

I considered bytemuck::AnyBitPattern as the bound instead, and it does not work
here: bytemuck only implements Pod for *const T/*mut T behind its
unsound_ptr_pod_impl feature and never for C function pointers, so
#[derive(AnyBitPattern)] rejects essentially every FFI struct a consumer of
this crate would want to copy. It would also be a breaking API change.

Effect on py-spy

Building py-spy (master, 32080cc) against this branch fails with exactly two
rejections, both genuine:

error[E0080]: evaluation panicked: this type has a niche: not every bit pattern is a valid value, ...
    = note: evaluation of `remoteprocess::AssertNoNiche::<python_bindings::v3_11_0::_PyInterpreterFrame>::ASSERT` failed here
note: the above error was encountered while instantiating `fn <Process as ProcessMemory>::copy_struct::<v3_11_0::_PyInterpreterFrame>`

error[E0080]: evaluation panicked: this type has a niche: ...
    = note: evaluation of `remoteprocess::AssertNoNiche::<bool>::ASSERT` failed here
note: the above error was encountered while instantiating `fn <Process as ProcessMemory>::copy_struct::<bool>`

The first is the known crash. The second was previously unknown:
python_data_access.rs reads a numpy.bool_'s payload byte straight into a Rust
bool (format_obval::<bool, P>), i.e. an arbitrary target byte materialized as
a bool. With those two fixed (is_entryc_char, numpy.boolu8),
py-spy builds clean with default features, --no-default-features,
--features cli and --all-targets, and its unit tests pass — so the check
accepts every other type py-spy copies across all 14 CPython binding modules,
including the function-pointer-heavy PyTypeObject/PyHeapTypeObject.

Tests

cargo test passes unchanged.

🤖 Generated with Claude Code

`copy_struct<T: Copy>` materializes a `T` out of bytes read from another
process, but `T: Copy` does not mean that every bit pattern is a valid `T`.
When `T` has a niche, rustc may store the discriminant of the returned
`Result<T, Error>` inside it - and if the niche leaves no room for a real tag,
it will. A stale read whose bytes happen to hit the niche value then comes back
as an `Err` whose payload is 32 bytes of target memory, and dropping that error
frees a pointer that came from the profiled process.

That is the root cause of a `free(): invalid pointer` crash in py-spy on Python
3.11, whose `_PyInterpreterFrame` binding declares `is_entry` as a `bool`: the
struct is 80 bytes with 3 bytes of tail padding, so
`Result<_PyInterpreterFrame, Error>` is 80 bytes too and the tag lives in
`is_entry`. See grafana/pyroscope-python#146 for the
disassembly and the decoded core dumps.

Assert at compile time that copied types have no niche, using the fact that a
niche-free type always makes `Option<T>` strictly larger than `T`. The bound
stays `T: Copy`, so this is not a breaking change: consumers that copy
niche-free types are unaffected, and the ones that aren't were already unsound.
`copy_pointer` is covered because it delegates to `copy_struct`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
korniltsev-grafanista added a commit to grafana/pyroscope-py-spy that referenced this pull request Sep 8, 2026
Carries benfred/remoteprocess#129, which rejects
niche-having types in `copy_struct`/`copy_vec` at compile time so the
previous commit's bug cannot come back.

Pinned by `rev` to the tip of the fork's still-open PR branch, so the rev
must move to the merge commit before this lands.

The fork is at 0.5.3, which needs `proc-maps ^0.5`, so the lockfile also
takes `libc 0.2.185 -> 0.2.189`, `anyhow 1.0.102 -> 1.0.104`,
`read-process-memory 0.1.6 -> 0.2.0`, and a second `proc-maps` alongside
py-spy's own 0.4.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
korniltsev-grafanista added a commit to grafana/pyroscope-py-spy that referenced this pull request Sep 8, 2026
Carries benfred/remoteprocess#129, which rejects
niche-having types in `copy_struct`/`copy_vec` at compile time so the
previous commit's bug cannot come back.

Pinned by `rev` to the tip of the fork's still-open PR branch, so the rev
must move to the merge commit before this lands.

The fork is at 0.5.3, which needs `proc-maps ^0.5`, so the lockfile also
takes `libc 0.2.185 -> 0.2.189`, `anyhow 1.0.102 -> 1.0.104`,
`read-process-memory 0.1.6 -> 0.2.0`, and a second `proc-maps` alongside
py-spy's own 0.4.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
korniltsev-grafanista added a commit to grafana/pyroscope-py-spy that referenced this pull request Sep 8, 2026
Carries benfred/remoteprocess#129, which rejects
niche-having types in `copy_struct`/`copy_vec` at compile time so the
previous commit's bug cannot come back.

Pinned by `rev` to the tip of the fork's still-open PR branch, so the rev
must move to the merge commit before this lands.

The fork is at 0.5.3, which needs `proc-maps ^0.5`, so the lockfile also
takes `libc 0.2.185 -> 0.2.189`, `anyhow 1.0.102 -> 1.0.104`,
`read-process-memory 0.1.6 -> 0.2.0`, and a second `proc-maps` alongside
py-spy's own 0.4.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
korniltsev-grafanista added a commit to grafana/pyroscope-py-spy that referenced this pull request Sep 8, 2026
…V) (#3)

* fix: don't read target-process bytes as `bool`

`_PyInterpreterFrame.is_entry` (py 3.11) and the `numpy.bool` payload are
read out of the profiled process, where the byte can hold any value. Only
0 and 1 are valid `bool` bit patterns, so materializing one is UB.

It also gave `_PyInterpreterFrame` a niche: the struct was 80 bytes with
`is_entry` its only niche, so rustc stored the `Err` tag of
`Result<_PyInterpreterFrame, remoteprocess::Error>` inside that byte. A
stale read whose byte was neither 0 nor 1 came back as an `Err` holding 32
bytes of target memory, and dropping it freed a pointer from the profiled
process. That is the `free(): invalid pointer` crash root-caused in
grafana/pyroscope-python#146.

`numpy.bool` locals now render as 0/1 instead of true/false, so
`test_local_vars` is updated to match.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* build: use grafana/pyroscope-remoteprocess with the niche guard

Carries benfred/remoteprocess#129, which rejects
niche-having types in `copy_struct`/`copy_vec` at compile time so the
previous commit's bug cannot come back.

Pinned by `rev` to the tip of the fork's still-open PR branch, so the rev
must move to the merge commit before this lands.

The fork is at 0.5.3, which needs `proc-maps ^0.5`, so the lockfile also
takes `libc 0.2.185 -> 0.2.189`, `anyhow 1.0.102 -> 1.0.104`,
`read-process-memory 0.1.6 -> 0.2.0`, and a second `proc-maps` alongside
py-spy's own 0.4.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@NaurisSadovskis

Copy link
Copy Markdown

@benfred hi, I reported initial issue in Grafana slack channel. Happy to provide more information if needed as it's fairly easy to reproduce it on our end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants