Skip to content

bufferqueue: Use size_t for the memory usage counter - #344

Merged
rikyoz merged 1 commit into
rikyoz:masterfrom
kraj:bufferqueue-size_t
Oct 28, 2026
Merged

rikyoz merged 1 commit into
rikyoz:masterfrom
kraj:bufferqueue-size_t

Conversation

@kraj

@kraj kraj commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

BufferQueue::mMemoryUsage is a std::atomic<std::uint64_t>. On 32-bit targets without native 64-bit atomic instructions (e.g. riscv32), its operations are lowered to __atomic_*_8 libcalls provided by libatomic. bit7z does not link libatomic, so every program linking the static library fails, e.g. the tests when cross-compiling for riscv32 with clang:

ld.lld: error: undefined symbol: __atomic_fetch_add_8
>>> referenced by bufferqueue.cpp:21 (src/internal/bufferqueue.cpp:21)
ld.lld: error: undefined symbol: __atomic_fetch_sub_8
ld.lld: error: undefined symbol: __atomic_load_8

The counter is the total size of the buffers currently held in the queue, i.e. a sum of std::vector::size() values of in-memory objects, so it always fits in a std::size_t. This changes its type to std::atomic<std::size_t>, which is lock-free on every target and avoids the libatomic dependency, rather than adding libatomic to the link of every consumer. All accesses to the counter already happen with the queue mutex held, so behaviour is unchanged.

Tested by cross-building bit7z 4.1.0 with the tests enabled in OpenEmbedded for riscv32 (musl, clang, lld): the library and the test executables link, with no __atomic_* references left.

BufferQueue::mMemoryUsage is a std::atomic<std::uint64_t>. On 32-bit
targets without native 64-bit atomic instructions, such as riscv32, its
operations become __atomic_*_8 libcalls, so every program linking the
static libbit7z needs libatomic, and the tests fail to link:

  ld.lld: error: undefined symbol: __atomic_fetch_add_8
  >>> referenced by bufferqueue.cpp:21 (src/internal/bufferqueue.cpp:21)
  ld.lld: error: undefined symbol: __atomic_fetch_sub_8
  ld.lld: error: undefined symbol: __atomic_load_8

The counter is the total size of the buffers currently held in the
queue, i.e. a sum of std::vector::size() values of in-memory objects,
so it always fits in a std::size_t. Use that type, which keeps the
atomic lock-free on every target and removes the libatomic dependency.

Signed-off-by: Khem Raj <khem.raj@oss.qualcomm.com>
@rikyoz rikyoz self-assigned this Oct 26, 2026
@rikyoz

rikyoz commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Thanks for the fix and the detailed write-up!

This changes its type to std::atomic<std::size_t> [...] rather than adding libatomic to the link of every consumer.

Agreed, I'd rather not make every consumer link libatomic.

All accesses to the counter already happen with the queue mutex held, so behaviour is unchanged.

Yeah, push() and pop() only touch the counter with the mutex held.

The fact it is std::atomic is a leftover of the early stages of development of BufferQueue 😅.

I'll likely drop the std::atomic on develop, since the mutex makes it unnecessary, but I'm merging this as it is for now (after the CI finishes).

Tested by cross-building bit7z 4.1.0 with the tests enabled in OpenEmbedded for riscv32 (musl, clang, lld): the library and the test executables link, with no __atomic_* references left.

Thanks for testing it on the actual target: CI doesn't cover riscv32.

I'm not planning a 4.1.1 just for it, unless you need it, so it'll ship with the next release.

@rikyoz

rikyoz commented Oct 28, 2026

Copy link
Copy Markdown
Owner

Some CI tests fail for reasons outside of this PR, so I'm merging it.

Thank you again!

@rikyoz
rikyoz merged commit da66bb1 into rikyoz:master Oct 28, 2026
1256 of 1353 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants