Skip to content

Use pod_vector for mmap zero-copy support in serialized data structures#12

Merged
jermp merged 3 commits intojermp:mainfrom
ratschlab:pr/pod-vector
Mar 9, 2026
Merged

Use pod_vector for mmap zero-copy support in serialized data structures#12
jermp merged 3 commits intojermp:mainfrom
ratschlab:pr/pod-vector

Conversation

@adamant-pwn
Copy link
Contributor

Summary

This PR replaces std::vector with essentials::pod_vector in all serialized data structure members across bits. This is a companion to jermp/essentials#11, which introduces pod_vector and mmap zero-copy loading support in generic_loader.

No behavioral change for existing code — pod_vector is a drop-in replacement for std::vector when used in owned mode (the default). The change enables downstream applications to use generic_loader::set_mmap() to deserialize these data structures via zero-copy views into memory-mapped files. See the essentials PR for full motivation, benchmarks, and design details.

Changes

File Member Before After
bit_vector.hpp m_bits std::vector<uint64_t> pod_vector<uint64_t>
compact_vector.hpp m_bits std::vector<uint64_t> pod_vector<uint64_t>
rank9.hpp m_block_rank_pairs std::vector<uint64_t> pod_vector<uint64_t>
darray.hpp m_positions std::vector<uint64_t> pod_vector<uint64_t>
cache_line_elias_fano.hpp m_data std::vector<uint8_t> pod_vector<uint8_t>
endpoints_sequence.hpp m_hints_0 std::vector<uint8_t> pod_vector<uint8_t>

Additionally, darray.hpp templatizes access() and at() to accept both const uint64_t* and pod_vector<uint64_t>::const_iterator for const correctness.

Depends on: jermp/essentials#11

- bit_vector, compact_vector, rank9, darray: rename pod_vector -> owning_span
- cache_line_elias_fano, endpoints_sequence: refactor encode() to build
  into local std::vector then freeze via owning_span move-construction
- Builder swap patterns converted to move-assignment
- Update essentials submodule
@adamant-pwn
Copy link
Contributor Author

Should I update submodule in the PR to point to jermp/essentials@e2bd442?

@jermp
Copy link
Owner

jermp commented Mar 6, 2026

Yes, thank you!

@adamant-pwn
Copy link
Contributor Author

Alright, should be mergeable now 🙂

@jermp jermp merged commit 10bfac0 into jermp:main Mar 9, 2026
7 checks passed
@jermp
Copy link
Owner

jermp commented Mar 9, 2026

Done! thank you again :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants