Migrate from Ethers to Alloy and adopt uint96 asset format - #2
Merged
Conversation
…elocation-safe
The ledger backends (cma_ledger_memory, cma_ledger_single) cache a
`managed_memory &m_memory` reference bound to their own `m_state` member, so a
`cma_ledger_t` is a self-referential C++ object that must never be memcpy-moved
after init. The wrapper held it inline by value, so moving a `Ledger` (e.g.
returning it by value from a constructor) relocated the bytes and left m_memory
dangling — the next map operation then freed a garbage pointer
("free(): invalid pointer", SIGABRT).
The multi-asset path happened to survive via copy-elision; the single-asset path
crashed on boot under the real riscv64 ledger. Box the storage so the C++ object
has a stable heap address and moving `Ledger` moves only the pointer.
Adds a 'host-real' feature so an off-chain consumer (e.g. a sequencer
predicting the machine's ledger) links the *real* C++ libcma instead of
the in-memory mock. build.rs branches the existing non-native path on
host-real: host toolchain (g++/gcc/ar, empty TOOLCHAIN_PREFIX) into a
separate build/host objdir, overriding the Makefile's build/riscv64.
The machine-asset-tools DEFS force SIMD-free/generic Boost paths, so the
32-byte account records are byte-identical across host and riscv64 — the
property that makes off-chain prediction of the on-chain ledger sound.
Smoke tests (tests/host_real_smoke.rs, --features host-real) establish:
- real libcma links and computes balances on the host (not the mock);
- the record layout is balance(u64 LE) | owner(20B) | pad(4B), i.e. the
libcma records image == the Cartesi accounts-drive image;
- init_single_from_buffer zero-initialises (does not re-attach), so
from_dump must rebuild by re-crediting — which reproduces a
byte-identical records prefix (asserted).
BREAKING CHANGE: public `Address`/`U256` are now `alloy_primitives::{Address, U256}`
(was ethers-core). Version bumped 0.0.1 -> 0.1.0.
Alloy migration
- Replace EOL `ethers-core` with `alloy-primitives` + `alloy-dyn-abi` (1.6),
aligned with the resolved 1.6.x the downstream sequencer uses.
- types.rs/helpers.rs: swap the U256/Address method surface
(from_big_endian -> from_be_slice, zero() -> ZERO, to_big_endian(&mut buf)
-> to_be_bytes::<32>(), low_u128() -> to::<u128>()).
- parser.rs: reimplement the ABI layer over a thin `abi_compat` shim on
alloy-dyn-abi. Byte-for-byte equivalence with the old ethers encoding is
load-bearing: the shim uses the unwrapped-params variants
(abi_encode_params/abi_decode_params), NOT the tuple-wrapped
abi_encode/abi_decode, which would silently change voucher/input bytes.
Pinned by the existing parser vectors (all pass).
Hardening for public use (P0-P3)
- Rename the default mock feature `native` -> `mock`; add compile_error!
guards so the three backends (mock/host-real/riscv64) are mutually
exclusive and a real build cannot silently fall back to the mock.
- build.rs: verify a pinned SHA-256 of the fetched nlohmann/json header;
emit a cargo:warning whenever the mock ledger is compiled in.
- Packaging: add LICENSE (MIT), rust-version (MSRV 1.74),
[package.metadata.docs.rs] (offline mock-only docs build), CHANGELOG,
CONTRIBUTING, SECURITY, deny.toml, rust-toolchain.toml, rustfmt.toml.
- CI: add fmt/clippy/doc jobs and a host-real test job; new
tests/host_real_records_layout.rs asserts the 32-byte record layout and
determinism on real libcma.
- Docs: README/STRUCTURE rewritten (3-backend table, footguns, the
reproducibility invariant, !Send/!Sync contract).
Multi-asset ledger tests corrected for real libcma
- The Jun-28 ledger_tests created assets as `TokenAddressId` (the
non-fungible ERC-721/1155 type, whose supply is capped at 1) and deposited
fungible amounts, which real libcma rejects with SupplyOverflow. The mock
never enforced this, so the tests only passed against the mock. Fungible
cases now use `TokenAddress`; a new (real-backend-gated)
test_nft_asset_deposit_is_capped_at_one covers the NFT rule.
- Make the assertions backend-agnostic: assert create==find rather than
absolute ids (real libcma is 0-based, the mock 1-based); probe "not found"
via deposit/Find (both backends error) instead of get_balance (returns 0
for an unknown pair on real libcma); back the ether test with a buffer
(Base is only supported by the memory backend, not `Ledger::new()`); and
fund a separate account to surface InsufficientFunds (supply underflow is
checked before per-account balance).
Verified: host-real 97 tests green (incl. 22/22 ledger); mock 91 green
(21/21 ledger, NFT test gated out); cargo fmt clean. No src changes were
needed for the ledger-test fix -- it is entirely in tests/.
Bump the vendored submodules to the upstream libcma update:
- machine-asset-tools 1288499 -> 19a1e5e (feat: uint96 single asset, PR #4)
- machine-guest-tools 3d838a2 -> 20eba47 (libcmt struct-name / ffi.h fixes)
What changed upstream (machine-asset-tools e4bfc24): the single-asset ledger's
withdrawable drive record widened its balance from uint64 to uint96, consuming
the former 4-byte pad. The 32-byte record is now
balance_lo (u64 LE) | balance_hi (u32 LE) | owner (20B)
so the owner moved from offset 8 to offset 12 and there is no trailing pad.
Total supply and virtual (internal account-id) balances widened to full 256-bit,
and the drive MemoryFooter VERSION bumped 1 -> 2 (a v1 drive is NOT compatible).
Impact on this crate is minimal: the public C API is unchanged (deposits,
withdrawals, balances and total supply already cross the FFI as 256-bit
cma_amount_t), so the Rust wrapper surface (src/) needs no changes — only code
that reads the raw 32-byte records is affected.
- tests/host_real_records_layout.rs: decode the uint96 balance (lo|hi) and read
the owner at offset 12; drop the pad assertion; document format v2.
- Doc comments (src/ledger.rs, src/lib.rs, README.md) updated to the v2 layout.
- CHANGELOG notes the format bump and that downstream raw-record readers
(a sequencer's create_dump/snapshot parser, the emergency-withdrawal output
builder) MUST adopt the offset-12 owner + uint96 balance.
Verified: host-real 97 tests green (records layout confirms lo=balance, hi=0,
owner@12), mock 91 green, cargo fmt clean.
…chine `build.rs` ran the blanket `make third-party`, which — after the v2 Makefile change — also downloads and extracts the prebuilt cartesi-machine emulator .deb (~57 MB, xz-compressed). That library is linked ONLY by the host-side `account-driver-reader` tool, never by `libcma.a`, so pulling it in is pure overhead: it needs `xz` and network for tens of MB of nothing, and it breaks minimal build environments — notably the Cartesi machine riscv64 cross-build image, which ships no `xz` (`tar (grandchild): xz: Cannot exec` -> build fails). Build only the three sub-targets libcma.a needs — `third-party-boost`, `third-party-guest-tools`, `third-party-nlohmann-json`. The `.o` compile rules have no cartesi-machine prerequisite, and `make libcma` no longer triggers it. Verified: host-real build + tests still green; unblocks the riscv64 machine cross-build (Docker) that consumes the crate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.