Prototype: compare node IDs with version/path stale-list GC

[prekucki]

Add a comparison prototype on top of #60. The engine stays unchanged until the new patch is applied.

The patch supports version/path keys and monotonic node IDs. Both variants use the same cache, batch updates, APPEND publication and durable stale-list GC. The remaining queue chunks survive aborts and process restarts. The ID allocator publishes its counter with the same transaction as nodes and head.

Results

Three paired rounds on gf-prov-1, with 1,000 mixed operations per measured commit:

Old versions retained	Version/path ops/s	Node ID ops/s	Change
0	6,452	6,463	+0.2%
8	6,376	6,386	+0.2%
64	6,551	6,501	−0.8%

This implementation gives no repeatable throughput gain. IDs reduce logical key bytes, but MDBX tables use more pages. At W=0, Merkle key/value bytes fall from 23.22 to 21.19 MiB, while Merkle table pages grow from 74.50 to 107.78 MiB. These results do not support replacing version/path addressing with this ID implementation.

All 18 runs have matching roots and final cell/posting digests. Node counts and value sizes also match. The data fit in RAM. The report includes timing, memory, physical sizes and limits. It separates measurements from the untested allocation-order hypothesis.

Validation

• 251 workspace unit/integration tests and 21 doc tests pass.
• Both layouts pass GC tests at W=0/8/64. These check retained graphs after 200 commits, discarded and competing seals, a process restart during bounded collection, format rejection and old MDBX reader snapshots.
• Formatting, Clippy with -D warnings, all-target build, benchmark compilation and harness compilation pass locally.
• The exported patch reconstructs all 18 tested files byte for byte on a clean Tries proposal on versioned keys, GC #60 checkout.

Includes the patch, exact harness and lockfile, reproduction scripts, per-run results and a CI job that applies and tests the prototype. The node-ID format is experimental and has no migration.

[prekucki]

A redb control test confirms that the node-ID space increase depends on the storage backend. redb is slower than MDBX in this workload.

The test ran on gf-prov-1: 30 runs, three rounds, 20,000 active records, 65 warmup commits and 60 measured commits of 1,000 mixed operations. Both backends use durable commits and synchronous stale-list GC. W counts old retained versions in addition to head. Values below are medians of per-run throughput.

Backend / layout	W=0, ops/s	W=64, ops/s
MDBX version/path	6,464	6,523
MDBX node ID	6,462	6,527
redb version/path	5,335	5,221
redb node ID	5,443	5,345

redb is 15.5–20.2% slower in matched runs. Its commit p50 is about 96–97 ms, compared with 65–70 ms for MDBX. Node IDs improve redb throughput by about 2.0–2.4%; their effect on MDBX throughput remains near zero.

Native allocation for the four Merkle tables in the final round:

Backend	W	Version/path, MiB	Node ID, MiB	Change
MDBX	0	74.50	107.78	+44.7%
redb	0	39.60	37.13	−6.3%
MDBX	64	419.07	447.01	+6.7%
redb	64	393.68	382.32	−2.9%

These are live table allocations, not total file sizes. redb uses less space for these tables, but its complete database files are larger. The test does not isolate the allocation or page-layout rule that causes the MDBX increase.

An additional MDBX control replaces APPEND with ordinary insert at W=0. Its median throughput differs by less than 0.4%. APPEND does not explain the main difference in this workload.

All 30 runs have matching commit roots and final cell/posting digests. Node counts and value sizes match. redb also passes the storage contract tests and the 200-commit retained-graph tests for both layouts at W=0/8/64.

The data fit in RAM. redb 4.4.0 uses a 1 GiB page cache; each container has a 24 GiB limit and CPUs 28–31. Backend results include their adapters: redb caches table handles; the current MDBX adapter checks the catalogue for each operation. This is not a bare KV-engine comparison or a production-capacity result.

Recommendation: keep MDBX. Test node IDs assigned in depth/path order next. Measure allocated table bytes and the full seal + commit + GC cycle. This ordering remains an untested hypothesis.