refactor(trie): replace bonsai-trie with pathfinder-inspired Merkle trie#463
Draft
refactor(trie): replace bonsai-trie with pathfinder-inspired Merkle trie#463
Conversation
Replace the external `bonsai-trie` dependency with a custom implementation ported from pathfinder's merkle-tree crate. The new implementation uses an immutable tree model with in-memory mutation tracking, proper DB-backed storage with leaf tables, and correct BitVec alignment handling during node serialization. Key changes: - New `MerkleTree` with `Storage` trait abstraction - `DbTrieStorage` and `MemStorage` implementations - Leaf value tables (TrieClassLeaves, TrieContractLeaves, TrieStorageLeaves) - Merkle proof generation support - Fix BitVec head-offset corruption in Compress/Decompress roundtrip Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add an LRU node cache to avoid redundant DB reads during trie operations. Both get() and hash() hit the same TrieNodeEntry, so caching the full entry serves both lookups. Uses quick_cache's unsync::Cache with a 4096-entry capacity and LRU eviction. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Leaf hashes are now stored directly in LeafBinary and LeafEdge node variants instead of in separate leaf tables. This eliminates the need for the Storage::leaf() method, removes 3 DB tables (TrieClassLeaves, TrieContractLeaves, TrieStorageLeaves), and enables correct historical trie access since leaf data is versioned alongside the tree structure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…t approach Load existing trie state from DB into MemStorage before computing mutations, so tree.set() calls operate purely in memory without triggering DB reads. The TrieUpdate is then persisted to DB afterward. - Change MemStorage backing from Vec to HashMap to support arbitrary indices when loading existing DB nodes - Add load_trie_to_memory() BFS loader and *_in_memory() factory methods - Update DbProvider TrieWriter to use in-memory tries for mutation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add comprehensive docs/trie.md covering the trie architecture, node types, storage trait, DB tables, persistence flow, and provider integration. Update docs/database.md with trie table definitions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
bonsai-triedependency with a new Merkle Patricia Trie implementation ported from pathfinder, eliminating the external trie dependency entirelyLeafBinary/LeafEdgevariants) instead of separate leaf tablesMemStorage, compute all mutations in memory, then persist the resultingTrieUpdateto DBDbTrieStoragefor read-only operations (proof generation, root queries)docs/trie.mdcovering architecture, node types, persistence flow, and design decisionsKey changes by commit
d59705af— Core rewrite: NewMerkleTree<H, HEIGHT>withStoragetrait,MemStorage, node types, proof generation. Removebonsai-trie,slabdepsf396e5e8— LRU cache: Addquick_cache-backed LRU toDbTrieStorage(4096 entries) for read-heavy proof/root operationsbafc21e7— Embed leaf hashes: Remove 3 leaf tables, store leaf hashes inLeafBinary/LeafEdgeparent nodes for correct historical accessffa72a3b— In-memory-first mutation:MemStorageusesHashMapfor arbitrary indices, addload_trie_to_memory()BFS loader,TrieWriternow loads→computes→persistse5fb14b6— Documentation: Adddocs/trie.md, updatedocs/database.mdwith trie tablesArchitecture
Test plan
cargo nextest run -p katana-trie— core trie unit testscargo nextest run -p katana-db— DB storage, loading, multi-block testscargo nextest run -p katana-provider -E 'not test(fork)'— provider integrationcargo nextest run -p katana-stage— sync stage tests./scripts/clippy.sh— lint cleancargo +nightly-2025-02-20 fmt --all— format clean🤖 Generated with Claude Code