diff --git a/benchmarks/sigma_v4_results.md b/benchmarks/sigma_v4_results.md new file mode 100644 index 0000000..ac12298 --- /dev/null +++ b/benchmarks/sigma_v4_results.md @@ -0,0 +1,85 @@ +# Sigma-v4 backend performance results + +Measured on the same class of host as the v3 results (4-core x86-64 +container, release builds, default fixed-point config `scale_bits=20, +value_bits=63, intermediate_bits=127`, trivial scaling). "v3" numbers are +from `native_v3_results.md` on identical hardware. The v4 backend has no +proving-key or SRS generation, so there is no warm/cold split: the first +proof of a shape costs the same as every other (v3 "cold" paid keygen on +every fresh process, which is the realistic first-use cost). + +Reproduce with `cargo run --release --example bench_prove -- sigma` +(`halo2` as the last argument benchmarks the legacy backend) and +`python benchmarks/run_benchmarks.py`. + +## Single proof + verify (Rust, `bench_prove`, steady state) + +| shape (in×rank×out) | v3 prove warm | v3 prove cold | v4 prove | speedup warm / cold | v3 verify | v4 verify | proof bytes | +|---|---|---|---|---|---|---|---| +| 2×1×2 | 0.73 s | 2.9 s | 15 ms | 49× / 193× | 0.020 s | 16 ms | 31 KB | +| 8×2×8 | 1.4 s | 6.0 s | 17 ms | 82× / 353× | 0.034 s | 17 ms | 36 KB | +| 16×2×16 | 7.0 s | 27.2 s | 21 ms | 333× / 1295× | 0.16 s | 20 ms | 45 KB | +| 64×4×64 | 10.8 s | 47.2 s | 35 ms | 309× / 1349× | 0.22 s | 36 ms | 99 KB | +| 768×2×256 | 81 s | 429 s | 97 ms | 835× / 4423× | 1.3 s | 101 ms | 327 KB | +| 768×4×768 | infeasible (>15 GB) | infeasible | 183 ms | ∞ | — | 205 ms | 611 KB | +| 768×4×2304 | infeasible (est. >200 GB pk) | infeasible | 470 ms | ∞ | — | 587 ms | 2.2 MB | + +Proving memory drops from gigabytes (12.1 GB at k=19; OOM beyond) to tens of +megabytes for every shape: the halo2 extended-domain evaluations are gone +entirely. + +## End-to-end Python pipeline (`run_benchmarks.py`, 4 workers) + +| shape | invocations | v3 prove wall | v4 prove wall | speedup | v3 verify wall | v4 verify wall | +|---|---|---|---|---|---|---| +| 16×2×16 | 8 | 71.4 s | 0.098 s | **729×** | 3.4 s | 0.23 s | +| 32×4×32 | 6 | 107.4 s | 0.097 s | **1107×** | 3.8 s | 0.61 s | +| 768×4×768 | 4 | infeasible | 0.48 s | ∞ | — | 12.7 s* | + +\* dominated by the one-time adapter-setup verification (Bulletproofs over +all 24,576 committed weights), paid once per adapter per verifier process +and cached afterwards; the per-invocation verification is ~0.2 s. + +## One-time adapter setup (manifest creation) + +The per-weight work that v3 paid inside **every** invocation proof is paid +once per adapter when the manifest is written: Pedersen row commitments, an +aggregated exact range proof for every weight, and a linking proof. + +| adapter shape | weights | setup prove | setup blob | +|---|---|---|---| +| 16×2×16 | 64 | 1.8 s | 17 KB | +| 768×2×256 | 2,048 | 14 s | 485 KB | +| 768×4×2304 | 12,288 | 82 s | 2.5 MB | + +This artifact ships inside the pinned adapter manifest; it contains no +weight or salt material. + +## Real-model end-to-end (distilgpt2 + ng0-k1/distilgpt2-finetuned-es) + +The full quickstart flow from the README — contributor server, base-model +client over sockets, native proof generation, CLI verification — run on the +same 4-core host. The adapter has six rank-16 768×2304 c_attn modules +(~49k quantized weights each); under v3 a single such module exceeded the +host by an order of magnitude (estimated >200 GB of params/keys), so none +of this was previously possible on any realistic machine. + +| stage | wall time | +|---|---| +| manifest write incl. one-time adapter setup proofs, 6 modules | ~6 min (one-time per adapter) | +| inference + proof generation, 60 invocation proofs (6 modules × 10 token rows) | 62 s | +| verification of all 60 proofs incl. one-time adapter-setup verification | 162 s | + +A byte-flipped proof artifact is rejected +(`proof bytes failed native sigma-v4 verification`), as are tampered +statements, transcripts, manifests, and verification keys (covered by the +test suites). + +## Range engines + +Per-invocation proofs default to the sumcheck-based LogUp range engine +(microseconds of field work per range entry). `ZKLORA_RANGE_ENGINE=bulletproofs` +opts into Bulletproofs instead: proofs shrink ~5-8× at ~5-10× slower proving +(still 50-100× faster than v3). Both engines prove the identical exact +intervals under the same discrete-log + Fiat-Shamir assumptions, and the +verifier accepts either. diff --git a/readme.md b/readme.md index 2ba749c..901cfcb 100644 --- a/readme.md +++ b/readme.md @@ -30,7 +30,7 @@ Low-Rank Adaptation (LoRA) is a widely adopted method for customizing large-scal To solve this, we created **zkLoRA** a zero-knowledge verification protocol that relies on commitments, succinct proofs, and multi-party inference to verify exact LoRA delta computation for a pre-agreed adapter without exposing LoRA weights. -This implementation uses a native Halo2 backend for transcript-bound proof artifacts. The v2 proof contract verifies exact quantized LoRA delta correctness for the statement the base user actually sent and received, and binds the proof to a pre-inference adapter manifest. It does not claim an end-to-end proof that the base model computed those activations. +This implementation uses a native commit-and-prove backend (sigma-v4) for transcript-bound proof artifacts. The proof contract verifies exact quantized LoRA delta correctness for the statement the base user actually sent and received, and binds the proof to a pre-inference adapter manifest. It does not claim an end-to-end proof that the base model computed those activations. Verifier trust boundary: `expected_adapters` must be obtained and pinned by the verifier out-of-band before inference starts, for example by recording the exact manifest file or digest. A contributor-generated adapter manifest is only a convenience handoff artifact; if it is first generated after inference or supplied only alongside proofs, it is not trusted verifier input. @@ -348,7 +348,7 @@ For detailed information about the codebase organization and implementation deta ✓Trust-Minimized Delta Verification: Zero-knowledge proofs validate exact quantized LoRA deltas for a pre-agreed adapter -✓Native Halo2 Backend: Proofs no longer depend on EZKL/ONNX artifacts +✓Native Sigma-v4 Backend: Commit-and-prove protocol over ristretto255; per-proof cost no longer scales with adapter size, so every real LoRA shape proves in milliseconds-to-subsecond on a laptop-class host (previously minutes to infeasible). Legacy Halo2 (v3) artifacts remain verifiable ✓Multi-Party Inference: Protected activation exchange between parties @@ -357,10 +357,10 @@ For detailed information about the codebase organization and implementation deta ✓Adapter Weight Privacy: LoRA weights remain confidential while the committed adapter identity is checked -✓Benchmark Required: Real-shape proving and verification performance should be measured for each deployment target (see benchmarks/run_benchmarks.py and cargo run --release --example bench_prove) +✓Benchmark Required: Real-shape proving and verification performance should be measured for each deployment target (see benchmarks/run_benchmarks.py, benchmarks/sigma_v4_results.md, and cargo run --release --example bench_prove) -✓Fast Native Backend (v3): Lookup-based range checks, shape-keyed SRS/key caches, and parallel batch proving/verification deliver order-of-magnitude speedups over v2 while keeping the same statement format and adapter commitment scheme +✓Fast Backend (v4): Pedersen-committed adapters with one-time exact weight range proofs, Fiat-Shamir random-projection sigma protocols for the quantized LoRA relation, and a zero-knowledge sumcheck LogUp range engine deliver 300-1100× faster proving than v3 on previously-feasible shapes and make all real LoRA shapes feasible, with the same statement semantics, the same discrete-log assumption class, and improved (salted, perfectly hiding) adapter commitments @@ -376,7 +376,10 @@ zkLoRA is built upon these outstanding open source projects: | [Transformers](https://github.com/huggingface/transformers) | State-of-the-art Natural Language Processing | | [dusk-merkle](https://github.com/dusk-network/dusk-merkle) | Merkle tree implementation in Rust | | [BLAKE3](https://github.com/BLAKE3-team/BLAKE3) | Cryptographic hash function | -| [Halo2](https://github.com/zcash/halo2) | Native zero-knowledge proving system used by the zkLoRA backend | +| [curve25519-dalek](https://github.com/dalek-cryptography/curve25519-dalek) | Ristretto255 group arithmetic underlying the sigma-v4 backend | +| [Bulletproofs](https://github.com/dalek-cryptography/bulletproofs) | Aggregated range proofs for one-time adapter setup | +| [merlin](https://github.com/dalek-cryptography/merlin) | Fiat-Shamir transcripts | +| [Halo2](https://github.com/zcash/halo2) | Proving system used by the legacy v3 backend |
diff --git a/src/Cargo.toml b/src/Cargo.toml index 3a31ee4..aba8bae 100644 --- a/src/Cargo.toml +++ b/src/Cargo.toml @@ -27,6 +27,12 @@ serde = { version = "1", features = ["derive"] } serde_json = "1" sha2 = "0.10" thiserror = "2" +bulletproofs = "4" +merlin = "3" +curve25519-dalek-ng = "4" +bincode = "1" +base64 = "0.22.1" +rand_chacha = "0.3" [dev-dependencies] tempfile = "3" diff --git a/src/README.md b/src/README.md index cfc7903..2fd5642 100644 --- a/src/README.md +++ b/src/README.md @@ -15,7 +15,9 @@ src/ │ │ └── merkle/ # Rust Merkle tree implementation │ ├── proof_contract.py # Transcript and artifact contract │ └── zk_proof_generator.py # Native proof generation core -├── src/lib.rs # Halo2/PyO3 native prover +├── src/lib.rs # PyO3 native prover (legacy halo2 circuit + fast paths) +├── src/sigma.rs # Sigma-v4 commit-and-prove backend (default) +├── src/logup.rs # Zero-knowledge sumcheck LogUp range engine ├── scripts/ # Sample usage scripts ├── pyproject.toml # Build configuration └── requirements.txt # Dependencies @@ -25,10 +27,10 @@ src/ ### Zero-Knowledge Architecture -The zero-knowledge proof system in zkLoRA is built on transcript-bound LoRA delta statements and native Halo2 proofs. The `zk_proof_generator.py` module orchestrates the proof generation process by: +The zero-knowledge proof system in zkLoRA is built on transcript-bound LoRA delta statements and a native commit-and-prove backend (sigma-v4). The `zk_proof_generator.py` module orchestrates the proof generation process by: 1. Capturing the base user's local transcript of activations and returned LoRA deltas -2. Binding each proof to a verifier-pinned pre-inference adapter manifest with a Poseidon adapter commitment +2. Binding each proof to a verifier-pinned pre-inference adapter manifest whose entries carry salted Pedersen row commitments plus a one-time exact range proof for every committed weight 3. Generating native `.zklora.*` proof artifacts for contributor-side LoRA invocations 4. Verifying proof artifacts against both the base user's transcript and expected adapter manifest before accepting a module @@ -56,18 +58,17 @@ The Rust implementation is wrapped with Python bindings in the `libs/merkle` dir ### Performance Considerations -Native Halo2 performance should be measured for the specific LoRA shapes being proven. The v3 backend keeps the v2 proof contract (same statement format, same Poseidon adapter-commitment scheme) while making proving and verification dramatically faster: +The sigma-v4 backend restructures the statement so that per-proof work no longer scales with the number of adapter weights (the v3 halo2 circuit re-hashed and re-range-checked every weight inside every invocation proof): -- **Lookup-based range checks**: signed interval checks decompose values into window-sized limbs via a running sum constrained against a lookup table, instead of one boolean row per bit. The interval semantics are exact (the top limb is scaled to its residual width), and the circuit drops provably redundant per-product checks whose bounds already follow from the range-checked operands. At a 768×4×2304 LoRA shape this reduces the circuit from k≈26 to k=21 (32× fewer rows). -- **Keyed SRS/proving-key/verifying-key caches**: params and keys are derived deterministically from the statement shape (dims, fixed-point bits, scaling) and reused across invocations of the same module. First proof per shape pays keygen; subsequent proofs and all verifications are keygen-free. Cache sizes are tunable via `ZKLORA_PARAMS_CACHE_CAP`, `ZKLORA_PK_CACHE_CAP`, and `ZKLORA_VK_CACHE_CAP`. -- **Parallel batch operations**: the PyO3 bindings release the GIL, and `generate_proofs` / `batch_verify_proofs` fan out across a thread pool (`ZKLORA_PROVE_WORKERS`, `ZKLORA_VERIFY_WORKERS`). +- **One-time adapter setup**: at manifest time the contributor produces salted Pedersen row commitments to A and B, per-weight commitments, an aggregated Bulletproofs range proof pinning every weight to the exact `[-value_bound, value_bound]` interval, and a Schnorr proof linking the two commitment forms. The pinned adapter commitment string is the SHA-256 of the deterministic commitment core, so pinning the manifest pins the whole setup. +- **Random-projection sigma protocols**: each invocation proof commits to the rounding quotients and remainders of the exact three-stage quantized pipeline, then Fiat-Shamir challenges project the matrix equations onto scalar equations over committed values (Schwartz-Zippel), proven with generalized Schnorr proofs plus one rank-sized quadratic inner-product argument. Per-proof group work is O(in + rank + out). +- **Zero-knowledge LogUp range engine** (`logup.rs`): rounding remainders and quotients are pinned to their exact intervals via 8-bit digit lookups proven with two sumchecks whose round polynomials are sent as Pedersen commitments; all sumcheck verifier relations are linear over committed scalars and fold into one Schnorr proof. `ZKLORA_RANGE_ENGINE=bulletproofs` opts into compact Bulletproofs instead (~5-8× smaller invocation proofs, ~5-10× slower proving). +- **Parallel batch operations**: the PyO3 bindings release the GIL, and `generate_proofs` / `batch_verify_proofs` fan out across a thread pool (`ZKLORA_PROVE_WORKERS`, `ZKLORA_VERIFY_WORKERS`); MSMs, range chunks, and sumcheck rounds parallelize internally with rayon. - **Native fast paths with exact fallbacks**: quantized delta computation and the hiding Merkle commitment have Rust implementations that are value-identical to the Python reference paths (covered by parity tests); the Python implementations remain as exact fallbacks. -Proofs and verifying keys from the v2 backend are not compatible with v3 (the `backend` field in statements changed to `zklora-halo2-v3`), but pinned adapter manifests remain valid: the adapter commitment scheme is unchanged. +Statement semantics are identical to v3: the same canonical half-up rounding, the same exact remainder intervals, the same value/intermediate bounds, the same transcript binding and verifier trust boundary. Binding rests on discrete log over ristretto255 plus SHA-256/BLAKE3 collision resistance (the same assumption class as halo2-IPA over Pasta), and adapter hiding improves: commitments are perfectly hiding, and the deterministic blinding factors are derived from the contributor's secret salt keyed together with the full adapter content, so two adapters never share blindings even when they share a shape and a salt (commitment differences across manifests therefore reveal nothing; the v3 Poseidon chain was unsalted). The salt persists via `ZKLORA_ADAPTER_SALT_FILE` (`LoRAServer` defaults it to a file next to its artifacts; keep it stable across restarts so pinned manifests keep matching proofs). v3 artifacts (schema 2, backend `zklora-halo2-v3`) remain verifiable through the legacy path, covered by a byte-faithful regression test. -Measured on a 4-core machine (warm key cache): a 2×1×2 relation proves in ~0.7s and verifies in ~0.02s (previously ~23s / ~18s), an 8×2×8 relation proves in ~1.4s (previously >5 minutes), and the real 768×4×2304 c_attn shape becomes feasible at k=21. - -Memory note: proving memory is dominated by halo2's extended-domain evaluations (the Poseidon chip's gate degree implies an 8× extended domain), and halo2 0.3 keeps all advice cosets resident during proof creation. Measured on a 15 GB host: a k=19 shape (768×2×256) proves in 81 s warm with a 12.1 GB peak, while k=20 and k=21 shapes (e.g. 768×4×768 and the full 768×4×2304 c_attn) exceed 15 GB and require a 32–64 GB prover host. Verification is lightweight once the verifying key is cached. +Measured on a 4-core host (see `benchmarks/sigma_v4_results.md` for the full tables): 768×2×256 proves in 97 ms and verifies in ~0.1 s (v3: 81 s warm / 429 s cold, 1.3 s verify); 16×2×16 proves in 21 ms (v3: 7 s warm); the real 768×4×768 and 768×4×2304 c_attn shapes, which exceeded a 15 GB host under v3, prove in 183 ms and 470 ms. Proving memory drops from gigabytes to tens of megabytes. There is no warm/cold split because there is no keygen. For detailed usage examples and high-level architecture, please refer to the [main README](../../README.md) in the project root. diff --git a/src/examples/bench_prove.rs b/src/examples/bench_prove.rs index b6aa792..cdced31 100644 --- a/src/examples/bench_prove.rs +++ b/src/examples/bench_prove.rs @@ -1,10 +1,17 @@ //! Benchmark harness for zkLoRA native proving and verification. //! -//! Usage: cargo run --release --example bench_prove -- [reps] +//! Usage: +//! cargo run --release --example bench_prove -- [reps] [backend] +//! +//! `backend` is `sigma` (default, the v4 commit-and-prove backend) or +//! `halo2` (the legacy v3 circuit). use std::time::Instant; -use _native_prover::bench_support::{bench_statement_and_witness, prove_verify_once}; +use _native_prover::bench_support::{ + bench_statement_and_witness, bench_statement_and_witness_v4, prove_verify_once, + prove_verify_once_v4, +}; fn main() { let args: Vec = std::env::args().collect(); @@ -12,13 +19,37 @@ fn main() { let rank: usize = args.get(2).map(|v| v.parse().unwrap()).unwrap_or(2); let out_dim: usize = args.get(3).map(|v| v.parse().unwrap()).unwrap_or(8); let reps: usize = args.get(4).map(|v| v.parse().unwrap()).unwrap_or(1); + let backend: String = args.get(5).cloned().unwrap_or_else(|| "sigma".to_string()); - let (statement_json, witness_json, k) = bench_statement_and_witness(in_dim, rank, out_dim); - println!("shape in_dim={in_dim} rank={rank} out_dim={out_dim} k={k} reps={reps}"); + if backend == "halo2" { + let (statement_json, witness_json, k) = bench_statement_and_witness(in_dim, rank, out_dim); + println!( + "backend=halo2 shape in_dim={in_dim} rank={rank} out_dim={out_dim} k={k} reps={reps}" + ); + for rep in 0..reps { + let start = Instant::now(); + let (prove_ms, verify_ms, proof_len) = + prove_verify_once(&statement_json, &witness_json); + println!( + "rep={rep} prove_ms={prove_ms:.1} verify_ms={verify_ms:.1} total_ms={:.1} proof_bytes={proof_len}", + start.elapsed().as_secs_f64() * 1000.0 + ); + } + return; + } + let setup_start = Instant::now(); + let (statement_json, witness_json, setup_json) = + bench_statement_and_witness_v4(in_dim, rank, out_dim); + println!( + "backend=sigma shape in_dim={in_dim} rank={rank} out_dim={out_dim} reps={reps} adapter_setup_ms={:.1} setup_bytes={}", + setup_start.elapsed().as_secs_f64() * 1000.0, + setup_json.len() + ); for rep in 0..reps { let start = Instant::now(); - let (prove_ms, verify_ms, proof_len) = prove_verify_once(&statement_json, &witness_json); + let (prove_ms, verify_ms, proof_len) = + prove_verify_once_v4(&statement_json, &witness_json, &setup_json); println!( "rep={rep} prove_ms={prove_ms:.1} verify_ms={verify_ms:.1} total_ms={:.1} proof_bytes={proof_len}", start.elapsed().as_secs_f64() * 1000.0 diff --git a/src/src/lib.rs b/src/src/lib.rs index f90cf7d..4502985 100644 --- a/src/src/lib.rs +++ b/src/src/lib.rs @@ -27,6 +27,9 @@ use std::collections::{HashMap, VecDeque}; use std::convert::TryInto; use std::sync::{Arc, Mutex, OnceLock}; +pub(crate) mod logup; +pub mod sigma; + const ADAPTER_COMMITMENT_DOMAIN: u64 = 0x5a4b4c4f5241; // "ZKLORA" const ADAPTER_COMMITMENT_VERSION: u64 = 1; // Must match proof_contract.SCHEMA_VERSION; it is hashed into adapter commitments. @@ -70,14 +73,14 @@ impl CircuitKey { /// Simple bounded FIFO cache. Proving keys and SRS params at large `k` are /// hundreds of MB, so we cap how many distinct shapes stay resident. -struct BoundedCache { +pub(crate) struct BoundedCache { map: HashMap>, order: VecDeque, cap: usize, } impl BoundedCache { - fn new(cap: usize) -> Self { + pub(crate) fn new(cap: usize) -> Self { Self { map: HashMap::new(), order: VecDeque::new(), @@ -85,7 +88,13 @@ impl BoundedCache { } } - fn get_or_create( + /// Cache lookup without creation, for callers that must not run the + /// constructor while holding the cache lock. + pub(crate) fn peek(&mut self, key: &K) -> Option> { + self.map.get(key).cloned() + } + + pub(crate) fn get_or_create( &mut self, key: &K, create: impl FnOnce() -> Result, @@ -113,13 +122,23 @@ fn cache_cap(env_var: &str, default: usize) -> usize { .max(1) } +// Cache lookups never hold their lock across construction: params/keygen use +// rayon internally, and a rayon worker blocked on the cache mutex while the +// lock holder waits for stolen subtasks can deadlock the whole pool. Racing +// constructions are deterministic, so a duplicate insert is harmless. + fn params_for(k: u32) -> Arc> { static CACHE: OnceLock>>> = OnceLock::new(); let cache = CACHE .get_or_init(|| Mutex::new(BoundedCache::new(cache_cap("ZKLORA_PARAMS_CACHE_CAP", 4)))); - let mut guard = cache.lock().expect("params cache poisoned"); - guard - .get_or_create::(&k, || Ok(Params::new(k))) + if let Some(hit) = cache.lock().expect("params cache poisoned").peek(&k) { + return hit; + } + let params = Params::new(k); + cache + .lock() + .expect("params cache poisoned") + .get_or_create::(&k, || Ok(params)) .expect("params creation is infallible") } @@ -131,11 +150,15 @@ fn proving_key_for( static CACHE: OnceLock>>> = OnceLock::new(); let cache = CACHE.get_or_init(|| Mutex::new(BoundedCache::new(cache_cap("ZKLORA_PK_CACHE_CAP", 2)))); - let mut guard = cache.lock().expect("pk cache poisoned"); - guard.get_or_create(key, || { - let vk = keygen_vk(params, empty_circuit).map_err(|e| NativeError::Halo2(e.to_string()))?; - keygen_pk(params, vk, empty_circuit).map_err(|e| NativeError::Halo2(e.to_string())) - }) + if let Some(hit) = cache.lock().expect("pk cache poisoned").peek(key) { + return Ok(hit); + } + let vk = keygen_vk(params, empty_circuit).map_err(|e| NativeError::Halo2(e.to_string()))?; + let pk = keygen_pk(params, vk, empty_circuit).map_err(|e| NativeError::Halo2(e.to_string()))?; + cache + .lock() + .expect("pk cache poisoned") + .get_or_create(key, || Ok::<_, NativeError>(pk)) } fn verifying_key_for( @@ -147,10 +170,14 @@ fn verifying_key_for( OnceLock::new(); let cache = CACHE.get_or_init(|| Mutex::new(BoundedCache::new(cache_cap("ZKLORA_VK_CACHE_CAP", 8)))); - let mut guard = cache.lock().expect("vk cache poisoned"); - guard.get_or_create(key, || { - keygen_vk(params, empty_circuit).map_err(|e| NativeError::Halo2(e.to_string())) - }) + if let Some(hit) = cache.lock().expect("vk cache poisoned").peek(key) { + return Ok(hit); + } + let vk = keygen_vk(params, empty_circuit).map_err(|e| NativeError::Halo2(e.to_string()))?; + cache + .lock() + .expect("vk cache poisoned") + .get_or_create(key, || Ok::<_, NativeError>(vk)) } #[derive(Debug, thiserror::Error)] @@ -201,7 +228,7 @@ pub struct NativeWitness { } #[derive(Clone, Debug, Serialize, Deserialize)] -struct AdapterCommitmentInput { +pub(crate) struct AdapterCommitmentInput { pub schema_version: u64, pub in_dim: usize, pub rank: usize, @@ -760,8 +787,8 @@ impl LoraCircuit { )); } for (label, values) in [ - ("x", self.x.iter().copied().collect::>()), - ("delta", self.delta.iter().copied().collect::>()), + ("x", self.x.to_vec()), + ("delta", self.delta.to_vec()), ("A", self.a.iter().flatten().copied().collect::>()), ("B", self.b.iter().flatten().copied().collect::>()), ] { @@ -1154,6 +1181,25 @@ fn canonical_remainder_interval(denominator: &BigInt) -> (BigInt, BigInt) { (-&floor_half, (denominator - BigInt::one()) / &two) } +/// Canonical remainder interval with NativeError reporting (sigma backend). +pub(crate) fn canonical_remainder_interval_int(denominator: &BigInt) -> (BigInt, BigInt) { + canonical_remainder_interval(denominator) +} + +/// Canonical half-up rounding division with NativeError reporting. +pub(crate) fn div_round_canonical_int( + numerator: &BigInt, + denominator: &BigInt, +) -> Result { + if denominator <= &BigInt::zero() { + return Err(NativeError::InvalidDimensions( + "denominator must be positive".into(), + )); + } + let floor_half = denominator / BigInt::from(2u8); + Ok((numerator + &floor_half).div_floor(denominator)) +} + fn div_round_to_canonical_interval( numerator: &BigInt, denominator: &BigInt, @@ -1949,6 +1995,60 @@ fn merkle_root(py: pyo3::Python<'_>, values: Vec, nonce: Vec) -> pyo3:: py.detach(|| Ok(merkle_root_f64(&values, &nonce).to_vec())) } +#[cfg(feature = "python")] +#[pyo3::pyfunction] +fn adapter_setup_v4( + py: pyo3::Python<'_>, + adapter_json: &str, + salt_hex: &str, +) -> pyo3::PyResult { + py.detach(|| Ok(sigma::adapter_setup_json(adapter_json, salt_hex)?)) +} + +#[cfg(feature = "python")] +#[pyo3::pyfunction] +fn adapter_commitment_v4( + py: pyo3::Python<'_>, + adapter_json: &str, + salt_hex: &str, +) -> pyo3::PyResult { + py.detach(|| Ok(sigma::adapter_commitment_v4_string(adapter_json, salt_hex)?)) +} + +#[cfg(feature = "python")] +#[pyo3::pyfunction] +fn verify_adapter_setup_v4(py: pyo3::Python<'_>, setup_json: &str) -> pyo3::PyResult { + py.detach(|| Ok(sigma::verify_adapter_setup_json(setup_json)?)) +} + +#[cfg(feature = "python")] +#[pyo3::pyfunction] +fn prove_v4( + py: pyo3::Python<'_>, + statement_json: &str, + witness_json: &str, + salt_hex: &str, +) -> pyo3::PyResult> { + py.detach(|| { + Ok(sigma::prove_v4_bytes( + statement_json, + witness_json, + salt_hex, + )?) + }) +} + +#[cfg(feature = "python")] +#[pyo3::pyfunction] +fn verify_v4( + py: pyo3::Python<'_>, + statement_json: &str, + proof: &[u8], + setup_json: &str, +) -> pyo3::PyResult { + py.detach(|| Ok(sigma::verify_v4_bytes(statement_json, proof, setup_json)?)) +} + #[cfg(feature = "python")] #[pyo3::pymodule] fn _native_prover(m: &pyo3::Bound<'_, pyo3::types::PyModule>) -> pyo3::PyResult<()> { @@ -1963,6 +2063,11 @@ fn _native_prover(m: &pyo3::Bound<'_, pyo3::types::PyModule>) -> pyo3::PyResult< m.add_function(wrap_pyfunction!(compute_delta_rows, m)?)?; m.add_function(wrap_pyfunction!(quantize_rows, m)?)?; m.add_function(wrap_pyfunction!(merkle_root, m)?)?; + m.add_function(wrap_pyfunction!(adapter_setup_v4, m)?)?; + m.add_function(wrap_pyfunction!(adapter_commitment_v4, m)?)?; + m.add_function(wrap_pyfunction!(verify_adapter_setup_v4, m)?)?; + m.add_function(wrap_pyfunction!(prove_v4, m)?)?; + m.add_function(wrap_pyfunction!(verify_v4, m)?)?; Ok(()) } @@ -2085,6 +2190,61 @@ pub mod bench_support { let verify_ms = verify_start.elapsed().as_secs_f64() * 1000.0; (prove_ms, verify_ms, proof.len()) } + + pub const BENCH_SALT: &str = "5eed5eed5eed5eed5eed5eed5eed5eed5eed5eed5eed5eed5eed5eed5eed5eed"; + + /// Same deterministic statement/witness as the halo2 bench, but with the + /// adapter commitment string of the sigma-v4 backend. Returns + /// (statement_json, witness_json, adapter_setup_json). + pub fn bench_statement_and_witness_v4( + in_dim: usize, + rank: usize, + out_dim: usize, + ) -> (String, String, String) { + let (statement_json, witness_json, _k) = bench_statement_and_witness(in_dim, rank, out_dim); + let mut statement: NativeStatement = + serde_json::from_str(&statement_json).expect("statement json"); + let witness: NativeWitness = serde_json::from_str(&witness_json).expect("witness json"); + let adapter_input = AdapterCommitmentInput { + schema_version: sigma::SIGMA_SCHEMA_VERSION, + in_dim, + rank, + out_dim, + fixed_point: statement.fixed_point.clone(), + scaling_num: statement.scaling_num, + scaling_den: statement.scaling_den, + a: witness.a.clone(), + b: witness.b.clone(), + }; + let adapter_json = serde_json::to_string(&adapter_input).expect("adapter json"); + let setup_json = + sigma::adapter_setup_json(&adapter_json, BENCH_SALT).expect("adapter setup"); + statement.adapter_commitment = + sigma::adapter_commitment_v4_string(&adapter_json, BENCH_SALT).expect("commitment"); + ( + serde_json::to_string(&statement).expect("statement json"), + witness_json, + setup_json, + ) + } + + pub fn prove_verify_once_v4( + statement_json: &str, + witness_json: &str, + setup_json: &str, + ) -> (f64, f64, usize) { + let prove_start = std::time::Instant::now(); + let proof = + sigma::prove_v4_bytes(statement_json, witness_json, BENCH_SALT).expect("prove v4"); + let prove_ms = prove_start.elapsed().as_secs_f64() * 1000.0; + let verify_start = std::time::Instant::now(); + assert!( + sigma::verify_v4_bytes(statement_json, &proof, setup_json).expect("verify v4 call"), + "v4 proof must verify" + ); + let verify_ms = verify_start.elapsed().as_secs_f64() * 1000.0; + (prove_ms, verify_ms, proof.len()) + } } #[cfg(test)] diff --git a/src/src/logup.rs b/src/src/logup.rs new file mode 100644 index 0000000..c77d90f --- /dev/null +++ b/src/src/logup.rs @@ -0,0 +1,1253 @@ +//! Zero-knowledge LogUp range engine. +//! +//! Proves that a batch of Pedersen-committed values v_e lie in [0, 2^{n_e}) +//! by decomposing each into 8-bit digits and proving every digit lies in the +//! table T = [0, 256) with a LogUp lookup argument: +//! +//! `sum_i 1/(alpha - d_i) == sum_t m_t/(alpha - t)` +//! +//! for a Fiat-Shamir challenge alpha drawn after the digit and multiplicity +//! commitments. The rational identity is enforced through committed helper +//! vectors h_i = 1/(alpha - d_i) and g_t = m_t/(alpha - t) whose defining +//! products are checked by two sumchecks over multilinear extensions. +//! +//! Zero knowledge is structural rather than analytic: every sumcheck round +//! polynomial coefficient is sent as a Pedersen commitment, never in the +//! clear, and all sumcheck verifier checks (round consistency, claim +//! chaining, final evaluation, MLE evaluations of the committed vectors, +//! digit recomposition) are linear relations over committed scalars, so they +//! are discharged by one generalized Schnorr proof. The single bilinear +//! relation (the final evaluation product E_h * E_d) uses a standard +//! product sigma protocol. Everything the verifier sees is a perfectly +//! hiding commitment, a uniformly distributed Schnorr response, or public +//! data. +//! +//! Soundness chain: Pedersen binding fixes all committed scalars; the +//! Schnorr extracts openings satisfying every sumcheck verifier relation +//! with honestly Fiat-Shamir-derived round challenges, so standard sumcheck +//! soundness applies; the two sumchecks force h and g to satisfy their +//! defining products on the whole cube (Schwartz-Zippel over tau); the +//! LogUp lemma then forces every digit into the table; and the random +//! projection over rho forces each committed value to equal its digit +//! recomposition mod l, hence to be an integer in [0, 2^{n_e}). All under +//! the same discrete-log + Fiat-Shamir assumptions as the rest of the +//! backend. + +use curve25519_dalek_ng::constants::RISTRETTO_BASEPOINT_TABLE; +use curve25519_dalek_ng::ristretto::RistrettoPoint; +use curve25519_dalek_ng::scalar::Scalar; +use curve25519_dalek_ng::traits::VartimeMultiscalarMul; +use merlin::Transcript; +use rayon::prelude::*; +use serde::{Deserialize, Serialize}; + +use crate::sigma::{ + absorb_points, blinding_table, compress, decompress, g_basis, par_msm, pc_gens, random_scalar, + transcript_scalar, transcript_scalars, RangeEntry, +}; +use crate::NativeError; + +const TABLE_BITS: usize = 8; +const TABLE_SIZE: usize = 1 << TABLE_BITS; + +// --------------------------------------------------------------------------- +// Small utilities +// --------------------------------------------------------------------------- + +fn batch_invert(values: &[Scalar]) -> Result, NativeError> { + // Montgomery batch inversion; rejects zero inputs (negligible-probability + // challenge collision -- the prover simply errors and the statement can + // be re-proven, soundness is unaffected). + let mut prefix = Vec::with_capacity(values.len()); + let mut acc = Scalar::one(); + for value in values { + if *value == Scalar::zero() { + return Err(NativeError::Halo2( + "logup challenge collided with a table value".into(), + )); + } + prefix.push(acc); + acc *= value; + } + let mut inv_acc = acc.invert(); + let mut out = vec![Scalar::zero(); values.len()]; + for index in (0..values.len()).rev() { + out[index] = inv_acc * prefix[index]; + inv_acc *= values[index]; + } + Ok(out) +} + +/// eq-weight table for challenge vector rs; index bits are consumed most +/// significant first, matching the sumcheck fold order below. +fn eq_table(rs: &[Scalar]) -> Vec { + let mut table = vec![Scalar::one()]; + for r in rs { + let mut next = Vec::with_capacity(table.len() * 2); + for w in &table { + next.push(w * (Scalar::one() - r)); + next.push(w * r); + } + table = next; + } + table +} + +fn eq_point(a: &[Scalar], b: &[Scalar]) -> Scalar { + a.iter() + .zip(b.iter()) + .map(|(x, y)| x * y + (Scalar::one() - x) * (Scalar::one() - y)) + .product() +} + +fn inner(a: &[Scalar], b: &[Scalar]) -> Scalar { + a.iter().zip(b.iter()).map(|(x, y)| x * y).sum() +} + +fn poly_eval4(coeffs: &[Scalar; 4], x: &Scalar) -> Scalar { + coeffs[0] + x * (coeffs[1] + x * (coeffs[2] + x * coeffs[3])) +} + +fn vector_commit(values: &[Scalar], blinding: &Scalar) -> RistrettoPoint { + let basis = g_basis(values.len()); + par_msm(values, &basis[..values.len()]) + blinding_table() * blinding +} + +fn scalar_commit(value: &Scalar, blinding: &Scalar) -> RistrettoPoint { + value * &RISTRETTO_BASEPOINT_TABLE + blinding_table() * blinding +} + +// --------------------------------------------------------------------------- +// zk sumcheck with committed round polynomials +// --------------------------------------------------------------------------- + +/// Prover state for one sumcheck of G(X) = e(X) * (p(X) * q(X) - w(X)) over +/// {0,1}^m, where all factors are multilinear. Round coefficients are +/// returned as values + blindings; their commitments are absorbed before +/// each challenge. +struct SumcheckRun { + coeff_values: Vec<[Scalar; 4]>, + coeff_blindings: Vec<[Scalar; 4]>, + coeff_commitments: Vec<[[u8; 32]; 4]>, + rs: Vec, +} + +fn coeffs_from_evals(g0: Scalar, g1: Scalar, g2: Scalar, g3: Scalar) -> [Scalar; 4] { + let inv2 = Scalar::from(2u64).invert(); + let inv6 = Scalar::from(6u64).invert(); + let c3 = (g3 - g2 * Scalar::from(3u64) + g1 * Scalar::from(3u64) - g0) * inv6; + let c2 = (g2 - g1 * Scalar::from(2u64) + g0 - c3 * Scalar::from(6u64)) * inv2; + let c0 = g0; + let c1 = g1 - c0 - c2 - c3; + [c0, c1, c2, c3] +} + +fn fold(values: &mut Vec, r: &Scalar) { + let half = values.len() / 2; + for index in 0..half { + let lo = values[index]; + let hi = values[index + half]; + values[index] = lo + r * (hi - lo); + } + values.truncate(half); +} + +fn sumcheck_prove( + transcript: &mut Transcript, + label: &'static [u8], + mut e: Vec, + mut p: Vec, + mut q: Vec, + mut w: Vec, +) -> SumcheckRun { + let mut run = SumcheckRun { + coeff_values: Vec::new(), + coeff_blindings: Vec::new(), + coeff_commitments: Vec::new(), + rs: Vec::new(), + }; + while e.len() > 1 { + let half = e.len() / 2; + let eval_pair = |index: usize| { + let mut local = [Scalar::zero(); 4]; + let (e0, e1) = (e[index], e[index + half]); + let (p0, p1) = (p[index], p[index + half]); + let (q0, q1) = (q[index], q[index + half]); + let (w0, w1) = (w[index], w[index + half]); + let (de, dp, dq, dw) = (e1 - e0, p1 - p0, q1 - q0, w1 - w0); + let (mut ev, mut pv, mut qv, mut wv) = (e0, p0, q0, w0); + local[0] = ev * (pv * qv - wv); + for eval in local.iter_mut().skip(1) { + ev += de; + pv += dp; + qv += dq; + wv += dw; + *eval = ev * (pv * qv - wv); + } + local + }; + let evals = if half >= 2048 { + (0..half).into_par_iter().map(eval_pair).reduce( + || [Scalar::zero(); 4], + |mut acc, local| { + for (a, l) in acc.iter_mut().zip(local.iter()) { + *a += l; + } + acc + }, + ) + } else { + let mut acc = [Scalar::zero(); 4]; + for index in 0..half { + let local = eval_pair(index); + for (a, l) in acc.iter_mut().zip(local.iter()) { + *a += l; + } + } + acc + }; + let coeffs = coeffs_from_evals(evals[0], evals[1], evals[2], evals[3]); + let blindings = [ + random_scalar(), + random_scalar(), + random_scalar(), + random_scalar(), + ]; + let commitments: [[u8; 32]; 4] = + std::array::from_fn(|k| compress(&scalar_commit(&coeffs[k], &blindings[k]))); + for commitment in &commitments { + transcript.append_message(label, commitment); + } + let r = transcript_scalar(transcript, label); + fold(&mut e, &r); + fold(&mut p, &r); + fold(&mut q, &r); + fold(&mut w, &r); + run.coeff_values.push(coeffs); + run.coeff_blindings.push(blindings); + run.coeff_commitments.push(commitments); + run.rs.push(r); + } + run +} + +/// Verifier-side challenge replay: absorb the committed round polynomials +/// and re-derive the round challenges. +fn sumcheck_replay( + transcript: &mut Transcript, + label: &'static [u8], + rounds: &[[[u8; 32]; 4]], +) -> Vec { + rounds + .iter() + .map(|commitments| { + for commitment in commitments { + transcript.append_message(label, commitment); + } + transcript_scalar(transcript, label) + }) + .collect() +} + +// --------------------------------------------------------------------------- +// Proof structures +// --------------------------------------------------------------------------- + +#[derive(Clone, Serialize, Deserialize)] +struct ProductProof { + a2: [u8; 32], + a3: [u8; 32], + s_y: Scalar, + s_by: Scalar, + s_t: Scalar, +} + +/// One scalar witness with its commitment blinding (for the mega-Schnorr). +#[derive(Clone, Serialize, Deserialize)] +struct ScalarOpen { + sigma: Scalar, + sigma_b: Scalar, +} + +#[derive(Clone, Serialize, Deserialize)] +struct MegaSchnorr { + r_d: [u8; 32], + r_m: [u8; 32], + r_h: [u8; 32], + r_g: [u8; 32], + r_v: [u8; 32], + r_coeffs_b: Vec<[[u8; 32]; 4]>, + r_coeffs_c: Vec<[[u8; 32]; 4]>, + r_eh: [u8; 32], + r_ed: [u8; 32], + r_eg: [u8; 32], + r_em: [u8; 32], + r_z: [u8; 32], + t_constraints: Vec, + sigma_d: Vec, + sigma_bd: Scalar, + sigma_m: Vec, + sigma_bm: Scalar, + sigma_h: Vec, + sigma_bh: Scalar, + sigma_g: Vec, + sigma_bg: Scalar, + sigma_v: ScalarOpen, + sigma_coeffs_b: Vec<[ScalarOpen; 4]>, + sigma_coeffs_c: Vec<[ScalarOpen; 4]>, + sigma_eh: ScalarOpen, + sigma_ed: ScalarOpen, + sigma_eg: ScalarOpen, + sigma_em: ScalarOpen, + sigma_z: ScalarOpen, +} + +#[derive(Clone, Serialize, Deserialize)] +pub(crate) struct LogUpProof { + c_d: [u8; 32], + c_m: [u8; 32], + c_h: [u8; 32], + c_g: [u8; 32], + rounds_b: Vec<[[u8; 32]; 4]>, + rounds_c: Vec<[[u8; 32]; 4]>, + c_eh: [u8; 32], + c_ed: [u8; 32], + c_eg: [u8; 32], + c_em: [u8; 32], + c_z: [u8; 32], + product: ProductProof, + schnorr: MegaSchnorr, +} + +// --------------------------------------------------------------------------- +// Shared linear-constraint system +// --------------------------------------------------------------------------- + +/// All witness scalars of the linear system, in one struct so prover +/// randomizers, witnesses, and verifier responses share the evaluation code. +struct LinearWitness { + d: Vec, + m: Vec, + h: Vec, + g: Vec, + v: Scalar, + cb: Vec<[Scalar; 4]>, + cc: Vec<[Scalar; 4]>, + eh: Scalar, + ed: Scalar, + eg: Scalar, + em: Scalar, + z: Scalar, +} + +struct PublicData { + /// recomposition weights over the digit domain (rho_e * 2^{8k}) + weights: Vec, + alpha: Scalar, + rs_b: Vec, + rs_c: Vec, + eq_r_b: Vec, + eq_r_c: Vec, + /// eq(tau, r) for each sumcheck + eqv_b: Scalar, + eqv_c: Scalar, + /// identity MLE of the table evaluated at rs_c + identv: Scalar, + /// MLE of the public real-slot indicator evaluated at rs_b + es: Scalar, +} + +/// Evaluate every linear constraint L_i(W). The proof is valid iff +/// L_i(witness) = K_i with K given by `constraint_constants`. +fn constraints_eval(w: &LinearWitness, p: &PublicData) -> Vec { + let mut out = Vec::new(); + // (A) sum h == sum g + let sum_h: Scalar = w.h.iter().sum(); + let sum_g: Scalar = w.g.iter().sum(); + out.push(sum_h - sum_g); + // (D) digit recomposition: == V + out.push(inner(&p.weights, &w.d) - w.v); + // sumcheck B round consistency: g_j(0) + g_j(1) == claim_j + for (j, coeffs) in w.cb.iter().enumerate() { + let round_sum = coeffs[0] + coeffs[0] + coeffs[1] + coeffs[2] + coeffs[3]; + let claim = if j == 0 { + Scalar::zero() + } else { + poly_eval4(&w.cb[j - 1], &p.rs_b[j - 1]) + }; + out.push(round_sum - claim); + } + // sumcheck B final: g_m(r_m) == eqv_b * (alpha*E_h - z - es), where es + // is the public real-slot-indicator MLE evaluated at r (the -es constant + // lands in `constraint_constants`). + let last_b = poly_eval4( + w.cb.last().expect("at least one round"), + p.rs_b.last().expect("at least one round"), + ); + out.push(last_b - p.eqv_b * (p.alpha * w.eh - w.z)); + // sumcheck C round consistency + for (j, coeffs) in w.cc.iter().enumerate() { + let round_sum = coeffs[0] + coeffs[0] + coeffs[1] + coeffs[2] + coeffs[3]; + let claim = if j == 0 { + Scalar::zero() + } else { + poly_eval4(&w.cc[j - 1], &p.rs_c[j - 1]) + }; + out.push(round_sum - claim); + } + // sumcheck C final: g_8(r_8) == eqv_c * (E_g*(alpha - identv) - E_m) + let last_c = poly_eval4( + w.cc.last().expect("table rounds"), + p.rs_c.last().expect("table rounds"), + ); + out.push(last_c - p.eqv_c * ((p.alpha - p.identv) * w.eg - w.em)); + // MLE evaluations of the committed vectors + out.push(w.eh - inner(&p.eq_r_b, &w.h)); + out.push(w.ed - inner(&p.eq_r_b, &w.d)); + out.push(w.eg - inner(&p.eq_r_c, &w.g)); + out.push(w.em - inner(&p.eq_r_c, &w.m)); + out +} + +/// Constants K_i such that a valid witness satisfies L_i(witness) = K_i. +fn constraint_constants(p: &PublicData) -> Vec { + let mut out = vec![Scalar::zero(); 2 + p.rs_b.len()]; + // final B constraint carries the public indicator term: L = -eqv_b * es + out.push(-(p.eqv_b * p.es)); + out.extend(vec![Scalar::zero(); p.rs_c.len() + 1]); + out.extend(vec![Scalar::zero(); 4]); + out +} + +impl LinearWitness { + /// Schnorr randomizers. Pad coordinates of d and h are publicly zero, so + /// their randomizers are zero too: responses stay zero there and the + /// announcement MSMs skip them. + fn random_like(&self, real_len: usize) -> Self { + let rand_vec = |len: usize| (0..len).map(|_| random_scalar()).collect::>(); + let rand_vec_real = |len: usize| { + let mut v = (0..real_len.min(len)) + .map(|_| random_scalar()) + .collect::>(); + v.resize(len, Scalar::zero()); + v + }; + let rand_coeffs = |len: usize| { + (0..len) + .map(|_| std::array::from_fn(|_| random_scalar())) + .collect::>() + }; + Self { + d: rand_vec_real(self.d.len()), + m: rand_vec(self.m.len()), + h: rand_vec_real(self.h.len()), + g: rand_vec(self.g.len()), + v: random_scalar(), + cb: rand_coeffs(self.cb.len()), + cc: rand_coeffs(self.cc.len()), + eh: random_scalar(), + ed: random_scalar(), + eg: random_scalar(), + em: random_scalar(), + z: random_scalar(), + } + } + + fn respond(&self, rho: &Self, c: &Scalar) -> Self { + let vec_resp = |r: &[Scalar], w: &[Scalar]| { + r.iter() + .zip(w.iter()) + .map(|(r, w)| r + c * w) + .collect::>() + }; + let coeff_resp = |r: &[[Scalar; 4]], w: &[[Scalar; 4]]| { + r.iter() + .zip(w.iter()) + .map(|(r, w)| std::array::from_fn(|k| r[k] + c * w[k])) + .collect::>() + }; + Self { + d: vec_resp(&rho.d, &self.d), + m: vec_resp(&rho.m, &self.m), + h: vec_resp(&rho.h, &self.h), + g: vec_resp(&rho.g, &self.g), + v: rho.v + c * self.v, + cb: coeff_resp(&rho.cb, &self.cb), + cc: coeff_resp(&rho.cc, &self.cc), + eh: rho.eh + c * self.eh, + ed: rho.ed + c * self.ed, + eg: rho.eg + c * self.eg, + em: rho.em + c * self.em, + z: rho.z + c * self.z, + } + } +} + +// --------------------------------------------------------------------------- +// Digit layout +// --------------------------------------------------------------------------- + +struct DigitLayout { + /// (entry index, digit shift) for each digit slot; pads excluded. + positions: Vec<(usize, usize)>, + padded_len: usize, +} + +fn digit_layout(entries: &[RangeEntry]) -> Result { + let mut positions = Vec::new(); + for (index, entry) in entries.iter().enumerate() { + if entry.n % TABLE_BITS != 0 || entry.n == 0 || entry.n > 64 { + return Err(NativeError::InvalidDimensions( + "range entry width must be a multiple of 8".into(), + )); + } + for digit in 0..(entry.n / TABLE_BITS) { + positions.push((index, digit)); + } + } + if positions.is_empty() { + return Err(NativeError::InvalidDimensions( + "no range entries to prove".into(), + )); + } + let padded_len = positions.len().next_power_of_two().max(2); + Ok(DigitLayout { + positions, + padded_len, + }) +} + +fn recomposition_weights(layout: &DigitLayout, rho: &[Scalar]) -> Vec { + let mut weights = vec![Scalar::zero(); layout.padded_len]; + for (slot, (entry_index, digit)) in layout.positions.iter().enumerate() { + weights[slot] = rho[*entry_index] * Scalar::from(1u64 << (TABLE_BITS * digit)); + } + weights +} + +// --------------------------------------------------------------------------- +// Prove / verify +// --------------------------------------------------------------------------- + +pub(crate) fn prove( + transcript: &mut Transcript, + entries: &[RangeEntry], +) -> Result { + let timing = std::env::var("ZKLORA_V4_TIMING").is_ok(); + let mut mark = std::time::Instant::now(); + let mut lap = |label: &str| { + if timing { + eprintln!(" logup {label}: {:?}", mark.elapsed()); + } + mark = std::time::Instant::now(); + }; + let gens = pc_gens(); + let layout = digit_layout(entries)?; + let n_pad = layout.padded_len; + + // Digits, zero-padded to a power of two. Pad coordinates are public + // zeros: they carry zero recomposition weight, the lookup claim is + // gated by the public indicator vector s (1 on real slots, 0 on pads), + // and announcement randomizers vanish there, so pads cost nothing in + // any MSM and a prover stuffing junk into pad coordinates constrains + // nothing (h is forced to 0 at pads and d pads are never used). + let real_len = layout.positions.len(); + let mut digits = vec![Scalar::zero(); n_pad]; + let mut digit_ints = vec![0u64; n_pad]; + for (slot, (entry_index, digit)) in layout.positions.iter().enumerate() { + let value = (entries[*entry_index].value >> (TABLE_BITS * digit)) & 0xff; + digits[slot] = Scalar::from(value); + digit_ints[slot] = value; + } + let mut multiplicities = vec![0u64; TABLE_SIZE]; + for value in &digit_ints[..real_len] { + multiplicities[*value as usize] += 1; + } + let m_scalars: Vec = multiplicities.iter().map(|m| Scalar::from(*m)).collect(); + + transcript.append_message(b"logup", b"v1"); + transcript.append_u64(b"logup-n", n_pad as u64); + // Bind the entries this engine is proving directly into its own + // transcript (defense in depth: the caller already absorbed every class + // commitment, but the recomposition challenge rho must be sound even if + // this engine is ever reused on a transcript that did not). + for entry in entries { + transcript.append_u64(b"logup-entry-bits", entry.n as u64); + transcript.append_message(b"logup-entry-commitment", &entry.commitment); + } + let b_d = random_scalar(); + let b_m = random_scalar(); + let (c_d, c_m) = rayon::join( + || vector_commit(&digits, &b_d), + || vector_commit(&m_scalars, &b_m), + ); + transcript.append_message(b"logup-c-d", &compress(&c_d)); + transcript.append_message(b"logup-c-m", &compress(&c_m)); + lap("digits+commit-dm"); + let alpha = transcript_scalar(transcript, b"logup-alpha"); + let rho = transcript_scalars(transcript, b"logup-rho", entries.len()); + lap("challenges-rho"); + + // Helper vectors h = s/(alpha - d), g = m/(alpha - t). + let k_vec: Vec = digits.iter().map(|d| alpha - d).collect(); + let mut h = batch_invert(&k_vec[..real_len])?; + h.resize(n_pad, Scalar::zero()); + let table_k: Vec = (0..TABLE_SIZE) + .map(|t| alpha - Scalar::from(t as u64)) + .collect(); + let table_k_inv = batch_invert(&table_k)?; + let g: Vec = m_scalars + .iter() + .zip(table_k_inv.iter()) + .map(|(m, inv)| m * inv) + .collect(); + let b_h = random_scalar(); + let b_g = random_scalar(); + let (c_h, c_g) = rayon::join(|| vector_commit(&h, &b_h), || vector_commit(&g, &b_g)); + transcript.append_message(b"logup-c-h", &compress(&c_h)); + transcript.append_message(b"logup-c-g", &compress(&c_g)); + + lap("h-g-commit"); + let m_rounds = n_pad.trailing_zeros() as usize; + let tau_b = transcript_scalars(transcript, b"logup-tau-b", m_rounds); + let tau_c = transcript_scalars(transcript, b"logup-tau-c", TABLE_BITS); + + // Sumcheck B over the digit domain: e*(h*k - s) sums to zero, with s + // the public real-slot indicator. + let mut indicator = vec![Scalar::one(); real_len]; + indicator.resize(n_pad, Scalar::zero()); + let run_b = sumcheck_prove( + transcript, + b"logup-sc-b", + eq_table(&tau_b), + h.clone(), + k_vec, + indicator, + ); + // Sumcheck C over the table: e*(g*k' - m) sums to zero. + let run_c = sumcheck_prove( + transcript, + b"logup-sc-c", + eq_table(&tau_c), + g.clone(), + table_k, + m_scalars.clone(), + ); + + lap("sumchecks"); + // Final evaluations and the product witness z = E_h * E_d. + let eq_r_b = eq_table(&run_b.rs); + let eq_r_c = eq_table(&run_c.rs); + let publics_es: Scalar = eq_r_b[..real_len].iter().sum(); + let eh = inner(&eq_r_b, &h); + let ed = inner(&eq_r_b, &digits); + let eg = inner(&eq_r_c, &g); + let em = inner(&eq_r_c, &m_scalars); + let z = eh * ed; + let (b_eh, b_ed, b_eg, b_em, b_z) = ( + random_scalar(), + random_scalar(), + random_scalar(), + random_scalar(), + random_scalar(), + ); + let c_eh = scalar_commit(&eh, &b_eh); + let c_ed = scalar_commit(&ed, &b_ed); + let c_eg = scalar_commit(&eg, &b_eg); + let c_em = scalar_commit(&em, &b_em); + let c_z = scalar_commit(&z, &b_z); + for point in [&c_eh, &c_ed, &c_eg, &c_em, &c_z] { + transcript.append_message(b"logup-evals", &compress(point)); + } + + // Product proof: z = E_h * E_d, proving the same E_d opens C_z over + // base (C_eh, B~): C_z = E_d * C_eh + t * B~ with t = b_z - E_d*b_eh. + let t = b_z - ed * b_eh; + let rho_y = random_scalar(); + let rho_by = random_scalar(); + let rho_t = random_scalar(); + let a2 = scalar_commit(&rho_y, &rho_by); + let a3 = c_eh * rho_y + gens.B_blinding * rho_t; + transcript.append_message(b"logup-prod-a2", &compress(&a2)); + transcript.append_message(b"logup-prod-a3", &compress(&a3)); + let e_chal = transcript_scalar(transcript, b"logup-prod-challenge"); + let product = ProductProof { + a2: compress(&a2), + a3: compress(&a3), + s_y: rho_y + e_chal * ed, + s_by: rho_by + e_chal * b_ed, + s_t: rho_t + e_chal * t, + }; + + // Mega-Schnorr over the full linear system. + let v_value: Scalar = rho + .iter() + .zip(entries.iter()) + .map(|(r, entry)| r * Scalar::from(entry.value)) + .sum(); + let v_blinding: Scalar = rho + .iter() + .zip(entries.iter()) + .map(|(r, entry)| r * entry.blinding) + .sum(); + let witness = LinearWitness { + d: digits, + m: m_scalars, + h, + g, + v: v_value, + cb: run_b.coeff_values.clone(), + cc: run_c.coeff_values.clone(), + eh, + ed, + eg, + em, + z, + }; + let publics = PublicData { + weights: recomposition_weights(&layout, &rho), + alpha, + rs_b: run_b.rs.clone(), + rs_c: run_c.rs.clone(), + eq_r_b, + eq_r_c, + eqv_b: eq_point(&tau_b, &run_b.rs), + eqv_c: eq_point(&tau_c, &run_c.rs), + identv: run_c + .rs + .iter() + .enumerate() + .map(|(j, r)| Scalar::from(1u64 << (TABLE_BITS - 1 - j)) * r) + .sum(), + es: publics_es, + }; + debug_assert_eq!( + constraints_eval(&witness, &publics), + constraint_constants(&publics) + ); + + let rho_w = witness.random_like(real_len); + // Blindings of the witness commitments, in the same component order. + let wb = LinearWitnessBlindings { + b_d, + b_m, + b_h, + b_g, + b_v: v_blinding, + cb: run_b.coeff_blindings.clone(), + cc: run_c.coeff_blindings.clone(), + b_eh, + b_ed, + b_eg, + b_em, + b_z, + }; + let rb = LinearWitnessBlindings { + b_d: random_scalar(), + b_m: random_scalar(), + b_h: random_scalar(), + b_g: random_scalar(), + b_v: random_scalar(), + cb: (0..run_b.coeff_blindings.len()) + .map(|_| std::array::from_fn(|_| random_scalar())) + .collect(), + cc: (0..run_c.coeff_blindings.len()) + .map(|_| std::array::from_fn(|_| random_scalar())) + .collect(), + b_eh: random_scalar(), + b_ed: random_scalar(), + b_eg: random_scalar(), + b_em: random_scalar(), + b_z: random_scalar(), + }; + + lap("evals+product+wb"); + let ((r_d, r_m), (r_h, r_g)) = rayon::join( + || { + rayon::join( + || vector_commit(&rho_w.d, &rb.b_d), + || vector_commit(&rho_w.m, &rb.b_m), + ) + }, + || { + rayon::join( + || vector_commit(&rho_w.h, &rb.b_h), + || vector_commit(&rho_w.g, &rb.b_g), + ) + }, + ); + let r_v = scalar_commit(&rho_w.v, &rb.b_v); + let commit_coeffs = |values: &[[Scalar; 4]], blindings: &[[Scalar; 4]]| { + values + .iter() + .zip(blindings.iter()) + .map(|(v, b)| std::array::from_fn(|k| compress(&scalar_commit(&v[k], &b[k])))) + .collect::>() + }; + let r_coeffs_b = commit_coeffs(&rho_w.cb, &rb.cb); + let r_coeffs_c = commit_coeffs(&rho_w.cc, &rb.cc); + let r_eh = scalar_commit(&rho_w.eh, &rb.b_eh); + let r_ed = scalar_commit(&rho_w.ed, &rb.b_ed); + let r_eg = scalar_commit(&rho_w.eg, &rb.b_eg); + let r_em = scalar_commit(&rho_w.em, &rb.b_em); + let r_z = scalar_commit(&rho_w.z, &rb.b_z); + let t_constraints = constraints_eval(&rho_w, &publics); + + lap("schnorr-announce"); + transcript.append_message(b"logup-schnorr-r-d", &compress(&r_d)); + transcript.append_message(b"logup-schnorr-r-m", &compress(&r_m)); + transcript.append_message(b"logup-schnorr-r-h", &compress(&r_h)); + transcript.append_message(b"logup-schnorr-r-g", &compress(&r_g)); + transcript.append_message(b"logup-schnorr-r-v", &compress(&r_v)); + for round in r_coeffs_b.iter().chain(r_coeffs_c.iter()) { + absorb_points(transcript, b"logup-schnorr-coeffs", round); + } + for point in [&r_eh, &r_ed, &r_eg, &r_em, &r_z] { + transcript.append_message(b"logup-schnorr-evals", &compress(point)); + } + for value in &t_constraints { + transcript.append_message(b"logup-schnorr-t", value.as_bytes()); + } + let c = transcript_scalar(transcript, b"logup-schnorr-challenge"); + + let response = witness.respond(&rho_w, &c); + let coeff_open = |resp: &[[Scalar; 4]], rho_b: &[[Scalar; 4]], w_b: &[[Scalar; 4]]| { + resp.iter() + .zip(rho_b.iter().zip(w_b.iter())) + .map(|(resp, (rho_b, w_b))| { + std::array::from_fn(|k| ScalarOpen { + sigma: resp[k], + sigma_b: rho_b[k] + c * w_b[k], + }) + }) + .collect::>() + }; + + let schnorr = MegaSchnorr { + r_d: compress(&r_d), + r_m: compress(&r_m), + r_h: compress(&r_h), + r_g: compress(&r_g), + r_v: compress(&r_v), + r_coeffs_b, + r_coeffs_c, + r_eh: compress(&r_eh), + r_ed: compress(&r_ed), + r_eg: compress(&r_eg), + r_em: compress(&r_em), + r_z: compress(&r_z), + t_constraints, + sigma_d: response.d.clone(), + sigma_bd: rb.b_d + c * wb.b_d, + sigma_m: response.m.clone(), + sigma_bm: rb.b_m + c * wb.b_m, + sigma_h: response.h.clone(), + sigma_bh: rb.b_h + c * wb.b_h, + sigma_g: response.g.clone(), + sigma_bg: rb.b_g + c * wb.b_g, + sigma_v: ScalarOpen { + sigma: response.v, + sigma_b: rb.b_v + c * wb.b_v, + }, + sigma_coeffs_b: coeff_open(&response.cb, &rb.cb, &wb.cb), + sigma_coeffs_c: coeff_open(&response.cc, &rb.cc, &wb.cc), + sigma_eh: ScalarOpen { + sigma: response.eh, + sigma_b: rb.b_eh + c * wb.b_eh, + }, + sigma_ed: ScalarOpen { + sigma: response.ed, + sigma_b: rb.b_ed + c * wb.b_ed, + }, + sigma_eg: ScalarOpen { + sigma: response.eg, + sigma_b: rb.b_eg + c * wb.b_eg, + }, + sigma_em: ScalarOpen { + sigma: response.em, + sigma_b: rb.b_em + c * wb.b_em, + }, + sigma_z: ScalarOpen { + sigma: response.z, + sigma_b: rb.b_z + c * wb.b_z, + }, + }; + + lap("schnorr-respond"); + Ok(LogUpProof { + c_d: compress(&c_d), + c_m: compress(&c_m), + c_h: compress(&c_h), + c_g: compress(&c_g), + rounds_b: run_b.coeff_commitments, + rounds_c: run_c.coeff_commitments, + c_eh: compress(&c_eh), + c_ed: compress(&c_ed), + c_eg: compress(&c_eg), + c_em: compress(&c_em), + c_z: compress(&c_z), + product, + schnorr, + }) +} + +struct LinearWitnessBlindings { + b_d: Scalar, + b_m: Scalar, + b_h: Scalar, + b_g: Scalar, + b_v: Scalar, + cb: Vec<[Scalar; 4]>, + cc: Vec<[Scalar; 4]>, + b_eh: Scalar, + b_ed: Scalar, + b_eg: Scalar, + b_em: Scalar, + b_z: Scalar, +} + +pub(crate) fn verify( + transcript: &mut Transcript, + entries: &[RangeEntry], + proof: &LogUpProof, +) -> Result<(), NativeError> { + let gens = pc_gens(); + let layout = digit_layout(entries)?; + let n_pad = layout.padded_len; + let m_rounds = n_pad.trailing_zeros() as usize; + let fail = |what: &str| NativeError::Halo2(format!("logup verification failed: {what}")); + + if proof.rounds_b.len() != m_rounds + || proof.rounds_c.len() != TABLE_BITS + || proof.schnorr.sigma_d.len() != n_pad + || proof.schnorr.sigma_h.len() != n_pad + || proof.schnorr.sigma_m.len() != TABLE_SIZE + || proof.schnorr.sigma_g.len() != TABLE_SIZE + || proof.schnorr.sigma_coeffs_b.len() != m_rounds + || proof.schnorr.sigma_coeffs_c.len() != TABLE_BITS + || proof.schnorr.r_coeffs_b.len() != m_rounds + || proof.schnorr.r_coeffs_c.len() != TABLE_BITS + { + return Err(fail("shape")); + } + + transcript.append_message(b"logup", b"v1"); + transcript.append_u64(b"logup-n", n_pad as u64); + for entry in entries { + transcript.append_u64(b"logup-entry-bits", entry.n as u64); + transcript.append_message(b"logup-entry-commitment", &entry.commitment); + } + transcript.append_message(b"logup-c-d", &proof.c_d); + transcript.append_message(b"logup-c-m", &proof.c_m); + let alpha = transcript_scalar(transcript, b"logup-alpha"); + let rho = transcript_scalars(transcript, b"logup-rho", entries.len()); + transcript.append_message(b"logup-c-h", &proof.c_h); + transcript.append_message(b"logup-c-g", &proof.c_g); + let tau_b = transcript_scalars(transcript, b"logup-tau-b", m_rounds); + let tau_c = transcript_scalars(transcript, b"logup-tau-c", TABLE_BITS); + let rs_b = sumcheck_replay(transcript, b"logup-sc-b", &proof.rounds_b); + let rs_c = sumcheck_replay(transcript, b"logup-sc-c", &proof.rounds_c); + for point in [proof.c_eh, proof.c_ed, proof.c_eg, proof.c_em, proof.c_z] { + transcript.append_message(b"logup-evals", &point); + } + + // Product proof. + transcript.append_message(b"logup-prod-a2", &proof.product.a2); + transcript.append_message(b"logup-prod-a3", &proof.product.a3); + let e_chal = transcript_scalar(transcript, b"logup-prod-challenge"); + let c_eh = decompress(&proof.c_eh)?; + let c_ed = decompress(&proof.c_ed)?; + let c_z = decompress(&proof.c_z)?; + let lhs = scalar_commit(&proof.product.s_y, &proof.product.s_by); + if lhs != decompress(&proof.product.a2)? + c_ed * e_chal { + return Err(fail("product proof (opening)")); + } + let lhs = c_eh * proof.product.s_y + gens.B_blinding * proof.product.s_t; + if lhs != decompress(&proof.product.a3)? + c_z * e_chal { + return Err(fail("product proof (relation)")); + } + + // Mega-Schnorr replay. + let eq_r_b = eq_table(&rs_b); + let publics = PublicData { + weights: recomposition_weights(&layout, &rho), + alpha, + eq_r_c: eq_table(&rs_c), + eqv_b: eq_point(&tau_b, &rs_b), + eqv_c: eq_point(&tau_c, &rs_c), + identv: rs_c + .iter() + .enumerate() + .map(|(j, r)| Scalar::from(1u64 << (TABLE_BITS - 1 - j)) * r) + .sum(), + es: eq_r_b[..layout.positions.len()].iter().sum(), + eq_r_b, + rs_b, + rs_c, + }; + let constants = constraint_constants(&publics); + if proof.schnorr.t_constraints.len() != constants.len() { + return Err(fail("constraint count")); + } + + let schnorr = &proof.schnorr; + transcript.append_message(b"logup-schnorr-r-d", &schnorr.r_d); + transcript.append_message(b"logup-schnorr-r-m", &schnorr.r_m); + transcript.append_message(b"logup-schnorr-r-h", &schnorr.r_h); + transcript.append_message(b"logup-schnorr-r-g", &schnorr.r_g); + transcript.append_message(b"logup-schnorr-r-v", &schnorr.r_v); + for round in schnorr.r_coeffs_b.iter().chain(schnorr.r_coeffs_c.iter()) { + absorb_points(transcript, b"logup-schnorr-coeffs", round); + } + for point in [ + &schnorr.r_eh, + &schnorr.r_ed, + &schnorr.r_eg, + &schnorr.r_em, + &schnorr.r_z, + ] { + transcript.append_message(b"logup-schnorr-evals", point); + } + for value in &schnorr.t_constraints { + transcript.append_message(b"logup-schnorr-t", value.as_bytes()); + } + let c = transcript_scalar(transcript, b"logup-schnorr-challenge"); + + // Commitment-equation checks. + let check_vector = + |sigma: &[Scalar], + sigma_b: &Scalar, + announcement: &[u8; 32], + commitment: &[u8; 32]| + -> Result { + Ok(vector_commit(sigma, sigma_b) + == decompress(announcement)? + decompress(commitment)? * c) + }; + // Pad coordinates of d and h are public zeros by construction (zero + // recomposition weight, indicator-gated lookup). Soundness does not + // depend on it, but enforcing it here turns the convention into a + // checked invariant. + let real_len = layout.positions.len(); + if schnorr.sigma_d[real_len..] + .iter() + .chain(schnorr.sigma_h[real_len..].iter()) + .any(|s| *s != Scalar::zero()) + { + return Err(fail("nonzero pad response")); + } + type VectorCheck<'a> = ( + &'a [Scalar], + &'a Scalar, + &'a [u8; 32], + &'a [u8; 32], + &'a str, + ); + let vector_checks: [VectorCheck; 4] = [ + ( + &schnorr.sigma_d, + &schnorr.sigma_bd, + &schnorr.r_d, + &proof.c_d, + "d opening", + ), + ( + &schnorr.sigma_m, + &schnorr.sigma_bm, + &schnorr.r_m, + &proof.c_m, + "m opening", + ), + ( + &schnorr.sigma_h, + &schnorr.sigma_bh, + &schnorr.r_h, + &proof.c_h, + "h opening", + ), + ( + &schnorr.sigma_g, + &schnorr.sigma_bg, + &schnorr.r_g, + &proof.c_g, + "g opening", + ), + ]; + vector_checks.into_par_iter().try_for_each( + |(sigma, sigma_b, announcement, commitment, what)| { + if check_vector(sigma, sigma_b, announcement, commitment)? { + Ok(()) + } else { + Err(fail(what)) + } + }, + )?; + // V opens P_v = sum rho_e C_e. + let p_v = RistrettoPoint::vartime_multiscalar_mul( + rho.iter(), + entries + .iter() + .map(|entry| decompress(&entry.commitment)) + .collect::, _>>()? + .iter(), + ); + if scalar_commit(&schnorr.sigma_v.sigma, &schnorr.sigma_v.sigma_b) + != decompress(&schnorr.r_v)? + p_v * c + { + return Err(fail("recomposition value opening")); + } + let check_scalar = |open: &ScalarOpen, + announcement: &[u8; 32], + commitment: &[u8; 32]| + -> Result { + Ok(scalar_commit(&open.sigma, &open.sigma_b) + == decompress(announcement)? + decompress(commitment)? * c) + }; + for ((opens, announcements), commitments) in schnorr + .sigma_coeffs_b + .iter() + .zip(schnorr.r_coeffs_b.iter()) + .zip(proof.rounds_b.iter()) + .chain( + schnorr + .sigma_coeffs_c + .iter() + .zip(schnorr.r_coeffs_c.iter()) + .zip(proof.rounds_c.iter()), + ) + { + for k in 0..4 { + if !check_scalar(&opens[k], &announcements[k], &commitments[k])? { + return Err(fail("round coefficient opening")); + } + } + } + for (open, (announcement, commitment)) in [ + (&schnorr.sigma_eh, (&schnorr.r_eh, &proof.c_eh)), + (&schnorr.sigma_ed, (&schnorr.r_ed, &proof.c_ed)), + (&schnorr.sigma_eg, (&schnorr.r_eg, &proof.c_eg)), + (&schnorr.sigma_em, (&schnorr.r_em, &proof.c_em)), + (&schnorr.sigma_z, (&schnorr.r_z, &proof.c_z)), + ] { + if !check_scalar(open, announcement, commitment)? { + return Err(fail("evaluation opening")); + } + } + + // Linear constraint checks: L(sigma) == T + c*K. + let response = LinearWitness { + d: schnorr.sigma_d.clone(), + m: schnorr.sigma_m.clone(), + h: schnorr.sigma_h.clone(), + g: schnorr.sigma_g.clone(), + v: schnorr.sigma_v.sigma, + cb: schnorr + .sigma_coeffs_b + .iter() + .map(|round| std::array::from_fn(|k| round[k].sigma)) + .collect(), + cc: schnorr + .sigma_coeffs_c + .iter() + .map(|round| std::array::from_fn(|k| round[k].sigma)) + .collect(), + eh: schnorr.sigma_eh.sigma, + ed: schnorr.sigma_ed.sigma, + eg: schnorr.sigma_eg.sigma, + em: schnorr.sigma_em.sigma, + z: schnorr.sigma_z.sigma, + }; + let evaluated = constraints_eval(&response, &publics); + for ((left, t), k) in evaluated + .iter() + .zip(schnorr.t_constraints.iter()) + .zip(constants.iter()) + { + if *left != t + c * k { + return Err(fail("linear constraint")); + } + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn entry(value: u64, n: usize) -> RangeEntry { + let blinding = random_scalar(); + RangeEntry { + n, + value, + blinding, + commitment: compress(&scalar_commit(&Scalar::from(value), &blinding)), + } + } + + fn entries_mixed() -> Vec { + vec![ + entry(0, 8), + entry(255, 8), + entry(65535, 16), + entry(123456, 32), + entry(u64::MAX, 64), + entry(1, 64), + entry((1 << 20) - 1, 32), + ] + } + + #[test] + fn roundtrip_mixed_widths() { + let entries = entries_mixed(); + let mut prover_transcript = Transcript::new(b"logup-test"); + let proof = prove(&mut prover_transcript, &entries).expect("prove"); + let mut verifier_transcript = Transcript::new(b"logup-test"); + verify(&mut verifier_transcript, &entries, &proof).expect("verify"); + } + + #[test] + fn single_entry_roundtrip() { + let entries = vec![entry(7, 8)]; + let mut prover_transcript = Transcript::new(b"logup-test"); + let proof = prove(&mut prover_transcript, &entries).expect("prove"); + let mut verifier_transcript = Transcript::new(b"logup-test"); + verify(&mut verifier_transcript, &entries, &proof).expect("verify"); + } + + #[test] + fn commitment_to_out_of_range_value_rejected() { + // A valid proof must not verify against a commitment whose opening + // exceeds the claimed width: the recomposition Schnorr ties the + // committed value to its in-range digit decomposition. + let entries = entries_mixed(); + let mut prover_transcript = Transcript::new(b"logup-test"); + let proof = prove(&mut prover_transcript, &entries).expect("prove"); + let mut tampered = entries.clone(); + tampered[0] = RangeEntry { + n: 8, + value: 0, + blinding: Scalar::zero(), + commitment: compress(&scalar_commit(&Scalar::from(300u64), &entries[0].blinding)), + }; + let mut verifier_transcript = Transcript::new(b"logup-test"); + assert!(verify(&mut verifier_transcript, &tampered, &proof).is_err()); + } + + #[test] + fn tampered_round_commitment_rejected() { + let entries = entries_mixed(); + let mut prover_transcript = Transcript::new(b"logup-test"); + let mut proof = prove(&mut prover_transcript, &entries).expect("prove"); + proof.rounds_b[0][1][7] ^= 0x20; + let mut verifier_transcript = Transcript::new(b"logup-test"); + assert!(verify(&mut verifier_transcript, &entries, &proof).is_err()); + } + + #[test] + fn transcript_binding_rejects_context_swap() { + let entries = entries_mixed(); + let mut prover_transcript = Transcript::new(b"logup-test"); + let proof = prove(&mut prover_transcript, &entries).expect("prove"); + let mut other_context = Transcript::new(b"logup-other"); + assert!(verify(&mut other_context, &entries, &proof).is_err()); + } +} diff --git a/src/src/sigma.rs b/src/src/sigma.rs new file mode 100644 index 0000000..92707a7 --- /dev/null +++ b/src/src/sigma.rs @@ -0,0 +1,2675 @@ +//! zklora-sigma-v4: commit-and-prove backend for the exact quantized LoRA +//! delta statement. +//! +//! The v3 halo2 circuit re-witnesses the whole adapter inside every +//! invocation proof: each weight costs a lookup range check plus ~96 rows of +//! in-circuit Poseidon for the adapter commitment, so proving scales with +//! `rank*in + out*rank` per invocation and real LoRA shapes are minutes to +//! infeasible. v4 splits the statement so that all per-weight work happens +//! once per adapter, outside any invocation proof: +//! +//! * Adapter setup (once, at manifest time): Pedersen row commitments to A +//! and B over a fixed ristretto255 basis, per-weight Pedersen value +//! commitments, an aggregated Bulletproofs range proof that every weight +//! lies in the exact `[-value_bound, value_bound]` interval, and a Schnorr +//! linking proof that the row commitments and the range-proved value +//! commitments open to the same integers. The pinned adapter commitment +//! string is the SHA-256 of the deterministic commitment core. +//! +//! * Invocation proof (per statement): the prover commits to the rounding +//! quotients and remainders of the three-stage quantized pipeline, then +//! Fiat-Shamir challenges project each matrix equation onto a single +//! scalar equation over committed values (Schwartz-Zippel over random +//! gamma/beta), proven with generalized Schnorr proofs plus one rank-sized +//! quadratic inner-product sigma protocol. Remainders and quotients are +//! range-bounded with aggregated Bulletproofs. Per-proof work is +//! O(in + rank + out) group operations -- independent of `rank*in`. +//! +//! Statement semantics are identical to the v3 circuit: the same canonical +//! half-up rounding, the same exact remainder intervals, the same value and +//! intermediate bounds. (The only language difference is a deliberate +//! domain extension: for multi-limb quotient caps the accepted interval is +//! `[-I, I+1]` instead of `[-I, I]`; the quotient is uniquely determined by +//! the remainder equation either way, so no false statement is accepted -- +//! see `soundness notes` below.) +//! +//! Assumptions: binding reduces to discrete log on ristretto255 and +//! collision resistance of SHA-256/BLAKE3 -- the same assumption class as +//! the v3 backend (halo2-IPA over Pasta is discrete-log based, and the v3 +//! transcript already used hash-based Fiat-Shamir). Hiding is improved: +//! Pedersen commitments are perfectly hiding and the adapter commitment is +//! salted, whereas the v3 Poseidon chain over raw weights was unsalted. +//! +//! Soundness notes (the chain from accepted proof to exact delta): +//! 1. All commitments are absorbed into the merlin transcript before any +//! challenge is squeezed, so beta/gamma/zeta are sound Fiat-Shamir +//! challenges over the full statement and witness commitments. +//! 2. The projected equations hold mod l (group order). Because beta and +//! gamma are uniform in F_l and every per-element coefficient was fixed +//! before the challenge, Schwartz-Zippel gives the per-element equations +//! mod l except with probability ~(rank+out)/l. +//! 3. `validate_field_safety_v4` bounds every integer magnitude that can +//! appear in a per-element equation by 2^250 < l/4, and the range proofs +//! pin each committed quotient/remainder into its exact interval, so the +//! mod-l equations are equations over Z. +//! 4. Over Z, `raw = s*q + rem` with `rem` in the canonical interval +//! `[-floor(s/2), ceil(s/2)-1]` has a unique solution (q, rem), which is +//! exactly the canonical half-up rounding used by the reference pipeline, +//! so delta is the unique exact function of (x, committed adapter). + +use blake3; +use bulletproofs::{BulletproofGens, PedersenGens, RangeProof}; +use curve25519_dalek_ng::constants::RISTRETTO_BASEPOINT_TABLE; +use curve25519_dalek_ng::ristretto::{ + CompressedRistretto, RistrettoBasepointTable, RistrettoPoint, +}; +use curve25519_dalek_ng::scalar::Scalar; +use curve25519_dalek_ng::traits::{Identity, MultiscalarMul, VartimeMultiscalarMul}; +use merlin::Transcript; +use num_bigint::{BigInt, BigUint, Sign}; +use num_traits::Signed; +use rand_core::OsRng; +use rayon::prelude::*; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use std::collections::HashSet; +use std::sync::{Mutex, OnceLock}; + +use crate::{ + canonical_remainder_interval_int, div_round_canonical_int, AdapterCommitmentInput, + BoundedCache, FixedPointConfig, NativeError, NativeStatement, NativeWitness, +}; + +pub const SIGMA_SCHEME_ID: &str = "zklora-sigma-v4-pedersen-ristretto255"; +pub const SIGMA_SCHEMA_VERSION: u64 = 3; +/// Integer magnitudes in any projected equation must stay below 2^250 so +/// that sums of two terms cannot reach l/2 (l ~ 2^252.5 on ristretto255). +const FIELD_SAFE_BITS_V4: usize = 250; +/// Max range-proof entries aggregated into one Bulletproof. Chunks are +/// proven/verified in parallel; smaller chunks parallelize better while +/// costing ~700 bytes each. +const BP_CHUNK: usize = 128; +const TRANSCRIPT_LABEL: &[u8] = b"zklora-sigma-v4"; + +// --------------------------------------------------------------------------- +// Generators +// --------------------------------------------------------------------------- + +pub(crate) fn pc_gens() -> &'static PedersenGens { + static GENS: OnceLock = OnceLock::new(); + GENS.get_or_init(PedersenGens::default) +} + +/// Precomputed table for the blinding generator: fixed-base multiplication +/// is ~5x faster than the generic path and every commitment pays one. +pub(crate) fn blinding_table() -> &'static RistrettoBasepointTable { + static TABLE: OnceLock = OnceLock::new(); + TABLE.get_or_init(|| RistrettoBasepointTable::create(&pc_gens().B_blinding)) +} + +fn bp_gens() -> &'static BulletproofGens { + static GENS: OnceLock = OnceLock::new(); + GENS.get_or_init(|| BulletproofGens::new(64, BP_CHUNK)) +} + +/// Vector-commitment basis, independent of the Pedersen pair (B, B~) by +/// construction: each point is hash-to-group output under a dedicated +/// domain, so no discrete-log relation between any of them is known. +/// +/// The lock is never held across the parallel generation: a rayon worker +/// that holds a lock while waiting on stolen subtasks can steal another job +/// that blocks on the same lock, deadlocking the pool. Racing growers may +/// duplicate generation work; the points are deterministic, so the longest +/// result simply wins. +pub(crate) fn g_basis(len: usize) -> std::sync::Arc> { + use std::sync::{Arc, RwLock}; + static CACHE: OnceLock>>> = OnceLock::new(); + let cache = CACHE.get_or_init(|| RwLock::new(Arc::new(Vec::new()))); + let current = { + let guard = cache.read().expect("basis cache poisoned"); + guard.clone() + }; + if current.len() >= len { + return current; + } + let extra: Vec = (current.len()..len) + .into_par_iter() + .map(|index| { + let mut hasher = blake3::Hasher::new(); + hasher.update(b"zklora-sigma-v4 row basis"); + hasher.update(&(index as u64).to_le_bytes()); + let mut bytes = [0u8; 64]; + hasher.finalize_xof().fill(&mut bytes); + RistrettoPoint::from_uniform_bytes(&bytes) + }) + .collect(); + let mut grown = current.as_ref().clone(); + grown.extend(extra); + let grown = Arc::new(grown); + let mut guard = cache.write().expect("basis cache poisoned"); + if guard.len() < grown.len() { + *guard = grown.clone(); + } + guard.clone() +} + +// --------------------------------------------------------------------------- +// Scalar / point helpers +// --------------------------------------------------------------------------- + +fn scalar_from_bigint(value: &BigInt) -> Result { + let magnitude = value.abs().to_biguint().expect("abs is non-negative"); + if magnitude.bits() as usize > FIELD_SAFE_BITS_V4 { + return Err(NativeError::InvalidDimensions( + "integer exceeds sigma-v4 field-safe bound".into(), + )); + } + let mut bytes = [0u8; 32]; + let raw = magnitude.to_bytes_le(); + bytes[..raw.len()].copy_from_slice(&raw); + let scalar = Scalar::from_bytes_mod_order(bytes); + Ok(if value.sign() == Sign::Minus { + -scalar + } else { + scalar + }) +} + +fn scalar_from_i64(value: i64) -> Scalar { + if value < 0 { + -Scalar::from(value.unsigned_abs()) + } else { + Scalar::from(value as u64) + } +} + +/// Variable-time MSM, split across rayon workers for large inputs. Splitting +/// an MSM into chunks and summing partial results is exact; per-chunk +/// Pippenger loses a little batching efficiency but wall time wins ~cores. +pub(crate) fn par_msm(scalars: &[Scalar], points: &[RistrettoPoint]) -> RistrettoPoint { + debug_assert_eq!(scalars.len(), points.len()); + if scalars.len() < 1024 { + return RistrettoPoint::vartime_multiscalar_mul(scalars.iter(), points.iter()); + } + let chunk = scalars.len().div_ceil(rayon::current_num_threads().max(1)); + scalars + .par_chunks(chunk) + .zip(points.par_chunks(chunk)) + .map(|(s, p)| RistrettoPoint::vartime_multiscalar_mul(s.iter(), p.iter())) + .reduce(RistrettoPoint::identity, |a, b| a + b) +} + +pub(crate) fn compress(point: &RistrettoPoint) -> [u8; 32] { + point.compress().to_bytes() +} + +pub(crate) fn decompress(bytes: &[u8; 32]) -> Result { + CompressedRistretto(*bytes) + .decompress() + .ok_or_else(|| NativeError::InvalidDimensions("invalid ristretto point".into())) +} + +pub(crate) fn random_scalar() -> Scalar { + use rand_core::SeedableRng; + thread_local! { + static RNG: std::cell::RefCell = + std::cell::RefCell::new(rand_chacha::ChaCha12Rng::from_rng(OsRng).expect("seed rng")); + } + RNG.with(|rng| Scalar::random(&mut *rng.borrow_mut())) +} + +/// Deterministic blinding factors for adapter commitments: keyed BLAKE3 +/// under a key that binds BOTH the contributor's secret salt AND the full +/// adapter content (see `adapter_blinding_key`). Binding the content is +/// essential: if blindings depended only on (salt, domain, index), two +/// same-shaped adapters published by the same contributor would share +/// blindings at every index, and commitment differences C1_i - C2_i = +/// (w1_i - w2_i)*G would leak exact weight differences from the public +/// manifest. The salt never leaves the prover; if it leaks, hiding degrades +/// to the (still binding) unsalted level of the v3 scheme. +fn derived_blinding(key: &[u8; 32], domain: &str, index: u64) -> Scalar { + let mut hasher = blake3::Hasher::new_keyed(key); + hasher.update(domain.as_bytes()); + hasher.update(&index.to_le_bytes()); + let mut bytes = [0u8; 64]; + hasher.finalize_xof().fill(&mut bytes); + Scalar::from_bytes_mod_order_wide(&bytes) +} + +/// Per-adapter blinding key: keyed BLAKE3 of the canonical adapter payload +/// (dims, config, scaling, and every weight) under the contributor salt. +/// Distinct adapters therefore get independent blindings even when they +/// share a shape and a salt, while the derivation stays deterministic so +/// manifest commitments and proof-time commitments always agree. +fn adapter_blinding_key( + input: &AdapterCommitmentInput, + salt: &[u8; 32], +) -> Result<[u8; 32], NativeError> { + let mut hasher = blake3::Hasher::new_keyed(salt); + hasher.update(b"zklora-sigma-v4 adapter blinding key"); + hasher.update( + serde_json::to_string(input) + .map_err(|e| NativeError::Json(e.to_string()))? + .as_bytes(), + ); + Ok(*hasher.finalize().as_bytes()) +} + +pub(crate) fn transcript_scalar(transcript: &mut Transcript, label: &'static [u8]) -> Scalar { + let mut bytes = [0u8; 64]; + transcript.challenge_bytes(label, &mut bytes); + Scalar::from_bytes_mod_order_wide(&bytes) +} + +pub(crate) fn transcript_scalars( + transcript: &mut Transcript, + label: &'static [u8], + count: usize, +) -> Vec { + (0..count) + .map(|_| { + let mut bytes = [0u8; 64]; + transcript.challenge_bytes(label, &mut bytes); + Scalar::from_bytes_mod_order_wide(&bytes) + }) + .collect() +} + +pub(crate) fn absorb_points( + transcript: &mut Transcript, + label: &'static [u8], + points: &[[u8; 32]], +) { + transcript.append_u64(b"count", points.len() as u64); + for point in points { + transcript.append_message(label, point); + } +} + +fn parse_salt(salt_hex: &str) -> Result<[u8; 32], NativeError> { + let cleaned = salt_hex.trim().trim_start_matches("0x"); + if cleaned.len() != 64 || !cleaned.chars().all(|c| c.is_ascii_hexdigit()) { + return Err(NativeError::InvalidDimensions( + "adapter salt must be 32 bytes of hex".into(), + )); + } + let mut salt = [0u8; 32]; + for (i, chunk) in cleaned.as_bytes().chunks(2).enumerate() { + salt[i] = u8::from_str_radix(std::str::from_utf8(chunk).expect("hex"), 16) + .map_err(|_| NativeError::InvalidDimensions("invalid salt hex".into()))?; + } + Ok(salt) +} + +// --------------------------------------------------------------------------- +// Range planning: exact signed intervals via 64-bit limbs + derived twins +// --------------------------------------------------------------------------- + +/// One Bulletproof entry: a committed u64 proven to lie in [0, 2^n). +#[derive(Clone)] +pub(crate) struct RangeEntry { + pub(crate) n: usize, + pub(crate) value: u64, // prover side only (0 on verify) + pub(crate) blinding: Scalar, // prover side only + pub(crate) commitment: [u8; 32], +} + +fn min_bp_bits(max: u64) -> usize { + for n in [8usize, 16, 32, 64] { + if n == 64 || max < (1u64 << n) { + return n; + } + } + 64 +} + +/// Plan for one committed signed integer `v` in the interval +/// `[lower, upper]`: the prover publishes commitments to the 64-bit limbs of +/// `shifted = v - lower`, full limbs are proven in [0, 2^64) (exact: that IS +/// their full width), and the top limb t (max value T = max >> 64*(L-1)) is +/// proven two-sided: t in [0, 2^n) and T - t in [0, 2^n), which pins +/// t in [0, T] exactly. Single-limb values (the only case for remainders, +/// which require exactness) are pinned with zero slack. For multi-limb +/// values the enforced bound is (T+1)*2^(64*(L-1)) - 1, i.e. a slack of +/// 2^64 - 1 - (max mod 2^64) above `max` in general; `limb_plan` REJECTS +/// any multi-limb width whose slack exceeds 1, so the only admitted +/// multi-limb intervals are the quotient caps `max = 2^k - 2` (slack +/// exactly +1, harmless: the quotient is uniquely determined by the +/// exactly-pinned remainder equation, and field-safety margins already +/// cover max + 1). +struct LimbPlan { + limb_maxes: Vec, // per limb: max admissible value +} + +fn limb_plan(width: &BigInt) -> Result { + // width = upper - lower >= 0; max shifted value. + if width.sign() == Sign::Minus { + return Err(NativeError::InvalidDimensions( + "range interval is empty".into(), + )); + } + let max = width.to_biguint().expect("non-negative"); + let bits = max.bits().max(1) as usize; + if bits > FIELD_SAFE_BITS_V4 { + return Err(NativeError::InvalidDimensions( + "range interval exceeds field-safe bound".into(), + )); + } + let limbs = bits.div_ceil(64); + if limbs > 1 { + // Enforce the documented at-most-+1 slack: the low limbs cover + // [0, 2^(64*(L-1)) - 1] exactly, so the slack above `max` is + // 2^(64*(L-1)) - 1 - (max mod 2^(64*(L-1))). + let low_mask = (BigUint::from(1u8) << (64 * (limbs - 1))) - 1u8; + let slack = &low_mask - (&max & &low_mask); + if slack > BigUint::from(1u8) { + return Err(NativeError::InvalidDimensions( + "multi-limb range interval would exceed the +1 slack bound".into(), + )); + } + } + let mut limb_maxes = vec![u64::MAX; limbs]; + let top = &max >> (64 * (limbs - 1)); + let top: u64 = top.try_into().expect("top limb fits u64"); + limb_maxes[limbs - 1] = top; + Ok(LimbPlan { limb_maxes }) +} + +fn limbs_of(shifted: &BigUint, count: usize) -> Vec { + let digits = shifted.to_u64_digits(); + (0..count) + .map(|i| digits.get(i).copied().unwrap_or(0)) + .collect() +} + +/// A committed bounded value class shared between prover and verifier: +/// `values.len()` integers, each in `[lower, lower+width]`. +struct BoundedClass { + lower: BigInt, + plan: LimbPlan, + /// limb commitments, value-major: commitments[value][limb] + commitments: Vec>, + /// prover-side openings (shifted limb values and blindings) + openings: Vec>, + /// prover-side: blinding of the derived whole-value commitment + value_blindings: Vec, +} + +/// (limb commitments, limb openings, derived value blinding) for one value. +type CommittedValue = (Vec<[u8; 32]>, Vec<(u64, Scalar)>, Scalar); + +impl BoundedClass { + /// Prover-side construction: commit every limb of every shifted value. + fn commit(values: &[BigInt], lower: &BigInt, width: &BigInt) -> Result { + let plan = limb_plan(width)?; + let _ = pc_gens(); + let results: Vec = values + .par_iter() + .map(|value| { + let shifted = (value - lower) + .to_biguint() + .ok_or_else(|| NativeError::InvalidDimensions("value below range".into()))?; + let limbs = limbs_of(&shifted, plan.limb_maxes.len()); + for (limb, max) in limbs.iter().zip(plan.limb_maxes.iter()) { + if limb > max { + return Err(NativeError::InvalidDimensions("value above range".into())); + } + } + let mut commitments = Vec::with_capacity(limbs.len()); + let mut openings = Vec::with_capacity(limbs.len()); + let mut value_blinding = Scalar::zero(); + let mut base = Scalar::one(); + let shift_64 = Scalar::from(u64::MAX) + Scalar::one(); + for limb in &limbs { + let blinding = random_scalar(); + let commitment = &Scalar::from(*limb) * &RISTRETTO_BASEPOINT_TABLE + + blinding_table() * &blinding; + commitments.push(compress(&commitment)); + openings.push((*limb, blinding)); + value_blinding += base * blinding; + base *= shift_64; + } + Ok((commitments, openings, value_blinding)) + }) + .collect::>()?; + let mut commitments = Vec::with_capacity(values.len()); + let mut openings = Vec::with_capacity(values.len()); + let mut value_blindings = Vec::with_capacity(values.len()); + for (c, o, b) in results { + commitments.push(c); + openings.push(o); + value_blindings.push(b); + } + Ok(Self { + lower: lower.clone(), + plan, + commitments, + openings, + value_blindings, + }) + } + + /// Verifier-side construction from published limb commitments. + fn from_commitments( + commitments: Vec>, + count: usize, + lower: &BigInt, + width: &BigInt, + ) -> Result { + let plan = limb_plan(width)?; + if commitments.len() != count + || commitments + .iter() + .any(|limbs| limbs.len() != plan.limb_maxes.len()) + { + return Err(NativeError::InvalidDimensions( + "range commitment shape mismatch".into(), + )); + } + Ok(Self { + lower: lower.clone(), + plan, + commitments, + openings: Vec::new(), + value_blindings: Vec::new(), + }) + } + + /// Derived commitment to the signed value: sum_i 2^(64 i) C_i + lower*B. + fn value_commitment(&self, index: usize) -> Result { + let mut scalars = Vec::with_capacity(self.plan.limb_maxes.len() + 1); + let mut points = Vec::with_capacity(self.plan.limb_maxes.len() + 1); + let mut base = Scalar::one(); + let shift_64 = Scalar::from(u64::MAX) + Scalar::one(); + for limb in &self.commitments[index] { + scalars.push(base); + points.push(decompress(limb)?); + base *= shift_64; + } + scalars.push(scalar_from_bigint(&self.lower)?); + points.push(pc_gens().B); + Ok(RistrettoPoint::vartime_multiscalar_mul( + scalars.iter(), + points.iter(), + )) + } + + fn absorb(&self, transcript: &mut Transcript, label: &'static [u8]) { + transcript.append_u64(b"class-count", self.commitments.len() as u64); + for limbs in &self.commitments { + absorb_points(transcript, label, limbs); + } + } + + /// Emit range entries: full limbs one-sided (exact), partial top limbs + /// two-sided with a derived twin commitment max*B - C. + fn push_entries(&self, entries: &mut Vec) -> Result<(), NativeError> { + let prover_side = !self.openings.is_empty(); + let per_value: Vec> = (0..self.commitments.len()) + .into_par_iter() + .map(|index| { + let mut local = Vec::with_capacity(self.plan.limb_maxes.len() * 2); + for (limb_index, max) in self.plan.limb_maxes.iter().enumerate() { + let n = min_bp_bits(*max); + let commitment = self.commitments[index][limb_index]; + let (value, blinding) = if prover_side { + self.openings[index][limb_index] + } else { + (0, Scalar::zero()) + }; + local.push(RangeEntry { + n, + value, + blinding, + commitment, + }); + if *max != u64::MAX { + // Twin: max - v with blinding -b; commitment derived + // so the verifier needs no extra published data. + let twin = &Scalar::from(*max) * &RISTRETTO_BASEPOINT_TABLE + - decompress(&commitment)?; + local.push(RangeEntry { + n, + value: max.wrapping_sub(value), + blinding: -blinding, + commitment: compress(&twin), + }); + } + } + Ok(local) + }) + .collect::>()?; + for mut local in per_value { + entries.append(&mut local); + } + Ok(()) + } +} + +#[derive(Clone, Serialize, Deserialize)] +enum RangeBundle { + /// Aggregated Bulletproofs grouped by bit width: (n, chunk proofs) in + /// the deterministic plan order. Compact (log-size) but ~ms per entry; + /// always used for the one-time adapter setup, where artifact size + /// matters more than the amortized-to-zero proving time. + Bulletproofs { groups: Vec<(usize, Vec>)> }, + /// Sumcheck-based LogUp lookup argument: microseconds of field work per + /// entry plus a few MSMs; the default for per-invocation proofs. Both + /// engines prove the identical statement (every committed value in its + /// exact interval) under the same discrete-log + Fiat-Shamir + /// assumptions, and the verifier accepts either. + LogUp(Box), +} + +/// Range engine for invocation proofs: LogUp by default for proving speed; +/// ZKLORA_RANGE_ENGINE=bulletproofs opts into compact proofs instead. +fn invocation_range_engine() -> &'static str { + static ENGINE: OnceLock = OnceLock::new(); + ENGINE.get_or_init( + || match std::env::var("ZKLORA_RANGE_ENGINE").ok().as_deref() { + Some("bulletproofs") => "bulletproofs".to_string(), + _ => "logup".to_string(), + }, + ) +} + +fn prove_ranges_with_engine( + transcript: &Transcript, + entries: Vec, + engine: &str, +) -> Result { + if engine == "logup" { + let mut fork = transcript.clone(); + return Ok(RangeBundle::LogUp(Box::new(crate::logup::prove( + &mut fork, &entries, + )?))); + } + prove_ranges(transcript, entries) +} + +fn verify_range_bundle( + transcript: &Transcript, + entries: Vec, + bundle: &RangeBundle, +) -> Result<(), NativeError> { + match bundle { + RangeBundle::LogUp(proof) => { + let mut fork = transcript.clone(); + crate::logup::verify(&mut fork, &entries, proof) + } + RangeBundle::Bulletproofs { .. } => verify_ranges(transcript, entries, bundle), + } +} + +/// Deterministic chunk plan: Bulletproofs aggregation requires a +/// power-of-two party count, so each width group is partitioned into +/// power-of-two chunks of at most BP_CHUNK entries (the binary decomposition +/// of the remainder). No padding parties are ever needed, and chunks +/// parallelize across cores. +fn chunk_plan(len: usize) -> Vec<(usize, usize)> { + let mut plan = Vec::new(); + let mut start = 0; + let mut remaining = len; + while remaining >= BP_CHUNK { + plan.push((start, BP_CHUNK)); + start += BP_CHUNK; + remaining -= BP_CHUNK; + } + while remaining > 0 { + let size = 1usize << (usize::BITS - 1 - remaining.leading_zeros()) as usize; + plan.push((start, size)); + start += size; + remaining -= size; + } + plan +} + +fn group_entries(entries: &[RangeEntry]) -> Vec<(usize, Vec)> { + let mut groups = Vec::new(); + for n in [8usize, 16, 32, 64] { + let group: Vec = entries.iter().filter(|e| e.n == n).cloned().collect(); + if !group.is_empty() { + groups.push((n, group)); + } + } + groups +} + +/// Aggregate-prove all entries, grouped by bit width and chunked for +/// parallelism. Each chunk transcript is forked from the caller transcript +/// (which has already absorbed every commitment and sigma response) and +/// domain-separated by group and chunk index, so Fiat-Shamir binding covers +/// the full statement. +fn prove_ranges( + transcript: &Transcript, + entries: Vec, +) -> Result { + let proved = group_entries(&entries) + .into_par_iter() + .map(|(n, group)| { + let chunks: Vec> = chunk_plan(group.len()) + .into_par_iter() + .enumerate() + .map(|(chunk_index, (start, size))| { + let chunk = &group[start..start + size]; + let values: Vec = chunk.iter().map(|e| e.value).collect(); + let blindings: Vec = chunk.iter().map(|e| e.blinding).collect(); + let mut chunk_transcript = transcript.clone(); + chunk_transcript.append_u64(b"bp-group-bits", n as u64); + chunk_transcript.append_u64(b"bp-chunk-index", chunk_index as u64); + let (proof, _commitments) = RangeProof::prove_multiple_with_rng( + bp_gens(), + pc_gens(), + &mut chunk_transcript, + &values, + &blindings, + n, + &mut OsRng, + ) + .map_err(|e| NativeError::Halo2(format!("range proof: {e:?}")))?; + Ok(proof.to_bytes()) + }) + .collect::>()?; + Ok((n, chunks)) + }) + .collect::, NativeError>>()?; + Ok(RangeBundle::Bulletproofs { groups: proved }) +} + +fn verify_ranges( + transcript: &Transcript, + entries: Vec, + bundle: &RangeBundle, +) -> Result<(), NativeError> { + let RangeBundle::Bulletproofs { groups } = bundle else { + return Err(NativeError::InvalidDimensions( + "expected bulletproofs range bundle".into(), + )); + }; + let expected = group_entries(&entries); + if expected.len() != groups.len() + || expected + .iter() + .zip(groups.iter()) + .any(|((n_expected, group), (n_actual, chunks))| { + n_expected != n_actual || chunk_plan(group.len()).len() != chunks.len() + }) + { + return Err(NativeError::InvalidDimensions( + "range bundle shape mismatch".into(), + )); + } + expected + .into_par_iter() + .zip(groups.par_iter()) + .try_for_each(|((n, group), (_, chunks))| { + chunk_plan(group.len()) + .into_par_iter() + .zip(chunks.par_iter()) + .enumerate() + .try_for_each(|(chunk_index, ((start, size), proof_bytes))| { + let commitments: Vec = group[start..start + size] + .iter() + .map(|e| CompressedRistretto(e.commitment)) + .collect(); + let proof = RangeProof::from_bytes(proof_bytes) + .map_err(|e| NativeError::Halo2(format!("range proof decode: {e:?}")))?; + let mut chunk_transcript = transcript.clone(); + chunk_transcript.append_u64(b"bp-group-bits", n as u64); + chunk_transcript.append_u64(b"bp-chunk-index", chunk_index as u64); + proof + .verify_multiple_with_rng( + bp_gens(), + pc_gens(), + &mut chunk_transcript, + &commitments, + n, + &mut OsRng, + ) + .map_err(|e| NativeError::Halo2(format!("range verify: {e:?}"))) + }) + }) +} + +// --------------------------------------------------------------------------- +// Adapter setup +// --------------------------------------------------------------------------- + +/// Deterministic commitment core: everything the pinned adapter commitment +/// string covers. Field order is fixed by this struct, and only Rust ever +/// serializes it, so `serde_json::to_string` is canonical. +#[derive(Clone, Serialize, Deserialize)] +pub struct AdapterCore { + pub scheme: String, + pub schema_version: u64, + pub in_dim: usize, + pub rank: usize, + pub out_dim: usize, + pub fixed_point: FixedPointConfig, + pub scaling_num: i64, + pub scaling_den: i64, + pub row_commitments_a: Vec<[u8; 32]>, + pub row_commitments_b: Vec<[u8; 32]>, + /// Per-weight commitments to (w + value_bound), A row-major then B + /// row-major. Only used by the one-time adapter range/link proofs. + pub weight_commitments: Vec<[u8; 32]>, +} + +#[derive(Serialize, Deserialize)] +struct LinkProof { + r1: [u8; 32], + r2: [u8; 32], + sigma_z: Vec, + sigma_s: Scalar, + sigma_t: Scalar, +} + +#[derive(Serialize, Deserialize)] +pub struct AdapterSetupPub { + pub core: AdapterCore, + link_a: LinkProof, + link_b: LinkProof, + ranges: RangeBundle, +} + +struct AdapterSecrets { + core: AdapterCore, + /// Precomputed adapter commitment string (SHA-256 over the serialized + /// core); serializing multi-MB cores on every prove call is measurable. + commitment: String, + row_blindings_a: Vec, + row_blindings_b: Vec, + weight_blindings: Vec, +} + +fn value_bound_int(config: &FixedPointConfig) -> BigInt { + (BigInt::from(1) << (config.value_bits - 1)) - 1 +} + +fn intermediate_bound_int(config: &FixedPointConfig) -> BigInt { + (BigInt::from(1) << (config.intermediate_bits - 1)) - 1 +} + +fn validate_adapter_input(input: &AdapterCommitmentInput) -> Result<(), NativeError> { + let rank = input.a.len(); + let in_dim = input.a.first().map_or(0, |row| row.len()); + let out_dim = input.b.len(); + if rank == 0 || in_dim == 0 || out_dim == 0 { + return Err(NativeError::InvalidDimensions( + "adapter dimensions must be positive".into(), + )); + } + if input.in_dim != in_dim || input.rank != rank || input.out_dim != out_dim { + return Err(NativeError::InvalidDimensions( + "adapter payload dimensions do not match matrices".into(), + )); + } + if input.a.iter().any(|row| row.len() != in_dim) || input.b.iter().any(|row| row.len() != rank) + { + return Err(NativeError::InvalidDimensions( + "adapter matrices are ragged".into(), + )); + } + if input.scaling_den <= 0 { + return Err(NativeError::InvalidDimensions( + "scaling denominator must be positive".into(), + )); + } + let config = &input.fixed_point; + if config.value_bits == 0 + || config.value_bits > 63 + || config.scale_bits >= config.value_bits + || config.intermediate_bits == 0 + { + return Err(NativeError::InvalidDimensions( + "invalid fixed-point bit widths".into(), + )); + } + let bound = value_bound_int(config); + for value in input.a.iter().flatten().chain(input.b.iter().flatten()) { + let value = BigInt::from(*value); + if value < -&bound || value > bound { + return Err(NativeError::InvalidDimensions( + "adapter weight exceeds value bound".into(), + )); + } + } + validate_field_safety_v4( + in_dim, + rank, + out_dim, + config, + input.scaling_num, + input.scaling_den, + ) +} + +/// Bound every integer magnitude appearing in a projected per-element +/// equation: || <= in*V^2, |s*u + rem| <= s*(2I+2), || <= +/// rank*V*(2I+2), |num*w| <= |num|*(2I+2), |den*delta + rem| <= den*(V+1). +/// Hard dimension caps: orders of magnitude beyond any model layer in use +/// (largest practical LoRA target is an embedding of a few hundred thousand +/// rows), but small enough that proving/verification allocations stay +/// bounded even for hostile inputs. +const MAX_DIM: usize = 1 << 22; +const MAX_RANK: usize = 1 << 16; +const MAX_WEIGHTS: usize = 1 << 26; + +fn validate_field_safety_v4( + in_dim: usize, + rank: usize, + out_dim: usize, + config: &FixedPointConfig, + scaling_num: i64, + scaling_den: i64, +) -> Result<(), NativeError> { + if in_dim > MAX_DIM + || out_dim > MAX_DIM + || rank > MAX_RANK + || rank.saturating_mul(in_dim) + out_dim.saturating_mul(rank) > MAX_WEIGHTS + { + return Err(NativeError::InvalidDimensions( + "adapter dimensions exceed sigma-v4 limits".into(), + )); + } + let value_bits = config.value_bits as usize; + let intermediate_bits = config.intermediate_bits as usize; + let log_in = usize::BITS as usize - in_dim.leading_zeros() as usize; + let log_rank = usize::BITS as usize - rank.leading_zeros() as usize; + let num_bits = 64 - scaling_num.unsigned_abs().leading_zeros() as usize; + let den_bits = 64 - (scaling_den as u64).leading_zeros() as usize; + let candidates = [ + 2 * value_bits + log_in, + config.scale_bits as usize + intermediate_bits + 2, + value_bits + intermediate_bits + log_rank + 2, + intermediate_bits + num_bits + 2, + den_bits + value_bits + 2, + ]; + if candidates.iter().any(|bits| *bits > FIELD_SAFE_BITS_V4) { + return Err(NativeError::InvalidDimensions( + "fixed-point config and dimensions exceed sigma-v4 field-safe bounds".into(), + )); + } + Ok(()) +} + +fn adapter_secrets( + input: &AdapterCommitmentInput, + salt: &[u8; 32], +) -> Result { + validate_adapter_input(input)?; + let blinding_key = adapter_blinding_key(input, salt)?; + let rank = input.a.len(); + let in_dim = input.a[0].len(); + let out_dim = input.b.len(); + let gens = pc_gens(); + let basis = g_basis(in_dim.max(rank)); + let value_bound = value_bound_int(&input.fixed_point); + let shift = scalar_from_bigint(&value_bound)?; + + let row_blindings_a: Vec = (0..rank) + .map(|k| derived_blinding(&blinding_key, "row-a", k as u64)) + .collect(); + let row_blindings_b: Vec = (0..out_dim) + .map(|j| derived_blinding(&blinding_key, "row-b", j as u64)) + .collect(); + let row_commitments_a: Vec<[u8; 32]> = input + .a + .par_iter() + .zip(row_blindings_a.par_iter()) + .map(|(row, blinding)| { + let scalars: Vec = row + .iter() + .map(|w| scalar_from_i64(*w)) + .chain(std::iter::once(*blinding)) + .collect(); + let points: Vec = basis[..in_dim] + .iter() + .copied() + .chain(std::iter::once(gens.B_blinding)) + .collect(); + compress(&RistrettoPoint::multiscalar_mul( + scalars.iter(), + points.iter(), + )) + }) + .collect(); + let row_commitments_b: Vec<[u8; 32]> = input + .b + .par_iter() + .zip(row_blindings_b.par_iter()) + .map(|(row, blinding)| { + let scalars: Vec = row + .iter() + .map(|w| scalar_from_i64(*w)) + .chain(std::iter::once(*blinding)) + .collect(); + let points: Vec = basis[..rank] + .iter() + .copied() + .chain(std::iter::once(gens.B_blinding)) + .collect(); + compress(&RistrettoPoint::multiscalar_mul( + scalars.iter(), + points.iter(), + )) + }) + .collect(); + + let weights: Vec = input + .a + .iter() + .flatten() + .chain(input.b.iter().flatten()) + .copied() + .collect(); + let weight_blindings: Vec = (0..weights.len()) + .map(|i| derived_blinding(&blinding_key, "weight", i as u64)) + .collect(); + let weight_commitments: Vec<[u8; 32]> = weights + .par_iter() + .zip(weight_blindings.par_iter()) + .map(|(w, blinding)| { + let shifted = scalar_from_i64(*w) + shift; + compress(&(&shifted * &RISTRETTO_BASEPOINT_TABLE + blinding_table() * blinding)) + }) + .collect(); + + let core = AdapterCore { + scheme: SIGMA_SCHEME_ID.to_string(), + schema_version: SIGMA_SCHEMA_VERSION, + in_dim, + rank, + out_dim, + fixed_point: input.fixed_point.clone(), + scaling_num: input.scaling_num, + scaling_den: input.scaling_den, + row_commitments_a, + row_commitments_b, + weight_commitments, + }; + Ok(AdapterSecrets { + commitment: adapter_commitment_string(&core), + core, + row_blindings_a, + row_blindings_b, + weight_blindings, + }) +} + +pub fn adapter_commitment_string(core: &AdapterCore) -> String { + let json = serde_json::to_string(core).expect("core serializes"); + let mut hasher = Sha256::new(); + hasher.update(json.as_bytes()); + format!("{:x}", hasher.finalize()) +} + +fn adapter_transcript(core: &AdapterCore) -> Transcript { + let mut transcript = Transcript::new(TRANSCRIPT_LABEL); + transcript.append_message(b"phase", b"adapter-setup"); + transcript.append_message( + b"core", + serde_json::to_string(core) + .expect("core serializes") + .as_bytes(), + ); + transcript +} + +/// Schnorr proof that row commitments (basis G, blinding B~) and shifted +/// per-value commitments (base B, blinding B~) open to the same weights, +/// folded over Fiat-Shamir challenges lambda (rows) and mu (columns). +fn prove_link( + transcript: &mut Transcript, + label: &'static [u8], + rows: &[Vec], + row_blindings: &[Scalar], + value_blindings: &[Scalar], + cols: usize, +) -> LinkProof { + let gens = pc_gens(); + let basis = g_basis(cols); + let row_count = rows.len(); + let lambda = transcript_scalars(transcript, label, row_count); + let mu = transcript_scalars(transcript, label, cols); + + // z_j = sum_k lambda_k rows[k][j]; folded blindings for both forms. + let mut z = vec![Scalar::zero(); cols]; + let mut s_fold = Scalar::zero(); + let mut t_fold = Scalar::zero(); + for (k, row) in rows.iter().enumerate() { + for (j, w) in row.iter().enumerate() { + z[j] += lambda[k] * scalar_from_i64(*w); + t_fold += lambda[k] * mu[j] * value_blindings[k * cols + j]; + } + s_fold += lambda[k] * row_blindings[k]; + } + + let rho: Vec = (0..cols).map(|_| random_scalar()).collect(); + let rho_s = random_scalar(); + let rho_t = random_scalar(); + let r1 = RistrettoPoint::multiscalar_mul( + rho.iter().chain(std::iter::once(&rho_s)), + basis[..cols] + .iter() + .chain(std::iter::once(&gens.B_blinding)), + ); + let mu_rho: Scalar = mu.iter().zip(rho.iter()).map(|(m, r)| m * r).sum(); + let r2 = &mu_rho * &RISTRETTO_BASEPOINT_TABLE + gens.B_blinding * rho_t; + transcript.append_message(b"link-r1", &compress(&r1)); + transcript.append_message(b"link-r2", &compress(&r2)); + let c = transcript_scalar(transcript, b"link-challenge"); + + LinkProof { + r1: compress(&r1), + r2: compress(&r2), + sigma_z: rho.iter().zip(z.iter()).map(|(r, zj)| r + c * zj).collect(), + sigma_s: rho_s + c * s_fold, + sigma_t: rho_t + c * t_fold, + } +} + +fn verify_link( + transcript: &mut Transcript, + label: &'static [u8], + row_commitments: &[[u8; 32]], + value_commitments: &[[u8; 32]], + cols: usize, + shift: &Scalar, + proof: &LinkProof, +) -> Result<(), NativeError> { + let gens = pc_gens(); + let basis = g_basis(cols); + let row_count = row_commitments.len(); + if proof.sigma_z.len() != cols || value_commitments.len() != row_count * cols { + return Err(NativeError::InvalidDimensions("link proof shape".into())); + } + let lambda = transcript_scalars(transcript, label, row_count); + let mu = transcript_scalars(transcript, label, cols); + transcript.append_message(b"link-r1", &proof.r1); + transcript.append_message(b"link-r2", &proof.r2); + let c = transcript_scalar(transcript, b"link-challenge"); + + // P1 = sum_k lambda_k C_k + let p1 = RistrettoPoint::vartime_multiscalar_mul( + lambda.iter(), + row_commitments + .iter() + .map(decompress) + .collect::, _>>()? + .iter(), + ); + // P2' = sum_{k,j} lambda_k mu_j W_kj - shift * Lambda * B, where the W + // commit shifted weights and Lambda = (sum lambda)(sum mu). + let mut weights_scalars = Vec::with_capacity(row_count * cols); + for lk in lambda.iter() { + for mj in mu.iter() { + weights_scalars.push(lk * mj); + } + } + let lambda_sum: Scalar = lambda.iter().sum(); + let mu_sum: Scalar = mu.iter().sum(); + let p2 = RistrettoPoint::vartime_multiscalar_mul( + weights_scalars.iter(), + value_commitments + .iter() + .map(decompress) + .collect::, _>>()? + .iter(), + ) - &(shift * lambda_sum * mu_sum) * &RISTRETTO_BASEPOINT_TABLE; + + let lhs1 = RistrettoPoint::vartime_multiscalar_mul( + proof.sigma_z.iter().chain(std::iter::once(&proof.sigma_s)), + basis[..cols] + .iter() + .chain(std::iter::once(&gens.B_blinding)), + ); + if lhs1 != decompress(&proof.r1)? + p1 * c { + return Err(NativeError::Halo2( + "adapter link proof (rows) failed".into(), + )); + } + let mu_sigma: Scalar = mu + .iter() + .zip(proof.sigma_z.iter()) + .map(|(m, s)| m * s) + .sum(); + let lhs2 = &mu_sigma * &RISTRETTO_BASEPOINT_TABLE + gens.B_blinding * proof.sigma_t; + if lhs2 != decompress(&proof.r2)? + p2 * c { + return Err(NativeError::Halo2( + "adapter link proof (values) failed".into(), + )); + } + Ok(()) +} + +pub(crate) fn adapter_setup( + input: &AdapterCommitmentInput, + salt: &[u8; 32], +) -> Result { + let secrets = adapter_secrets(input, salt)?; + let mut transcript = adapter_transcript(&secrets.core); + let in_dim = secrets.core.in_dim; + let rank = secrets.core.rank; + let a_values = rank * in_dim; + + let link_a = prove_link( + &mut transcript, + b"link-a", + &input.a, + &secrets.row_blindings_a, + &secrets.weight_blindings[..a_values], + in_dim, + ); + let link_b = prove_link( + &mut transcript, + b"link-b", + &input.b, + &secrets.row_blindings_b, + &secrets.weight_blindings[a_values..], + rank, + ); + + // One-time exact range proof: every shifted weight in [0, 2*value_bound]. + let value_bound = value_bound_int(&input.fixed_point); + let weights: Vec = input + .a + .iter() + .flatten() + .chain(input.b.iter().flatten()) + .map(|w| BigInt::from(*w)) + .collect(); + let mut class = BoundedClass::from_commitments( + secrets + .core + .weight_commitments + .iter() + .map(|c| vec![*c]) + .collect(), + weights.len(), + &-&value_bound, + &(&value_bound * 2), + )?; + class.openings = weights + .iter() + .zip(secrets.weight_blindings.iter()) + .map(|(w, b)| { + let shifted: u64 = (w + &value_bound).try_into().expect("shifted weight fits"); + vec![(shifted, *b)] + }) + .collect(); + let mut entries = Vec::new(); + class.push_entries(&mut entries)?; + let ranges = prove_ranges(&transcript, entries)?; + + Ok(AdapterSetupPub { + core: secrets.core, + link_a, + link_b, + ranges, + }) +} + +/// Verify the one-time adapter setup proofs. Results are cached by the +/// commitment string, so a batch verification pays this once per adapter. +pub fn verify_adapter_setup(setup: &AdapterSetupPub) -> Result<(), NativeError> { + let commitment = adapter_commitment_string(&setup.core); + static VERIFIED: OnceLock>> = OnceLock::new(); + let verified = VERIFIED.get_or_init(|| Mutex::new(HashSet::new())); + if verified + .lock() + .expect("verified cache poisoned") + .contains(&commitment) + { + return Ok(()); + } + + let core = &setup.core; + if core.scheme != SIGMA_SCHEME_ID || core.schema_version != SIGMA_SCHEMA_VERSION { + return Err(NativeError::InvalidDimensions( + "unsupported adapter setup scheme".into(), + )); + } + if core.row_commitments_a.len() != core.rank + || core.row_commitments_b.len() != core.out_dim + || core.weight_commitments.len() != core.rank * core.in_dim + core.out_dim * core.rank + || core.in_dim == 0 + || core.rank == 0 + || core.out_dim == 0 + { + return Err(NativeError::InvalidDimensions( + "adapter setup shape mismatch".into(), + )); + } + if core.scaling_den <= 0 + || core.fixed_point.value_bits == 0 + || core.fixed_point.value_bits > 63 + || core.fixed_point.scale_bits >= core.fixed_point.value_bits + || core.fixed_point.intermediate_bits == 0 + { + return Err(NativeError::InvalidDimensions( + "adapter setup config invalid".into(), + )); + } + validate_field_safety_v4( + core.in_dim, + core.rank, + core.out_dim, + &core.fixed_point, + core.scaling_num, + core.scaling_den, + )?; + + let mut transcript = adapter_transcript(core); + let value_bound = value_bound_int(&core.fixed_point); + let shift = scalar_from_bigint(&value_bound)?; + let a_values = core.rank * core.in_dim; + verify_link( + &mut transcript, + b"link-a", + &core.row_commitments_a, + &core.weight_commitments[..a_values], + core.in_dim, + &shift, + &setup.link_a, + )?; + verify_link( + &mut transcript, + b"link-b", + &core.row_commitments_b, + &core.weight_commitments[a_values..], + core.rank, + &shift, + &setup.link_b, + )?; + + let class = BoundedClass::from_commitments( + core.weight_commitments.iter().map(|c| vec![*c]).collect(), + core.weight_commitments.len(), + &-&value_bound, + &(&value_bound * 2), + )?; + let mut entries = Vec::new(); + class.push_entries(&mut entries)?; + verify_ranges(&transcript, entries, &setup.ranges)?; + + let mut guard = verified.lock().expect("verified cache poisoned"); + if guard.len() >= 1024 { + guard.clear(); + } + guard.insert(commitment); + Ok(()) +} + +// --------------------------------------------------------------------------- +// Invocation proof +// --------------------------------------------------------------------------- + +#[derive(Serialize, Deserialize)] +struct LinearProof { + r_pa: [u8; 32], + r_su: [u8; 32], + r_sra: [u8; 32], + r_sw: Option<[u8; 32]>, + r_srf: Option<[u8; 32]>, + r_eq: Scalar, + sigma_a: Vec, + sigma_ba: Scalar, + sigma_su: Scalar, + sigma_bsu: Scalar, + sigma_sra: Scalar, + sigma_bsra: Scalar, + sigma_sw: Option, + sigma_bsw: Option, + sigma_srf: Option, + sigma_bsrf: Option, +} + +#[derive(Serialize, Deserialize)] +struct QuadProof { + r_b: [u8; 32], + r_u: Vec<[u8; 32]>, + c_t1: [u8; 32], + c_t0: [u8; 32], + sigma_b: Vec, + sigma_bb: Scalar, + sigma_u: Vec, + sigma_bu: Vec, + omega: Scalar, +} + +#[derive(Serialize, Deserialize)] +struct InvocationProof { + c_rema: Vec>, + c_u: Vec>, + c_w: Option>>, + c_remb: Vec>, + c_remf: Option>>, + linear: LinearProof, + quad: QuadProof, + ranges: RangeBundle, +} + +struct StatementContext { + statement: NativeStatement, + in_dim: usize, + rank: usize, + out_dim: usize, + scale: BigInt, + trivial_scaling: bool, + has_remf: bool, +} + +fn statement_context(statement_json: &str) -> Result { + let statement: NativeStatement = + serde_json::from_str(statement_json).map_err(|e| NativeError::Json(e.to_string()))?; + let in_dim = statement.x.len(); + let out_dim = statement.delta.len(); + let rank = statement.rank; + if in_dim == 0 || out_dim == 0 || rank == 0 { + return Err(NativeError::InvalidDimensions( + "statement dimensions must be positive".into(), + )); + } + let config = &statement.fixed_point; + if config.value_bits == 0 + || config.value_bits > 63 + || config.scale_bits >= config.value_bits + || config.intermediate_bits == 0 + || statement.scaling_den <= 0 + { + return Err(NativeError::InvalidDimensions( + "invalid statement fixed-point/scaling config".into(), + )); + } + validate_field_safety_v4( + in_dim, + rank, + out_dim, + config, + statement.scaling_num, + statement.scaling_den, + )?; + let value_bound = value_bound_int(config); + for value in statement.x.iter().chain(statement.delta.iter()) { + let value = BigInt::from(*value); + if value < -&value_bound || value > value_bound { + return Err(NativeError::InvalidDimensions( + "public statement value exceeds bound".into(), + )); + } + } + let digest = statement + .statement_digest + .strip_prefix("0x") + .unwrap_or(&statement.statement_digest); + if digest.len() != 64 || !digest.chars().all(|c| c.is_ascii_hexdigit()) { + return Err(NativeError::InvalidDimensions( + "statement_digest must be 32 bytes of hex".into(), + )); + } + let commitment = &statement.adapter_commitment; + if commitment.len() != 64 || !commitment.chars().all(|c| c.is_ascii_hexdigit()) { + return Err(NativeError::InvalidDimensions( + "adapter_commitment must be a sha256 hex string".into(), + )); + } + let trivial_scaling = statement.scaling_num == 1 && statement.scaling_den == 1; + let has_remf = statement.scaling_den != 1; + Ok(StatementContext { + scale: BigInt::from(1) << statement.fixed_point.scale_bits, + statement, + in_dim, + rank, + out_dim, + trivial_scaling, + has_remf, + }) +} + +fn invocation_transcript(statement_json: &str, ctx: &StatementContext) -> Transcript { + let mut transcript = Transcript::new(TRANSCRIPT_LABEL); + transcript.append_message(b"phase", b"invocation"); + // The canonical statement JSON binds x, delta, dims, config, scaling, + // the adapter commitment and the upstream statement digest. + transcript.append_message( + b"statement", + blake3::hash(statement_json.as_bytes()).as_bytes(), + ); + transcript.append_message(b"adapter", ctx.statement.adapter_commitment.as_bytes()); + transcript +} + +struct ProjectedCommitments { + p_a: RistrettoPoint, + p_b: RistrettoPoint, + d_u: Vec, + d_su: RistrettoPoint, + d_sra: RistrettoPoint, + d_sw: Option, + d_srb: RistrettoPoint, + d_srf: Option, + /// Public S_w = sum_j beta_j delta_j when scaling is trivial. + public_sw: Scalar, + public_d_delta: Scalar, +} + +#[allow(clippy::too_many_arguments)] +fn projected_commitments( + ctx: &StatementContext, + row_commitments_a: &[[u8; 32]], + row_commitments_b: &[[u8; 32]], + rema: &BoundedClass, + u: &BoundedClass, + w: Option<&BoundedClass>, + remb: &BoundedClass, + remf: Option<&BoundedClass>, + gamma: &[Scalar], + beta: &[Scalar], +) -> Result { + let combine = + |scalars: &[Scalar], commitments: &[[u8; 32]]| -> Result { + Ok(RistrettoPoint::vartime_multiscalar_mul( + scalars.iter(), + commitments + .iter() + .map(decompress) + .collect::, _>>()? + .iter(), + )) + }; + // One flat MSM per class: sum_j s_j * (sum_i 2^(64 i) C_{j,i} + lower*B) + // = sum_{j,i} (s_j 2^(64 i)) C_{j,i} + lower*(sum_j s_j)*B. + let combine_class = + |scalars: &[Scalar], class: &BoundedClass| -> Result { + let limbs = class.plan.limb_maxes.len(); + let shift_64 = Scalar::from(u64::MAX) + Scalar::one(); + let mut msm_scalars = Vec::with_capacity(scalars.len() * limbs + 1); + let mut msm_points = Vec::with_capacity(scalars.len() * limbs + 1); + for (scalar, limb_row) in scalars.iter().zip(class.commitments.iter()) { + let mut base = *scalar; + for limb in limb_row { + msm_scalars.push(base); + msm_points.push(decompress(limb)?); + base *= shift_64; + } + } + msm_scalars.push(scalar_from_bigint(&class.lower)? * scalars.iter().sum::()); + msm_points.push(pc_gens().B); + Ok(RistrettoPoint::vartime_multiscalar_mul( + msm_scalars.iter(), + msm_points.iter(), + )) + }; + let d_u: Vec = (0..ctx.rank) + .map(|k| u.value_commitment(k)) + .collect::>()?; + let mut d_su = RistrettoPoint::identity(); + for (scalar, point) in gamma.iter().zip(d_u.iter()) { + d_su += point * scalar; + } + let public_sw: Scalar = beta + .iter() + .zip(ctx.statement.delta.iter()) + .map(|(b, d)| b * scalar_from_i64(*d)) + .sum(); + Ok(ProjectedCommitments { + p_a: combine(gamma, row_commitments_a)?, + p_b: combine(beta, row_commitments_b)?, + d_su, + d_u, + d_sra: combine_class(gamma, rema)?, + d_sw: w.map(|class| combine_class(beta, class)).transpose()?, + d_srb: combine_class(beta, remb)?, + d_srf: remf.map(|class| combine_class(beta, class)).transpose()?, + public_d_delta: public_sw, + public_sw, + }) +} + +pub fn prove_invocation( + statement_json: &str, + witness_json: &str, + salt: &[u8; 32], +) -> Result, NativeError> { + let timing = std::env::var("ZKLORA_V4_TIMING").is_ok(); + let mut mark = std::time::Instant::now(); + let mut lap = |label: &str| { + if timing { + eprintln!(" prove {label}: {:?}", mark.elapsed()); + } + mark = std::time::Instant::now(); + }; + let ctx = statement_context(statement_json)?; + let witness: NativeWitness = + serde_json::from_str(witness_json).map_err(|e| NativeError::Json(e.to_string()))?; + if witness.a.len() != ctx.rank + || witness.a.iter().any(|row| row.len() != ctx.in_dim) + || witness.b.len() != ctx.out_dim + || witness.b.iter().any(|row| row.len() != ctx.rank) + { + return Err(NativeError::InvalidDimensions( + "witness shape does not match statement".into(), + )); + } + let input = AdapterCommitmentInput { + schema_version: SIGMA_SCHEMA_VERSION, + in_dim: ctx.in_dim, + rank: ctx.rank, + out_dim: ctx.out_dim, + fixed_point: ctx.statement.fixed_point.clone(), + scaling_num: ctx.statement.scaling_num, + scaling_den: ctx.statement.scaling_den, + a: witness.a.clone(), + b: witness.b.clone(), + }; + let secrets = cached_adapter_secrets(&input, salt)?; + if secrets.commitment != ctx.statement.adapter_commitment { + return Err(NativeError::InvalidDimensions( + "witness does not match statement adapter commitment".into(), + )); + } + lap("setup"); + + // --- exact reference pipeline (arbitrary precision) ------------------- + let scale = &ctx.scale; + let mut u_values = Vec::with_capacity(ctx.rank); + let mut rema_values = Vec::with_capacity(ctx.rank); + for row in &witness.a { + let raw: BigInt = row + .iter() + .zip(ctx.statement.x.iter()) + .map(|(w, xi)| BigInt::from(*w) * BigInt::from(*xi)) + .sum(); + let q = div_round_canonical_int(&raw, scale)?; + rema_values.push(&raw - &q * scale); + u_values.push(q); + } + let mut w_values = Vec::with_capacity(ctx.out_dim); + let mut remb_values = Vec::with_capacity(ctx.out_dim); + let mut remf_values = Vec::with_capacity(ctx.out_dim); + let num = BigInt::from(ctx.statement.scaling_num); + let den = BigInt::from(ctx.statement.scaling_den); + for (row, delta) in witness.b.iter().zip(ctx.statement.delta.iter()) { + let raw: BigInt = row + .iter() + .zip(u_values.iter()) + .map(|(weight, u)| BigInt::from(*weight) * u) + .sum(); + let w = div_round_canonical_int(&raw, scale)?; + remb_values.push(&raw - &w * scale); + let scaled = &w * # + let final_delta = div_round_canonical_int(&scaled, &den)?; + if final_delta != BigInt::from(*delta) { + return Err(NativeError::InvalidDimensions( + "witness does not satisfy statement delta".into(), + )); + } + remf_values.push(&scaled - &final_delta * &den); + w_values.push(w); + } + + // --- commitments ------------------------------------------------------- + let intermediate_bound = intermediate_bound_int(&ctx.statement.fixed_point); + let (rema_lower, rema_upper) = canonical_remainder_interval_int(scale); + let rema_width = &rema_upper - &rema_lower; + let rema = BoundedClass::commit(&rema_values, &rema_lower, &rema_width)?; + let u = BoundedClass::commit(&u_values, &-&intermediate_bound, &(&intermediate_bound * 2))?; + let remb = BoundedClass::commit(&remb_values, &rema_lower, &rema_width)?; + let w = if ctx.trivial_scaling { + None + } else { + Some(BoundedClass::commit( + &w_values, + &-&intermediate_bound, + &(&intermediate_bound * 2), + )?) + }; + let remf = if ctx.has_remf { + let (lower, upper) = canonical_remainder_interval_int(&den); + Some(BoundedClass::commit( + &remf_values, + &lower, + &(&upper - &lower), + )?) + } else { + None + }; + + lap("pipeline+commit"); + let mut transcript = invocation_transcript(statement_json, &ctx); + rema.absorb(&mut transcript, b"c-rema"); + u.absorb(&mut transcript, b"c-u"); + if let Some(class) = &w { + class.absorb(&mut transcript, b"c-w"); + } + remb.absorb(&mut transcript, b"c-remb"); + if let Some(class) = &remf { + class.absorb(&mut transcript, b"c-remf"); + } + let gamma = transcript_scalars(&mut transcript, b"gamma", ctx.rank); + let beta = transcript_scalars(&mut transcript, b"beta", ctx.out_dim); + let zeta = transcript_scalar(&mut transcript, b"zeta"); + + let projected = projected_commitments( + &ctx, + &secrets.core.row_commitments_a, + &secrets.core.row_commitments_b, + &rema, + &u, + w.as_ref(), + &remb, + remf.as_ref(), + &gamma, + &beta, + )?; + + // Secret openings of the projected quantities. + let gens = pc_gens(); + let basis = g_basis(ctx.in_dim.max(ctx.rank)); + let scale_scalar = scalar_from_bigint(scale)?; + let a_proj: Vec = (0..ctx.in_dim) + .map(|j| { + (0..ctx.rank) + .map(|k| gamma[k] * scalar_from_i64(witness.a[k][j])) + .sum() + }) + .collect(); + let b_proj: Vec = (0..ctx.rank) + .map(|k| { + (0..ctx.out_dim) + .map(|j| beta[j] * scalar_from_i64(witness.b[j][k])) + .sum() + }) + .collect(); + let blinding_a: Scalar = gamma + .iter() + .zip(secrets.row_blindings_a.iter()) + .map(|(g, b)| g * b) + .sum(); + let blinding_b: Scalar = beta + .iter() + .zip(secrets.row_blindings_b.iter()) + .map(|(bj, blind)| bj * blind) + .sum(); + let fold = |scalars: &[Scalar], values: &[BigInt], blindings: &[Scalar]| -> (Scalar, Scalar) { + let mut value_sum = Scalar::zero(); + let mut blinding_sum = Scalar::zero(); + for ((scalar, value), blinding) in scalars.iter().zip(values).zip(blindings) { + value_sum += scalar * scalar_from_bigint(value).expect("bounded value"); + blinding_sum += scalar * blinding; + } + (value_sum, blinding_sum) + }; + let (s_u, b_su) = fold(&gamma, &u_values, &u.value_blindings); + let (s_ra, b_sra) = fold(&gamma, &rema_values, &rema.value_blindings); + let (_s_rb, b_srb) = fold(&beta, &remb_values, &remb.value_blindings); + let (s_w, b_sw) = match &w { + Some(class) => fold(&beta, &w_values, &class.value_blindings), + None => (projected.public_sw, Scalar::zero()), + }; + let (_s_rf, b_srf) = match &remf { + Some(class) => fold(&beta, &remf_values, &class.value_blindings), + None => (Scalar::zero(), Scalar::zero()), + }; + let num_scalar = scalar_from_i64(ctx.statement.scaling_num); + + // --- linear Schnorr: E1 + zeta*E3 over (a_proj, S_u, S_ra[, S_w, S_rf]) + let x_scalars: Vec = ctx + .statement + .x + .iter() + .map(|v| scalar_from_i64(*v)) + .collect(); + let rho_a: Vec = (0..ctx.in_dim).map(|_| random_scalar()).collect(); + let rho_ba = random_scalar(); + let rho_su = random_scalar(); + let rho_bsu = random_scalar(); + let rho_sra = random_scalar(); + let rho_bsra = random_scalar(); + let (rho_sw, rho_bsw, rho_srf, rho_bsrf) = if ctx.trivial_scaling { + ( + Scalar::zero(), + Scalar::zero(), + Scalar::zero(), + Scalar::zero(), + ) + } else { + ( + random_scalar(), + random_scalar(), + if ctx.has_remf { + random_scalar() + } else { + Scalar::zero() + }, + if ctx.has_remf { + random_scalar() + } else { + Scalar::zero() + }, + ) + }; + let r_pa = RistrettoPoint::multiscalar_mul( + rho_a.iter().chain(std::iter::once(&rho_ba)), + basis[..ctx.in_dim] + .iter() + .chain(std::iter::once(&gens.B_blinding)), + ); + let commit_pair = |value: &Scalar, blinding: &Scalar| -> RistrettoPoint { + value * &RISTRETTO_BASEPOINT_TABLE + blinding_table() * blinding + }; + let r_su = commit_pair(&rho_su, &rho_bsu); + let r_sra = commit_pair(&rho_sra, &rho_bsra); + let r_sw = (!ctx.trivial_scaling).then(|| commit_pair(&rho_sw, &rho_bsw)); + let r_srf = ctx.has_remf.then(|| commit_pair(&rho_srf, &rho_bsrf)); + let x_rho: Scalar = x_scalars.iter().zip(rho_a.iter()).map(|(x, r)| x * r).sum(); + // E1: - s*S_u - S_ra = 0 ; E3: num*S_w - den*D_delta - S_rf = 0. + let mut r_eq = x_rho - scale_scalar * rho_su - rho_sra; + if !ctx.trivial_scaling { + r_eq += zeta * (num_scalar * rho_sw - rho_srf); + } + transcript.append_message(b"lin-r-pa", &compress(&r_pa)); + transcript.append_message(b"lin-r-su", &compress(&r_su)); + transcript.append_message(b"lin-r-sra", &compress(&r_sra)); + if let Some(point) = &r_sw { + transcript.append_message(b"lin-r-sw", &compress(point)); + } + if let Some(point) = &r_srf { + transcript.append_message(b"lin-r-srf", &compress(point)); + } + transcript.append_message(b"lin-r-eq", r_eq.as_bytes()); + let c1 = transcript_scalar(&mut transcript, b"lin-challenge"); + + let linear = LinearProof { + r_pa: compress(&r_pa), + r_su: compress(&r_su), + r_sra: compress(&r_sra), + r_sw: r_sw.as_ref().map(compress), + r_srf: r_srf.as_ref().map(compress), + r_eq, + sigma_a: rho_a + .iter() + .zip(a_proj.iter()) + .map(|(rho, a)| rho + c1 * a) + .collect(), + sigma_ba: rho_ba + c1 * blinding_a, + sigma_su: rho_su + c1 * s_u, + sigma_bsu: rho_bsu + c1 * b_su, + sigma_sra: rho_sra + c1 * s_ra, + sigma_bsra: rho_bsra + c1 * b_sra, + sigma_sw: (!ctx.trivial_scaling).then(|| rho_sw + c1 * s_w), + sigma_bsw: (!ctx.trivial_scaling).then(|| rho_bsw + c1 * b_sw), + sigma_srf: ctx.has_remf.then(|| rho_srf + c1 * _s_rf), + sigma_bsrf: ctx.has_remf.then(|| rho_bsrf + c1 * b_srf), + }; + + // --- quadratic sigma: = s*S_w + S_rb ----------------------- + let u_scalars: Vec = u_values + .iter() + .map(|v| scalar_from_bigint(v).expect("bounded")) + .collect(); + let rho_b: Vec = (0..ctx.rank).map(|_| random_scalar()).collect(); + let rho_bb = random_scalar(); + let rho_u: Vec = (0..ctx.rank).map(|_| random_scalar()).collect(); + let rho_bu: Vec = (0..ctx.rank).map(|_| random_scalar()).collect(); + let r_b = RistrettoPoint::multiscalar_mul( + rho_b.iter().chain(std::iter::once(&rho_bb)), + basis[..ctx.rank] + .iter() + .chain(std::iter::once(&gens.B_blinding)), + ); + let r_u: Vec = rho_u + .iter() + .zip(rho_bu.iter()) + .map(|(value, blinding)| commit_pair(value, blinding)) + .collect(); + let t1: Scalar = rho_b + .iter() + .zip(u_scalars.iter()) + .map(|(r, u)| r * u) + .sum::() + + b_proj + .iter() + .zip(rho_u.iter()) + .map(|(b, r)| b * r) + .sum::(); + let t0: Scalar = rho_b.iter().zip(rho_u.iter()).map(|(rb, ru)| rb * ru).sum(); + let bt1 = random_scalar(); + let bt0 = random_scalar(); + let c_t1 = commit_pair(&t1, &bt1); + let c_t0 = commit_pair(&t0, &bt0); + transcript.append_message(b"quad-r-b", &compress(&r_b)); + absorb_points( + &mut transcript, + b"quad-r-u", + &r_u.iter().map(compress).collect::>(), + ); + transcript.append_message(b"quad-c-t1", &compress(&c_t1)); + transcript.append_message(b"quad-c-t0", &compress(&c_t0)); + let c2 = transcript_scalar(&mut transcript, b"quad-challenge"); + + // omega opens C_T0 + c2 C_T1 + c2^2 (s*D_Sw + D_Srb) on the blinding base. + let omega = bt0 + c2 * bt1 + c2 * c2 * (scale_scalar * b_sw + b_srb); + let quad = QuadProof { + r_b: compress(&r_b), + r_u: r_u.iter().map(compress).collect(), + c_t1: compress(&c_t1), + c_t0: compress(&c_t0), + sigma_b: rho_b + .iter() + .zip(b_proj.iter()) + .map(|(rho, b)| rho + c2 * b) + .collect(), + sigma_bb: rho_bb + c2 * blinding_b, + sigma_u: rho_u + .iter() + .zip(u_scalars.iter()) + .map(|(rho, u)| rho + c2 * u) + .collect(), + sigma_bu: rho_bu + .iter() + .zip(u.value_blindings.iter()) + .map(|(rho, b)| rho + c2 * b) + .collect(), + omega, + }; + lap("sigma-protocols"); + // Absorb responses so the range-proof transcripts bind the sigma layer. + for scalar in linear + .sigma_a + .iter() + .chain(quad.sigma_b.iter()) + .chain(quad.sigma_u.iter()) + { + transcript.append_message(b"sigma-response", scalar.as_bytes()); + } + transcript.append_message(b"sigma-omega", quad.omega.as_bytes()); + + // --- range proofs ------------------------------------------------------- + let mut entries = Vec::new(); + rema.push_entries(&mut entries)?; + u.push_entries(&mut entries)?; + if let Some(class) = &w { + class.push_entries(&mut entries)?; + } + remb.push_entries(&mut entries)?; + if let Some(class) = &remf { + class.push_entries(&mut entries)?; + } + lap("push-entries"); + let ranges = prove_ranges_with_engine(&transcript, entries, invocation_range_engine())?; + lap("ranges"); + + let proof = InvocationProof { + c_rema: rema.commitments, + c_u: u.commitments, + c_w: w.map(|class| class.commitments), + c_remb: remb.commitments, + c_remf: remf.map(|class| class.commitments), + linear, + quad, + ranges, + }; + bincode::serialize(&proof).map_err(|e| NativeError::Json(e.to_string())) +} + +pub fn verify_invocation( + statement_json: &str, + proof_bytes: &[u8], + setup: &AdapterSetupPub, +) -> Result { + let commitment = adapter_commitment_string(&setup.core); + verify_invocation_cached(statement_json, proof_bytes, setup, &commitment) +} + +fn verify_invocation_cached( + statement_json: &str, + proof_bytes: &[u8], + setup: &AdapterSetupPub, + commitment: &str, +) -> Result { + match verify_invocation_inner(statement_json, proof_bytes, setup, commitment) { + Ok(()) => Ok(true), + Err(NativeError::Halo2(_)) => Ok(false), + Err(NativeError::InvalidDimensions(_)) => Ok(false), + Err(other) => Err(other), + } +} + +fn verify_invocation_inner( + statement_json: &str, + proof_bytes: &[u8], + setup: &AdapterSetupPub, + commitment: &str, +) -> Result<(), NativeError> { + let ctx = statement_context(statement_json)?; + let core = &setup.core; + if commitment != ctx.statement.adapter_commitment { + return Err(NativeError::InvalidDimensions( + "statement adapter commitment does not match setup".into(), + )); + } + if core.in_dim != ctx.in_dim + || core.rank != ctx.rank + || core.out_dim != ctx.out_dim + || core.scaling_num != ctx.statement.scaling_num + || core.scaling_den != ctx.statement.scaling_den + || serde_json::to_string(&core.fixed_point).map_err(|e| NativeError::Json(e.to_string()))? + != serde_json::to_string(&ctx.statement.fixed_point) + .map_err(|e| NativeError::Json(e.to_string()))? + { + return Err(NativeError::InvalidDimensions( + "statement does not match adapter setup".into(), + )); + } + verify_adapter_setup(setup)?; + + // Bound deserialization by the input size: bincode pre-allocates + // collections from length prefixes, so an attacker-supplied blob could + // otherwise demand far more memory than its own length. The options + // mirror `bincode::serialize`'s wire format (fixint, trailing allowed). + let proof: InvocationProof = { + use bincode::Options; + bincode::DefaultOptions::new() + .with_fixint_encoding() + .allow_trailing_bytes() + .with_limit(proof_bytes.len().saturating_mul(2) as u64) + .deserialize(proof_bytes) + .map_err(|e| NativeError::Json(e.to_string()))? + }; + if (proof.c_w.is_some() == ctx.trivial_scaling) || (proof.c_remf.is_some() != ctx.has_remf) { + return Err(NativeError::InvalidDimensions( + "proof scaling structure mismatch".into(), + )); + } + if proof.linear.sigma_a.len() != ctx.in_dim + || proof.quad.sigma_b.len() != ctx.rank + || proof.quad.sigma_u.len() != ctx.rank + || proof.quad.sigma_bu.len() != ctx.rank + || proof.quad.r_u.len() != ctx.rank + || proof.linear.r_sw.is_some() == ctx.trivial_scaling + || proof.linear.sigma_sw.is_some() == ctx.trivial_scaling + || proof.linear.sigma_bsw.is_some() == ctx.trivial_scaling + || proof.linear.r_srf.is_some() != ctx.has_remf + || proof.linear.sigma_srf.is_some() != ctx.has_remf + || proof.linear.sigma_bsrf.is_some() != ctx.has_remf + { + return Err(NativeError::InvalidDimensions( + "proof shape mismatch".into(), + )); + } + + let scale = &ctx.scale; + let intermediate_bound = intermediate_bound_int(&ctx.statement.fixed_point); + let (rema_lower, rema_upper) = canonical_remainder_interval_int(scale); + let rema_width = &rema_upper - &rema_lower; + let rema = + BoundedClass::from_commitments(proof.c_rema.clone(), ctx.rank, &rema_lower, &rema_width)?; + let u = BoundedClass::from_commitments( + proof.c_u.clone(), + ctx.rank, + &-&intermediate_bound, + &(&intermediate_bound * 2), + )?; + let remb = BoundedClass::from_commitments( + proof.c_remb.clone(), + ctx.out_dim, + &rema_lower, + &rema_width, + )?; + let w = proof + .c_w + .clone() + .map(|commitments| { + BoundedClass::from_commitments( + commitments, + ctx.out_dim, + &-&intermediate_bound, + &(&intermediate_bound * 2), + ) + }) + .transpose()?; + let den = BigInt::from(ctx.statement.scaling_den); + let remf = proof + .c_remf + .clone() + .map(|commitments| { + let (lower, upper) = canonical_remainder_interval_int(&den); + BoundedClass::from_commitments(commitments, ctx.out_dim, &lower, &(&upper - &lower)) + }) + .transpose()?; + + let mut transcript = invocation_transcript(statement_json, &ctx); + rema.absorb(&mut transcript, b"c-rema"); + u.absorb(&mut transcript, b"c-u"); + if let Some(class) = &w { + class.absorb(&mut transcript, b"c-w"); + } + remb.absorb(&mut transcript, b"c-remb"); + if let Some(class) = &remf { + class.absorb(&mut transcript, b"c-remf"); + } + let gamma = transcript_scalars(&mut transcript, b"gamma", ctx.rank); + let beta = transcript_scalars(&mut transcript, b"beta", ctx.out_dim); + let zeta = transcript_scalar(&mut transcript, b"zeta"); + + let projected = projected_commitments( + &ctx, + &core.row_commitments_a, + &core.row_commitments_b, + &rema, + &u, + w.as_ref(), + &remb, + remf.as_ref(), + &gamma, + &beta, + )?; + + let gens = pc_gens(); + let basis = g_basis(ctx.in_dim.max(ctx.rank)); + let scale_scalar = scalar_from_bigint(scale)?; + let num_scalar = scalar_from_i64(ctx.statement.scaling_num); + let den_scalar = scalar_from_i64(ctx.statement.scaling_den); + + // --- linear Schnorr ----------------------------------------------------- + let linear = &proof.linear; + transcript.append_message(b"lin-r-pa", &linear.r_pa); + transcript.append_message(b"lin-r-su", &linear.r_su); + transcript.append_message(b"lin-r-sra", &linear.r_sra); + if let Some(point) = &linear.r_sw { + transcript.append_message(b"lin-r-sw", point); + } + if let Some(point) = &linear.r_srf { + transcript.append_message(b"lin-r-srf", point); + } + transcript.append_message(b"lin-r-eq", linear.r_eq.as_bytes()); + let c1 = transcript_scalar(&mut transcript, b"lin-challenge"); + + let lhs_pa = RistrettoPoint::vartime_multiscalar_mul( + linear + .sigma_a + .iter() + .chain(std::iter::once(&linear.sigma_ba)), + basis[..ctx.in_dim] + .iter() + .chain(std::iter::once(&gens.B_blinding)), + ); + if lhs_pa != decompress(&linear.r_pa)? + projected.p_a * c1 { + return Err(NativeError::Halo2("linear proof: P_A check failed".into())); + } + let check_scalar_commitment = |sigma: &Scalar, + sigma_blind: &Scalar, + announcement: &[u8; 32], + derived: &RistrettoPoint| + -> Result<(), NativeError> { + let lhs = sigma * &RISTRETTO_BASEPOINT_TABLE + gens.B_blinding * sigma_blind; + if lhs != decompress(announcement)? + derived * c1 { + return Err(NativeError::Halo2( + "linear proof: scalar check failed".into(), + )); + } + Ok(()) + }; + check_scalar_commitment( + &linear.sigma_su, + &linear.sigma_bsu, + &linear.r_su, + &projected.d_su, + )?; + check_scalar_commitment( + &linear.sigma_sra, + &linear.sigma_bsra, + &linear.r_sra, + &projected.d_sra, + )?; + if let (Some(sigma), Some(blind), Some(announcement), Some(derived)) = ( + &linear.sigma_sw, + &linear.sigma_bsw, + &linear.r_sw, + &projected.d_sw, + ) { + check_scalar_commitment(sigma, blind, announcement, derived)?; + } + if let (Some(sigma), Some(blind), Some(announcement), Some(derived)) = ( + &linear.sigma_srf, + &linear.sigma_bsrf, + &linear.r_srf, + &projected.d_srf, + ) { + check_scalar_commitment(sigma, blind, announcement, derived)?; + } + let x_scalars: Vec = ctx + .statement + .x + .iter() + .map(|v| scalar_from_i64(*v)) + .collect(); + let x_sigma: Scalar = x_scalars + .iter() + .zip(linear.sigma_a.iter()) + .map(|(x, s)| x * s) + .sum(); + let mut lhs_eq = x_sigma - scale_scalar * linear.sigma_su - linear.sigma_sra; + let mut rhs_eq = linear.r_eq; + if !ctx.trivial_scaling { + let sigma_sw = linear.sigma_sw.expect("checked above"); + let sigma_srf = linear.sigma_srf.unwrap_or_else(Scalar::zero); + lhs_eq += zeta * (num_scalar * sigma_sw - sigma_srf); + rhs_eq += c1 * zeta * den_scalar * projected.public_d_delta; + } + if lhs_eq != rhs_eq { + return Err(NativeError::Halo2( + "linear proof: projected equation failed".into(), + )); + } + + // --- quadratic sigma ---------------------------------------------------- + let quad = &proof.quad; + transcript.append_message(b"quad-r-b", &quad.r_b); + absorb_points(&mut transcript, b"quad-r-u", &quad.r_u); + transcript.append_message(b"quad-c-t1", &quad.c_t1); + transcript.append_message(b"quad-c-t0", &quad.c_t0); + let c2 = transcript_scalar(&mut transcript, b"quad-challenge"); + + let lhs_b = RistrettoPoint::vartime_multiscalar_mul( + quad.sigma_b.iter().chain(std::iter::once(&quad.sigma_bb)), + basis[..ctx.rank] + .iter() + .chain(std::iter::once(&gens.B_blinding)), + ); + if lhs_b != decompress(&quad.r_b)? + projected.p_b * c2 { + return Err(NativeError::Halo2("quad proof: P_B check failed".into())); + } + for k in 0..ctx.rank { + let lhs = + &quad.sigma_u[k] * &RISTRETTO_BASEPOINT_TABLE + gens.B_blinding * quad.sigma_bu[k]; + if lhs != decompress(&quad.r_u[k])? + projected.d_u[k] * c2 { + return Err(NativeError::Halo2("quad proof: u_k check failed".into())); + } + } + let inner: Scalar = quad + .sigma_b + .iter() + .zip(quad.sigma_u.iter()) + .map(|(b, u)| b * u) + .sum(); + // B + omega B~ == C_T0 + c2 C_T1 + c2^2 (s D_Sw + D_Srb) + // with the public-S_w variant adding c2^2 s S_w B on the right. + let lhs = &inner * &RISTRETTO_BASEPOINT_TABLE + gens.B_blinding * quad.omega; + let mut rhs = + decompress(&quad.c_t0)? + decompress(&quad.c_t1)? * c2 + projected.d_srb * (c2 * c2); + match &projected.d_sw { + Some(d_sw) => rhs += d_sw * (c2 * c2 * scale_scalar), + None => rhs += &(c2 * c2 * scale_scalar * projected.public_sw) * &RISTRETTO_BASEPOINT_TABLE, + } + if lhs != rhs { + return Err(NativeError::Halo2( + "quad proof: projected equation failed".into(), + )); + } + + for scalar in linear + .sigma_a + .iter() + .chain(quad.sigma_b.iter()) + .chain(quad.sigma_u.iter()) + { + transcript.append_message(b"sigma-response", scalar.as_bytes()); + } + transcript.append_message(b"sigma-omega", quad.omega.as_bytes()); + + // --- range proofs ------------------------------------------------------- + let mut entries = Vec::new(); + rema.push_entries(&mut entries)?; + u.push_entries(&mut entries)?; + if let Some(class) = &w { + class.push_entries(&mut entries)?; + } + remb.push_entries(&mut entries)?; + if let Some(class) = &remf { + class.push_entries(&mut entries)?; + } + verify_range_bundle(&transcript, entries, &proof.ranges) +} + +// --------------------------------------------------------------------------- +// Adapter secrets cache (per prove call the setup core is re-derived; cache +// it by salt + witness content so a batch of invocations pays one MSM pass) +// --------------------------------------------------------------------------- + +fn cached_adapter_secrets( + input: &AdapterCommitmentInput, + salt: &[u8; 32], +) -> Result, NativeError> { + static CACHE: OnceLock>> = OnceLock::new(); + let cache = CACHE.get_or_init(|| Mutex::new(BoundedCache::new(8))); + let mut hasher = blake3::Hasher::new_keyed(salt); + hasher.update( + serde_json::to_string(input) + .map_err(|e| NativeError::Json(e.to_string()))? + .as_bytes(), + ); + let key = *hasher.finalize().as_bytes(); + // Never hold the cache lock across the (parallel) derivation: rayon + // workers blocked on this mutex while the holder waits for stolen + // subtasks can deadlock the pool. Racing derivations are deterministic, + // so a duplicate insert is harmless. + { + let mut guard = cache.lock().expect("adapter secrets cache poisoned"); + if let Some(hit) = guard.peek(&key) { + return Ok(hit); + } + } + let secrets = adapter_secrets(input, salt)?; + let mut guard = cache.lock().expect("adapter secrets cache poisoned"); + guard.get_or_create(&key, || Ok::<_, NativeError>(secrets)) +} + +// --------------------------------------------------------------------------- +// Public JSON entry points (used by the pyo3 layer and benchmarks) +// --------------------------------------------------------------------------- + +pub fn adapter_setup_json(adapter_json: &str, salt_hex: &str) -> Result { + let input: AdapterCommitmentInput = + serde_json::from_str(adapter_json).map_err(|e| NativeError::Json(e.to_string()))?; + let salt = parse_salt(salt_hex)?; + let setup = adapter_setup(&input, &salt)?; + serde_json::to_string(&setup).map_err(|e| NativeError::Json(e.to_string())) +} + +pub fn adapter_commitment_v4_string( + adapter_json: &str, + salt_hex: &str, +) -> Result { + let input: AdapterCommitmentInput = + serde_json::from_str(adapter_json).map_err(|e| NativeError::Json(e.to_string()))?; + let salt = parse_salt(salt_hex)?; + let secrets = cached_adapter_secrets(&input, &salt)?; + Ok(secrets.commitment.clone()) +} + +pub fn verify_adapter_setup_json(setup_json: &str) -> Result { + let setup: AdapterSetupPub = + serde_json::from_str(setup_json).map_err(|e| NativeError::Json(e.to_string()))?; + match verify_adapter_setup(&setup) { + Ok(()) => Ok(true), + Err(NativeError::Halo2(_)) | Err(NativeError::InvalidDimensions(_)) => Ok(false), + Err(other) => Err(other), + } +} + +pub fn prove_v4_bytes( + statement_json: &str, + witness_json: &str, + salt_hex: &str, +) -> Result, NativeError> { + let salt = parse_salt(salt_hex)?; + prove_invocation(statement_json, witness_json, &salt) +} + +pub fn verify_v4_bytes( + statement_json: &str, + proof: &[u8], + setup_json: &str, +) -> Result { + // Batch verification passes the same multi-MB setup JSON for every + // artifact of a module; cache the parsed setup (and its commitment + // string) by content hash. The hash covers the full JSON, so a + // different setup can never alias a cache entry. + type ParsedSetup = (AdapterSetupPub, String); + static CACHE: OnceLock>> = OnceLock::new(); + let cache = CACHE.get_or_init(|| Mutex::new(BoundedCache::new(16))); + let key = *blake3::hash(setup_json.as_bytes()).as_bytes(); + let cached = { + let mut guard = cache.lock().expect("setup cache poisoned"); + guard.peek(&key) + }; + let entry = match cached { + Some(entry) => entry, + None => { + let setup: AdapterSetupPub = + serde_json::from_str(setup_json).map_err(|e| NativeError::Json(e.to_string()))?; + let commitment = adapter_commitment_string(&setup.core); + let mut guard = cache.lock().expect("setup cache poisoned"); + guard.get_or_create(&key, || Ok::<_, NativeError>((setup, commitment)))? + } + }; + let (setup, commitment) = entry.as_ref(); + verify_invocation_cached(statement_json, proof, setup, commitment) +} + +#[cfg(test)] +mod tests { + use super::*; + use num_integer::Integer; + + /// Deterministic test case builder: exact reference pipeline in BigInt. + #[allow(clippy::too_many_arguments)] + fn make_case( + in_dim: usize, + rank: usize, + out_dim: usize, + scaling_num: i64, + scaling_den: i64, + scale_bits: u32, + value_bits: u32, + intermediate_bits: u32, + seed: u64, + ) -> (String, String, String, String) { + let fixed_point = FixedPointConfig { + scale_bits, + value_bits, + intermediate_bits, + }; + let mut state = seed; + let magnitude = 1i64 << scale_bits.min(20); + let mut next = |bound: i64| -> i64 { + state = state + .wrapping_mul(6364136223846793005) + .wrapping_add(1442695040888963407); + ((state >> 16) as i64) % (2 * bound + 1) - bound + }; + let a: Vec> = (0..rank) + .map(|_| (0..in_dim).map(|_| next(magnitude)).collect()) + .collect(); + let b: Vec> = (0..out_dim) + .map(|_| (0..rank).map(|_| next(magnitude)).collect()) + .collect(); + let x: Vec = (0..in_dim).map(|_| next(magnitude)).collect(); + + let scale = BigInt::from(1) << scale_bits; + let div_round = + |n: &BigInt, d: &BigInt| -> BigInt { (n + d / BigInt::from(2u8)).div_floor(d) }; + let u: Vec = a + .iter() + .map(|row| { + let raw: BigInt = row + .iter() + .zip(x.iter()) + .map(|(w, xi)| BigInt::from(*w) * BigInt::from(*xi)) + .sum(); + div_round(&raw, &scale) + }) + .collect(); + let delta: Vec = b + .iter() + .map(|row| { + let raw: BigInt = row + .iter() + .zip(u.iter()) + .map(|(w, v)| BigInt::from(*w) * v) + .sum(); + let w = div_round(&raw, &scale); + let scaled = w * BigInt::from(scaling_num); + i64::try_from(div_round(&scaled, &BigInt::from(scaling_den))).expect("delta fits") + }) + .collect(); + + let adapter_input = AdapterCommitmentInput { + schema_version: SIGMA_SCHEMA_VERSION, + in_dim, + rank, + out_dim, + fixed_point: fixed_point.clone(), + scaling_num, + scaling_den, + a: a.clone(), + b: b.clone(), + }; + let adapter_json = serde_json::to_string(&adapter_input).expect("adapter json"); + let salt_hex = "ab".repeat(32); + let setup_json = adapter_setup_json(&adapter_json, &salt_hex).expect("setup"); + let commitment = + adapter_commitment_v4_string(&adapter_json, &salt_hex).expect("commitment"); + + let statement = NativeStatement { + x, + delta, + fixed_point, + rank, + scaling_num, + scaling_den, + adapter_commitment: commitment, + statement_digest: "ab".repeat(32), + }; + let witness = NativeWitness { a, b }; + ( + serde_json::to_string(&statement).expect("statement json"), + serde_json::to_string(&witness).expect("witness json"), + setup_json, + salt_hex, + ) + } + + fn roundtrip(case: (String, String, String, String)) { + let (statement, witness, setup, salt) = case; + assert!(verify_adapter_setup_json(&setup).expect("setup verify call")); + let proof = prove_v4_bytes(&statement, &witness, &salt).expect("prove"); + assert!(verify_v4_bytes(&statement, &proof, &setup).expect("verify call")); + } + + #[test] + fn roundtrip_trivial_scaling() { + roundtrip(make_case(4, 2, 3, 1, 1, 20, 63, 127, 7)); + } + + #[test] + fn roundtrip_nontrivial_scaling() { + roundtrip(make_case(5, 2, 4, 3, 2, 8, 24, 60, 11)); + } + + #[test] + fn roundtrip_negative_scaling_num() { + roundtrip(make_case(3, 1, 2, -7, 4, 6, 20, 48, 13)); + } + + #[test] + fn roundtrip_single_limb_intermediate() { + roundtrip(make_case(4, 2, 2, 1, 1, 4, 16, 40, 17)); + } + + #[test] + fn roundtrip_rank_one_min_dims() { + roundtrip(make_case(1, 1, 1, 1, 1, 2, 8, 24, 19)); + } + + #[test] + fn tampered_delta_rejected() { + let (statement, witness, setup, salt) = make_case(4, 2, 3, 1, 1, 20, 63, 127, 23); + let proof = prove_v4_bytes(&statement, &witness, &salt).expect("prove"); + let mut tampered: NativeStatement = serde_json::from_str(&statement).unwrap(); + tampered.delta[0] += 1; + let tampered_json = serde_json::to_string(&tampered).unwrap(); + assert!(!verify_v4_bytes(&tampered_json, &proof, &setup).expect("verify call")); + } + + #[test] + fn tampered_x_rejected() { + let (statement, witness, setup, salt) = make_case(4, 2, 3, 3, 2, 10, 32, 80, 29); + let proof = prove_v4_bytes(&statement, &witness, &salt).expect("prove"); + let mut tampered: NativeStatement = serde_json::from_str(&statement).unwrap(); + tampered.x[1] += 1; + let tampered_json = serde_json::to_string(&tampered).unwrap(); + assert!(!verify_v4_bytes(&tampered_json, &proof, &setup).expect("verify call")); + } + + #[test] + fn tampered_proof_bytes_rejected() { + let (statement, witness, setup, salt) = make_case(4, 2, 3, 1, 1, 20, 63, 127, 31); + let mut proof = prove_v4_bytes(&statement, &witness, &salt).expect("prove"); + let index = proof.len() / 2; + proof[index] ^= 0x40; + // Either a decode error (treated as false) or a failed check. + assert!(!verify_v4_bytes(&statement, &proof, &setup).unwrap_or(false)); + } + + #[test] + fn wrong_adapter_rejected_at_prove_and_verify() { + let (statement, witness, _setup, salt) = make_case(4, 2, 3, 1, 1, 20, 63, 127, 37); + // A different adapter (different seed) has a different commitment. + let (other_statement, other_witness, other_setup, other_salt) = + make_case(4, 2, 3, 1, 1, 20, 63, 127, 41); + // Proving with a witness that does not match the statement commitment + // must fail outright. + assert!(prove_v4_bytes(&statement, &other_witness, &other_salt).is_err()); + // A valid proof for one statement must not verify against another + // adapter's setup. + let proof = prove_v4_bytes(&other_statement, &other_witness, &other_salt).unwrap(); + assert!(verify_v4_bytes(&other_statement, &proof, &other_setup).unwrap()); + let (_, _, setup, _) = make_case(4, 2, 3, 1, 1, 20, 63, 127, 37); + assert!(!verify_v4_bytes(&other_statement, &proof, &setup).unwrap()); + let _ = (witness, salt); + } + + #[test] + fn wrong_salt_changes_commitment() { + let (statement, witness, _setup, _salt) = make_case(4, 2, 3, 1, 1, 20, 63, 127, 43); + let wrong_salt = "cd".repeat(32); + assert!(prove_v4_bytes(&statement, &witness, &wrong_salt).is_err()); + } + + #[test] + fn tampered_adapter_setup_rejected() { + let (_, _, setup, _) = make_case(3, 1, 2, 1, 1, 8, 24, 56, 47); + let mut parsed: AdapterSetupPub = serde_json::from_str(&setup).unwrap(); + // Flip one weight commitment: links/ranges must fail. + parsed.core.weight_commitments[0][5] ^= 0x11; + let tampered = serde_json::to_string(&parsed).unwrap(); + assert!(!verify_adapter_setup_json(&tampered).unwrap_or(false)); + } + + #[test] + fn statement_digest_must_be_well_formed() { + let (statement, witness, _setup, salt) = make_case(3, 1, 2, 1, 1, 8, 24, 56, 53); + let mut parsed: NativeStatement = serde_json::from_str(&statement).unwrap(); + parsed.statement_digest = "zz".repeat(32); + let bad = serde_json::to_string(&parsed).unwrap(); + assert!(prove_v4_bytes(&bad, &witness, &salt).is_err()); + } + + #[test] + fn proof_is_bound_to_statement_digest() { + let (statement, witness, setup, salt) = make_case(4, 2, 3, 1, 1, 20, 63, 127, 59); + let proof = prove_v4_bytes(&statement, &witness, &salt).expect("prove"); + let mut parsed: NativeStatement = serde_json::from_str(&statement).unwrap(); + parsed.statement_digest = "cd".repeat(32); + let other = serde_json::to_string(&parsed).unwrap(); + assert!(!verify_v4_bytes(&other, &proof, &setup).expect("verify call")); + } + + #[test] + fn out_of_range_public_values_rejected() { + let (statement, witness, _setup, salt) = make_case(3, 1, 2, 1, 1, 8, 24, 56, 61); + let mut parsed: NativeStatement = serde_json::from_str(&statement).unwrap(); + parsed.x[0] = 1 << 40; // exceeds 24-bit value bound + let bad = serde_json::to_string(&parsed).unwrap(); + assert!(prove_v4_bytes(&bad, &witness, &salt).is_err()); + } +} + +#[cfg(test)] +mod security_tests { + use super::*; + + fn adapter(in_dim: usize, rank: usize, out_dim: usize, tweak: i64) -> AdapterCommitmentInput { + AdapterCommitmentInput { + schema_version: SIGMA_SCHEMA_VERSION, + in_dim, + rank, + out_dim, + fixed_point: FixedPointConfig { + scale_bits: 4, + value_bits: 16, + intermediate_bits: 40, + }, + scaling_num: 1, + scaling_den: 1, + a: (0..rank) + .map(|k| { + (0..in_dim) + .map(|j| (k * in_dim + j) as i64 + tweak) + .collect() + }) + .collect(), + b: (0..out_dim) + .map(|j| (0..rank).map(|k| (j * rank + k) as i64).collect()) + .collect(), + } + } + + #[test] + fn same_shape_adapters_share_no_blindings() { + // Two same-shaped adapters under one salt differ only in the A + // matrix. If blindings depended only on (salt, index), every + // commitment at an unchanged index would collide, and the + // difference at a changed index would open to (w1 - w2) * B with + // zero blinding, leaking exact weight differences from the public + // manifest. + let salt = [9u8; 32]; + let first = adapter(3, 2, 3, 0); + let second = adapter(3, 2, 3, 1); // shifts every A entry by 1 + let s1 = adapter_secrets(&first, &salt).unwrap(); + let s2 = adapter_secrets(&second, &salt).unwrap(); + + // B matrices are identical across the two adapters; their + // commitments must still differ at every index (independent + // blindings), and so must every row commitment. + let a_values = 2 * 3; + for (c1, c2) in s1.core.weight_commitments[a_values..] + .iter() + .zip(s2.core.weight_commitments[a_values..].iter()) + { + assert_ne!( + c1, c2, + "identical weights must not produce equal commitments" + ); + } + for (c1, c2) in s1 + .core + .row_commitments_b + .iter() + .zip(s2.core.row_commitments_b.iter()) + { + assert_ne!(c1, c2); + } + // And the difference at a changed A index must not open to + // delta_w * B (which a manifest holder could brute-force). + let delta_w = Scalar::one(); // w2 - w1 = 1 at every A index + let c1 = decompress(&s1.core.weight_commitments[0]).unwrap(); + let c2 = decompress(&s2.core.weight_commitments[0]).unwrap(); + assert_ne!(c2 - c1, &delta_w * &RISTRETTO_BASEPOINT_TABLE); + } + + #[test] + fn oversized_dimensions_rejected() { + let statement = NativeStatement { + x: vec![0], + delta: vec![0], + fixed_point: FixedPointConfig { + scale_bits: 4, + value_bits: 16, + intermediate_bits: 40, + }, + rank: MAX_RANK + 1, + scaling_num: 1, + scaling_den: 1, + adapter_commitment: "0".repeat(64), + statement_digest: "ab".repeat(32), + }; + let json = serde_json::to_string(&statement).unwrap(); + assert!(statement_context(&json).is_err()); + } + + #[test] + fn limb_plan_enforces_slack_invariant() { + // Single-limb widths: always exact, any value admitted. + assert!(limb_plan(&BigInt::from(12345)).is_ok()); + // Quotient-cap form 2^k - 2: slack exactly +1, admitted. + assert!(limb_plan(&((BigInt::from(1) << 127) - 2)).is_ok()); + // Sloppy multi-limb width (2^70): low limbs would admit values up + // to ~2^70 + 2^64 above the cap; must be rejected. + assert!(limb_plan(&(BigInt::from(1) << 70)).is_err()); + } +} diff --git a/src/zklora/lora_contributor_mpi/__init__.py b/src/zklora/lora_contributor_mpi/__init__.py index 3475d15..e33bc95 100644 --- a/src/zklora/lora_contributor_mpi/__init__.py +++ b/src/zklora/lora_contributor_mpi/__init__.py @@ -51,6 +51,23 @@ def __init__( self.out_dir = out_dir self.fixed_point = fixed_point or FixedPointConfig() os.makedirs(self.out_dir, exist_ok=True) + # Persist the adapter commitment salt next to the proof artifacts so + # commitments stay stable across server restarts (the pinned manifest + # must keep matching future proofs). The salt is contributor-secret; + # it never enters manifests or proofs. The env var is process-wide: + # when several LoRAServers share one process, they share the first + # server's salt file, which is safe for hiding (blinding keys are + # derived per adapter content) but means a server restarted standalone + # must keep pointing at the SAME salt file for its pinned manifest to + # stay valid -- hence the warning instead of silently rebinding. + own_salt_file = os.path.join(self.out_dir, "adapter-commitment.salt") + configured = os.environ.setdefault("ZKLORA_ADAPTER_SALT_FILE", own_salt_file) + if os.path.abspath(configured) != os.path.abspath(own_salt_file): + print( + f"[A-Server] WARNING: adapter salt file is {configured} (shared), " + f"not {own_salt_file}; keep ZKLORA_ADAPTER_SALT_FILE stable across " + "restarts or previously pinned manifests will not match new proofs." + ) # 1) Load model, disable cache => no 'past_key_values' base_model = AutoModelForCausalLM.from_pretrained(base_model_name) @@ -76,6 +93,10 @@ def __init__( self.session_data: dict[str, list[InvocationWitness]] = {} self._invocation_counts: dict[tuple[str, str], int] = {} + # INVARIANT: last_scaling / last_q_delta are single shared snapshot + # slots, written by apply_lora and read by the socket layer. They are + # only safe because LoRAServerSocket serializes every request under + # one _server_lock; any reader added outside that lock would race. self.last_scaling: tuple[int, int] = (1, 1) self.last_q_delta: list[list[int]] = [] self._module_cache: dict[str, dict] = {} diff --git a/src/zklora/proof_contract.py b/src/zklora/proof_contract.py index 5fe26ec..a2c514d 100644 --- a/src/zklora/proof_contract.py +++ b/src/zklora/proof_contract.py @@ -5,6 +5,7 @@ import json import os import re +import secrets import threading from dataclasses import asdict, dataclass from decimal import Decimal, ROUND_HALF_UP @@ -12,14 +13,26 @@ from typing import Any, Iterable -# v3: lookup-based range checks in the native circuit (same statement format, -# same adapter commitment scheme; proofs/vk are not compatible with v2). -BACKEND_ID = "zklora-halo2-v3" -SCHEMA_VERSION = 2 +# v4: commit-and-prove sigma backend over ristretto255. The adapter is bound +# once per manifest through salted Pedersen row commitments plus a one-time +# exact weight range proof; each invocation proof checks the same exact +# quantized LoRA relation as the v3 circuit through Fiat-Shamir random +# projections and aggregated range proofs, so per-proof work no longer scales +# with rank*in + out*rank weights. Statement semantics, transcript binding, +# and the verifier trust boundary are unchanged. v3 halo2 artifacts (schema +# version 2) remain verifiable through the legacy path below. +BACKEND_ID = "zklora-sigma-v4" +SCHEMA_VERSION = 3 +PROOF_KIND = "native-sigma-v4" COMMITMENT_SCHEME = "sha256-canonical-lora-matrices-v1" -ADAPTER_COMMITMENT_SCHEME = "poseidon-pasta-fp-adapter-v1" +ADAPTER_COMMITMENT_SCHEME = "pedersen-ristretto255-salted-v1" INVOCATION_STRATEGY = "one-proof-per-module-invocation-v1" +LEGACY_BACKEND_ID = "zklora-halo2-v3" +LEGACY_SCHEMA_VERSION = 2 +LEGACY_PROOF_KIND = "native-halo2" +LEGACY_ADAPTER_COMMITMENT_SCHEME = "poseidon-pasta-fp-adapter-v1" + class ProofContractError(ValueError): """Raised when a transcript, witness, or proof artifact violates the contract.""" @@ -365,11 +378,101 @@ def adapter_commitment_payload( _ADAPTER_COMMITMENT_CACHE_MAX = 64 _ADAPTER_COMMITMENT_LOCK = threading.Lock() +_ADAPTER_SALT_LOCK = threading.Lock() +# Salts keyed by resolved salt-file path (None = process-ephemeral). Keying +# by path instead of a single process global means a changed +# ZKLORA_ADAPTER_SALT_FILE takes effect at the next call rather than being +# silently shadowed by whichever caller touched the salt first. +_ADAPTER_SALTS: dict[str | None, str] = {} + + +def _validate_salt(value: str, origin: str) -> str: + if len(value) != 64 or any(c not in "0123456789abcdefABCDEF" for c in value): + raise ProofContractError( + f"adapter salt from {origin} must be 64 hex characters" + ) + return value.lower() + + +def adapter_salt() -> str: + """Secret salt for the contributor's adapter commitments (64 hex chars). + + The salt only affects hiding, never binding: commitments stay binding + under discrete log even if the salt leaks, and per-adapter blinding keys + are derived from the salt together with the full adapter content, so two + adapters never share blindings even under one salt. It must stay stable + across manifest writing and proving so the pinned commitment matches the + proofs; set ZKLORA_ADAPTER_SALT_FILE to persist it across processes + (LoRAServer defaults this to a file in its output directory). The salt + is contributor secret material: it never appears in manifests or proof + artifacts. + """ + path = os.environ.get("ZKLORA_ADAPTER_SALT_FILE") or None + key = str(Path(path).resolve()) if path else None + with _ADAPTER_SALT_LOCK: + cached = _ADAPTER_SALTS.get(key) + if cached is not None: + return cached + if key is None: + value = secrets.token_hex(32) + else: + value = _read_or_create_salt_file(key) + _ADAPTER_SALTS[key] = value + return value + + +def _read_or_create_salt_file(path: str) -> str: + """Atomically create-or-read the salt file. + + O_CREAT|O_EXCL makes exactly one creator win a cross-process race; losers + fall through to reading the winner's file. The winner writes to a + same-directory temp file and os.replace()s it into place so a reader can + never observe a partial salt. + """ + import time + + for attempt in range(20): + try: + fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) + except FileExistsError: + try: + value = Path(path).read_text(encoding="utf-8").strip() + except OSError as exc: + raise ProofContractError( + f"cannot read adapter salt file {path}: {exc}" + ) from exc + if value: + return _validate_salt(value, path) + # Lost a race against the creator between its O_EXCL placeholder + # and the atomic replace; wait briefly and retry. + time.sleep(0.05 * (attempt + 1)) + continue + # We are the creator: write atomically via temp file + replace, then + # drop the (empty) placeholder created by O_EXCL. + os.close(fd) + value = secrets.token_hex(32) + tmp = f"{path}.tmp.{os.getpid()}" + tmp_fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600) + with os.fdopen(tmp_fd, "w", encoding="utf-8") as f: + f.write(value + "\n") + os.replace(tmp, path) + return value + raise ProofContractError(f"adapter salt file {path} is empty") + def _freeze_matrix(matrix: list[list[int]]) -> tuple[tuple[int, ...], ...]: return tuple(tuple(int(v) for v in row) for row in matrix) +def _require_native(): + native = _native_module() + if native is None: + raise ProofContractError( + "native zkLoRA prover is unavailable; build/install zklora with maturin" + ) + return native + + def adapter_commitment( a: list[list[int]], b: list[list[int]], @@ -379,15 +482,13 @@ def adapter_commitment( ) -> str: # The commitment is recomputed for every invocation statement of a module, # over the same (large) adapter matrices. Content-keyed memoisation keeps - # the Poseidon chain to one evaluation per adapter. The backend identity is - # part of the key so monkeypatched/fake backends never share entries. - native = _native_module() - if native is None: - raise ProofContractError( - "native Halo2 prover is unavailable; build/install zklora with maturin" - ) + # the commitment derivation to one pass per adapter. The backend identity + # is part of the key so monkeypatched/fake backends never share entries. + native = _require_native() + salt = adapter_salt() key = ( id(native), + salt, _freeze_matrix(a), _freeze_matrix(b), int(scaling_num), @@ -399,10 +500,11 @@ def adapter_commitment( if cached is not None: return cached value = str( - native.adapter_commitment( + native.adapter_commitment_v4( canonical_json( adapter_commitment_payload(a, b, scaling_num, scaling_den, fixed_point) - ) + ), + salt, ) ) with _ADAPTER_COMMITMENT_LOCK: @@ -412,14 +514,37 @@ def adapter_commitment( return value +def adapter_setup( + a: list[list[int]], + b: list[list[int]], + scaling_num: int, + scaling_den: int, + fixed_point: FixedPointConfig, +) -> dict[str, Any]: + """Public adapter setup for the verifier: salted Pedersen row commitments + plus the one-time proofs that every committed weight lies in the exact + value-bound interval. Ships inside the pinned adapter manifest; contains + no weight or salt material.""" + native = _require_native() + return json.loads( + native.adapter_setup_v4( + canonical_json( + adapter_commitment_payload(a, b, scaling_num, scaling_den, fixed_point) + ), + adapter_salt(), + ) + ) + + def circuit_id( in_dim: int, rank: int, out_dim: int, fixed_point: FixedPointConfig, + backend: str = BACKEND_ID, ) -> str: payload = { - "backend": BACKEND_ID, + "backend": backend, "commitment_scheme": COMMITMENT_SCHEME, "invocation_strategy": INVOCATION_STRATEGY, "in_dim": in_dim, @@ -430,8 +555,8 @@ def circuit_id( return digest_hex(payload) -def vk_fingerprint(circuit: str) -> str: - return digest_hex({"backend": BACKEND_ID, "vk_for_circuit": circuit}) +def vk_fingerprint(circuit: str, backend: str = BACKEND_ID) -> str: + return digest_hex({"backend": backend, "vk_for_circuit": circuit}) def statement_from_witness(witness: InvocationWitness) -> dict[str, Any]: @@ -580,15 +705,11 @@ def write_invocation_artifacts( statement_path = Path(f"{prefix}.zklora.statement.json") meta_path = Path(f"{prefix}.zklora.meta.json") - native = _native_module() - if native is None: - raise ProofContractError( - "native Halo2 prover is unavailable; build/install zklora with maturin" - ) - proof_bytes = native.prove( - _native_statement_json(statement), _native_witness_json(witness) + native = _require_native() + proof_bytes = native.prove_v4( + _native_statement_json(statement), _native_witness_json(witness), adapter_salt() ) - proof_kind = "native-halo2" + proof_kind = PROOF_KIND vk = { "schema_version": SCHEMA_VERSION, "backend": BACKEND_ID, @@ -702,6 +823,12 @@ def adapter_manifest_entry( a, b, int(scaling_num), int(scaling_den), fixed_point ), }, + # Public verification material: row commitments plus the one-time + # weight range/link proofs. The commitment value above is the SHA-256 + # of the deterministic core, so pinning the manifest pins the setup. + "adapter_setup": adapter_setup( + a, b, int(scaling_num), int(scaling_den), fixed_point + ), } @@ -764,9 +891,20 @@ def verify_artifact_set( ) -> None: statement = load_json(statement_path) if ( - statement.get("schema_version") != SCHEMA_VERSION - or statement.get("backend") != BACKEND_ID + statement.get("schema_version") == LEGACY_SCHEMA_VERSION + and statement.get("backend") == LEGACY_BACKEND_ID ): + backend = LEGACY_BACKEND_ID + proof_kind = LEGACY_PROOF_KIND + adapter_scheme = LEGACY_ADAPTER_COMMITMENT_SCHEME + elif ( + statement.get("schema_version") == SCHEMA_VERSION + and statement.get("backend") == BACKEND_ID + ): + backend = BACKEND_ID + proof_kind = PROOF_KIND + adapter_scheme = ADAPTER_COMMITMENT_SCHEME + else: raise ProofContractError("unsupported proof artifact schema/backend") if statement.get("statement_digest") != digest_hex( _statement_digest_payload(statement) @@ -809,19 +947,23 @@ def verify_artifact_set( int(statement["adapter_metadata"].get("rank", 0)), len(statement["delta"]), FixedPointConfig(**statement["fixed_point"]), + backend=backend, ) if statement["circuit_id"] != expected_circuit: raise ProofContractError("statement circuit_id does not match expected circuit") - if statement["vk_fingerprint"] != vk_fingerprint(expected_circuit): + if statement["vk_fingerprint"] != vk_fingerprint(expected_circuit, backend=backend): raise ProofContractError( "statement vk_fingerprint does not match expected circuit" ) adapter = statement.get("adapter_commitment", {}) - if adapter.get("scheme") != ADAPTER_COMMITMENT_SCHEME: + if adapter.get("scheme") != adapter_scheme: raise ProofContractError("statement adapter commitment scheme mismatch") vk = load_json(vk_path) - if vk.get("schema_version") != SCHEMA_VERSION or vk.get("backend") != BACKEND_ID: + if ( + vk.get("schema_version") != statement["schema_version"] + or vk.get("backend") != backend + ): raise ProofContractError("unsupported verification key schema/backend") if vk.get("circuit_id") != expected_circuit: raise ProofContractError("verification key circuit_id mismatch") @@ -831,17 +973,32 @@ def verify_artifact_set( meta = load_json(meta_path) proof_bytes = proof_path.read_bytes() if ( - meta.get("schema_version") != SCHEMA_VERSION - or meta.get("backend") != BACKEND_ID + meta.get("schema_version") != statement["schema_version"] + or meta.get("backend") != backend ): raise ProofContractError("unsupported metadata schema/backend") - if meta.get("proof_kind") != "native-halo2": - raise ProofContractError("unsupported proof kind; expected native-halo2") + if meta.get("proof_kind") != proof_kind: + raise ProofContractError(f"unsupported proof kind; expected {proof_kind}") native = _native_module() if native is None: - raise ProofContractError("native Halo2 verifier is unavailable") - if not native.verify(_native_statement_json(statement), proof_bytes): - raise ProofContractError("proof bytes failed native Halo2 verification") + raise ProofContractError("native zkLoRA verifier is unavailable") + if backend == BACKEND_ID: + setup = expected_adapter.get("adapter_setup") + if not isinstance(setup, dict): + raise ProofContractError( + "expected adapter manifest entry lacks adapter_setup for sigma-v4" + ) + # verify_v4 checks that SHA-256 of the setup core equals the + # statement's pinned adapter commitment, verifies the one-time + # weight range/link proofs (cached per adapter), and verifies the + # invocation proof against the row commitments. + if not native.verify_v4( + _native_statement_json(statement), proof_bytes, canonical_json(setup) + ): + raise ProofContractError("proof bytes failed native sigma-v4 verification") + else: + if not native.verify(_native_statement_json(statement), proof_bytes): + raise ProofContractError("proof bytes failed native Halo2 verification") if meta.get("statement_digest") != statement["statement_digest"]: raise ProofContractError("metadata statement digest mismatch") diff --git a/src/zklora/zk_proof_generator.py b/src/zklora/zk_proof_generator.py index d341ef6..2da3a1b 100644 --- a/src/zklora/zk_proof_generator.py +++ b/src/zklora/zk_proof_generator.py @@ -40,6 +40,16 @@ def generate_proofs( raise ProofContractError( "native zkLoRA proof generation requires captured invocation records" ) + # Artifact paths are derived from (session_id, module_name, + # invocation_index); duplicate keys would race and silently overwrite + # each other's files during concurrent generation, so reject them here + # rather than relying on the verifier's coverage check. + seen_keys: set[tuple[str, str, int]] = set() + for record in record_list: + key = (record.session_id, record.module_name, int(record.invocation_index)) + if key in seen_keys: + raise ProofContractError(f"duplicate invocation record for {key}") + seen_keys.add(key) def _generate(record: InvocationWitness) -> None: write_invocation_artifacts(output_dir, record) diff --git a/tests/test_legacy_and_hardening.py b/tests/test_legacy_and_hardening.py new file mode 100644 index 0000000..c12f109 --- /dev/null +++ b/tests/test_legacy_and_hardening.py @@ -0,0 +1,235 @@ +"""Legacy v3 (halo2, schema 2) artifacts must keep verifying. + +This test reconstructs proof artifacts byte-for-byte the way `main` +(commit 8b53724, backend `zklora-halo2-v3`, schema_version 2) wrote them -- +poseidon adapter commitment, halo2 proof bytes, legacy circuit/vk +fingerprints -- and checks that the current verifier accepts them through +the legacy path while still rejecting tampering. It needs the built native +extension for the halo2 prover. +""" + +import hashlib +import importlib +import os +from dataclasses import asdict +from pathlib import Path + +import pytest + +if os.environ.get("ZKLORA_REQUIRE_NATIVE_EXTENSION") == "1": + native = importlib.import_module("zklora._native_prover") +else: + native = pytest.importorskip( + "zklora._native_prover", + reason="native PyO3 extension is not built in this environment", + ) + +from zklora.proof_contract import ( + COMMITMENT_SCHEME, + LEGACY_ADAPTER_COMMITMENT_SCHEME, + LEGACY_BACKEND_ID, + LEGACY_PROOF_KIND, + LEGACY_SCHEMA_VERSION, + FixedPointConfig, + ProofContractError, + artifact_prefix, + canonical_json, + circuit_id, + compute_delta_quantized, + digest_hex, + lora_commitment, + transcript_entry_from_statement, + verify_artifacts, + vk_fingerprint, +) + + +def _legacy_adapter_commitment(a, b, scaling_num, scaling_den, fp): + # main's adapter_commitment_payload used SCHEMA_VERSION = 2 and fed the + # canonical JSON to the native poseidon hasher. + payload = { + "schema_version": LEGACY_SCHEMA_VERSION, + "in_dim": len(a[0]), + "rank": len(a), + "out_dim": len(b), + "fixed_point": asdict(fp), + "scaling_num": int(scaling_num), + "scaling_den": int(scaling_den), + "a": a, + "b": b, + } + return str(native.adapter_commitment(canonical_json(payload))) + + +def _write_legacy_artifacts(output_dir, fp): + """Replicates main's statement_from_witness + write_invocation_artifacts.""" + a = [[2, -1]] + b = [[3], [-2]] + x = [4, 5] + delta = compute_delta_quantized(a, b, x, 1, 2, fp) + adapter_metadata = {"rank": 1, "in_dim": 2, "out_dim": 2} + circuit = circuit_id(2, 1, 2, fp, backend=LEGACY_BACKEND_ID) + statement = { + "schema_version": LEGACY_SCHEMA_VERSION, + "backend": LEGACY_BACKEND_ID, + "session_id": "legacy-session", + "module_name": "transformer.h.0.attn.c_attn", + "invocation_index": 0, + "input_shape": [2], + "output_shape": [2], + "x": x, + "delta": delta, + "scaling": {"num": 1, "den": 2}, + "fixed_point": asdict(fp), + "adapter_metadata": adapter_metadata, + "lora_commitment": lora_commitment(a, b, adapter_metadata), + "adapter_commitment": { + "scheme": LEGACY_ADAPTER_COMMITMENT_SCHEME, + "value": _legacy_adapter_commitment(a, b, 1, 2, fp), + }, + "circuit_id": circuit, + "vk_fingerprint": vk_fingerprint(circuit, backend=LEGACY_BACKEND_ID), + "commitment_scheme": COMMITMENT_SCHEME, + "invocation_strategy": "one-proof-per-module-invocation-v1", + } + statement["statement_digest"] = digest_hex( + {k: v for k, v in statement.items() if k != "statement_digest"} + ) + + native_statement = canonical_json( + { + "x": statement["x"], + "delta": statement["delta"], + "fixed_point": statement["fixed_point"], + "rank": 1, + "scaling_num": 1, + "scaling_den": 2, + "adapter_commitment": statement["adapter_commitment"]["value"], + "statement_digest": statement["statement_digest"], + } + ) + proof_bytes = native.prove(native_statement, canonical_json({"a": a, "b": b})) + + prefix = artifact_prefix(output_dir, statement) + vk = { + "schema_version": LEGACY_SCHEMA_VERSION, + "backend": LEGACY_BACKEND_ID, + "circuit_id": circuit, + "vk_fingerprint": statement["vk_fingerprint"], + } + pk = { + "schema_version": LEGACY_SCHEMA_VERSION, + "backend": LEGACY_BACKEND_ID, + "circuit_id": circuit, + "pk_fingerprint": digest_hex({"pk_for_circuit": circuit}), + } + meta = { + "schema_version": LEGACY_SCHEMA_VERSION, + "backend": LEGACY_BACKEND_ID, + "proof_file": f"{prefix.name}.zklora.proof", + "statement_file": f"{prefix.name}.zklora.statement.json", + "vk_file": f"{prefix.name}.zklora.vk", + "pk_file": f"{prefix.name}.zklora.pk", + "statement_digest": statement["statement_digest"], + "statement_file_digest": digest_hex(statement), + "proof_digest": hashlib.sha256(proof_bytes).hexdigest(), + "proof_kind": LEGACY_PROOF_KIND, + "vk_digest": digest_hex(vk), + "circuit_id": circuit, + "vk_fingerprint": statement["vk_fingerprint"], + } + Path(f"{prefix}.zklora.proof").write_bytes(proof_bytes) + Path(f"{prefix}.zklora.vk").write_text(canonical_json(vk) + "\n", encoding="utf-8") + Path(f"{prefix}.zklora.pk").write_text(canonical_json(pk) + "\n", encoding="utf-8") + Path(f"{prefix}.zklora.statement.json").write_text( + canonical_json(statement) + "\n", encoding="utf-8" + ) + Path(f"{prefix}.zklora.meta.json").write_text( + canonical_json(meta) + "\n", encoding="utf-8" + ) + + manifest_entry = { + "module_name": statement["module_name"], + "rank": 1, + "in_dim": 2, + "out_dim": 2, + "fixed_point": asdict(fp), + "scaling": {"num": 1, "den": 2}, + "adapter_commitment": statement["adapter_commitment"], + } + return statement, manifest_entry, prefix + + +def test_legacy_v3_artifacts_still_verify_and_reject_tampering(tmp_path): + fp = FixedPointConfig(scale_bits=0, value_bits=8, intermediate_bits=16) + statement, manifest_entry, prefix = _write_legacy_artifacts(tmp_path, fp) + transcript = [transcript_entry_from_statement(statement)] + + total_time, count = verify_artifacts(tmp_path, transcript, [manifest_entry]) + assert count == 1 + assert total_time >= 0 + + Path(f"{prefix}.zklora.proof").write_bytes(b"tampered-legacy-proof") + with pytest.raises(ProofContractError, match="proof bytes"): + verify_artifacts(tmp_path, transcript, [manifest_entry]) + + +def test_adapter_salt_is_path_keyed_and_atomic(tmp_path, monkeypatch): + import concurrent.futures + + import zklora.proof_contract as pc + + monkeypatch.setattr(pc, "_ADAPTER_SALTS", {}) + salt_file = tmp_path / "salt-a" + monkeypatch.setenv("ZKLORA_ADAPTER_SALT_FILE", str(salt_file)) + with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool: + salts = {pool.submit(pc.adapter_salt).result() for _ in range(16)} + assert len(salts) == 1 + persisted = salt_file.read_text().strip() + assert persisted == next(iter(salts)) + + # A different configured path yields its own salt at the next call + # instead of being shadowed by the first one touched. + other_file = tmp_path / "salt-b" + monkeypatch.setenv("ZKLORA_ADAPTER_SALT_FILE", str(other_file)) + other = pc.adapter_salt() + assert other != persisted + monkeypatch.setenv("ZKLORA_ADAPTER_SALT_FILE", str(salt_file)) + assert pc.adapter_salt() == persisted + + +def test_generate_proofs_rejects_duplicate_invocation_keys(tmp_path, monkeypatch): + import zklora.proof_contract as pc + from zklora.proof_contract import InvocationWitness, quantize_nested + from zklora.zk_proof_generator import generate_proofs + + class FakeNative: + def adapter_commitment_v4(self, adapter_json, salt): + return hashlib.sha256(f"{salt}|{adapter_json}".encode()).hexdigest() + + def prove_v4(self, statement_json, witness_json, salt): + return b"proof" + + monkeypatch.setattr(pc, "_native_module", lambda: FakeNative()) + fp = FixedPointConfig(scale_bits=2, value_bits=16, intermediate_bits=32) + a = quantize_nested([[1.0, 1.0]], fp) + b = quantize_nested([[1.0], [1.0]], fp) + x = [4, 4] + delta = compute_delta_quantized(a, b, x, 1, 1, fp) + witness = InvocationWitness( + session_id="s", + module_name="m.c_attn", + invocation_index=0, + input_shape=[2], + output_shape=[2], + x=x, + delta=delta, + a=a, + b=b, + scaling_num=1, + scaling_den=1, + adapter_metadata={"rank": 1}, + fixed_point=fp, + ) + with pytest.raises(ProofContractError, match="duplicate invocation record"): + generate_proofs(records=[witness, witness], output_dir=str(tmp_path)) diff --git a/tests/test_zklora_proof_contract.py b/tests/test_zklora_proof_contract.py index b7046c2..2e50a9f 100644 --- a/tests/test_zklora_proof_contract.py +++ b/tests/test_zklora_proof_contract.py @@ -1,3 +1,4 @@ +import hashlib import importlib import json import sys @@ -26,15 +27,23 @@ class FakeNative: - def adapter_commitment(self, adapter_json): - return str(abs(hash(adapter_json))) + def adapter_commitment_v4(self, adapter_json, salt): + return hashlib.sha256(f"{salt}|{adapter_json}".encode()).hexdigest() - def prove(self, statement_json, witness_json): + def adapter_setup_v4(self, adapter_json, salt): + return json.dumps( + {"core": {"digest": self.adapter_commitment_v4(adapter_json, salt)}} + ) + + def prove_v4(self, statement_json, witness_json, salt): return f"{statement_json}|{witness_json}".encode() - def verify(self, statement_json, proof): + def verify_v4(self, statement_json, proof, setup_json): return proof.startswith(statement_json.encode() + b"|") + def verify_adapter_setup_v4(self, setup_json): + return True + @pytest.fixture(autouse=True) def fake_native(monkeypatch): @@ -100,7 +109,7 @@ def test_statement_round_trip_is_canonical_and_transcript_bound(tmp_path): meta = load_json(paths["meta"]) transcript_entry = transcript_entry_from_statement(statement) - assert meta["proof_kind"] == "native-halo2" + assert meta["proof_kind"] == "native-sigma-v4" total_time, count = verify_artifacts( tmp_path, [transcript_entry], _manifest_for(witness) ) @@ -144,9 +153,7 @@ def test_expected_adapter_manifest_mismatch_rejects(tmp_path): statement = load_json(paths["statement"]) transcript = [transcript_entry_from_statement(statement)] manifest = _manifest_for(witness) - manifest[0]["adapter_commitment"]["value"] = str( - int(manifest[0]["adapter_commitment"]["value"]) + 1 - ) + manifest[0]["adapter_commitment"]["value"] = "0" * 64 with pytest.raises(ProofContractError, match="expected adapter manifest"): verify_artifacts(tmp_path, transcript, manifest) @@ -174,9 +181,7 @@ def test_tampered_vk_and_proof_reject(tmp_path): paths = write_invocation_artifacts(tmp_path, witness) statement = load_json(paths["statement"]) - statement["adapter_commitment"]["value"] = str( - int(statement["adapter_commitment"]["value"]) + 1 - ) + statement["adapter_commitment"]["value"] = "0" * 64 statement["statement_digest"] = "00" * 32 Path(paths["statement"]).write_text( json.dumps(statement, sort_keys=True, separators=(",", ":")) + "\n",