Fail closed on lossy GGUF quantization preservation

docs/api/build_from_gguf.md

@@ -181,7 +181,7 @@ def build_from_gguf(
 | `gguf_path` | `str \| Path` | (required) | Local `.gguf` path or `owner/repo:filename.gguf` Hub reference. |
 | `task` | `str \| None` | `None` | Override the model task (e.g. `"text-generation"`). When `None`, the task is auto-detected from the model type. |
 | `dtype` | `str \| None` | `None` | Override model dtype (e.g. `"f16"`). When `None`, defaults to float32. |
-| `keep_quantized` | `bool` | `True` | Preserve quantization when present. Supported affine blocks are repacked as `MatMulNBits`; in text-only builds, supported native IQ/MXFP4 projection blocks retain their bytes. Multimodal and mixed presets may require dequantization/requantization. Set to `False` to dequantize all weights. |
+| `keep_quantized` | `bool` | `True` | Preserve quantization when present. Supported affine blocks are repacked as `MatMulNBits`; in text-only builds, supported native IQ/MXFP4 projection blocks retain their bytes. Projection tensors that would require lossy requantization are rejected. Set to `False` to dequantize all weights. |
 | `execution_provider` | `str` | `"default"` | Target EP for EP-aware graph optimization. |
 | `mmproj` | `str \| Path \| None` | `None` | Optional companion multimodal-projector GGUF. |
 | `static_cache` | `bool` | `False` | Build a fixed-width KV cache when the architecture supports it. |
@@ -267,20 +267,23 @@ mobius build-gguf draft.gguf --target-config target/ --output output/draft/
 
 1. Reads GGUF metadata to detect architecture and config
 2. Maps GGUF tensor names to HuggingFace weight names
-3. Preserves supported quantized tensors by default, using repacking,
-   text-only native-block retention, or dequantize/requantize according to the
-   source qtype and build path; `keep_quantized=False` dequantizes every tensor
+3. Preserves supported quantized tensors by default using value-preserving
+   repacking or text-only native-block retention; any projection requiring
+   lossy requantization fails closed, while `keep_quantized=False` explicitly
+   dequantizes every tensor
 4. Applies architecture-specific tensor processors (e.g. Q/K permute)
 5. Builds the ONNX graph using the same pipeline as `build()`
 6. Runs `preprocess_weights()` (HF → ONNX name mapping)
 7. Applies weights to the graph
 
 F32-, F16-, and BF16-only GGUFs use the normal float import path even though
 `keep_quantized=True` is the default: there is no quantization to preserve.
-Quantized GGUFs whose qtypes have no trustworthy decoder or compatible runtime
-kernel (currently `Q2_0`) fail with an actionable error. Decoder-backed formats
-such as `Q5_K` use the explicit dequantize/requantize route; they are not
-silently treated as preserved source quantization.
+Quantized GGUFs containing only qtypes with no supported preservation target
+(for example, pure Q5_K weights) fail with an actionable error rather
+than silently becoming float. Pass `keep_quantized=False` to request that float
+conversion explicitly.
+Mixed presets such as Q4_K_M also fail when their projection inventory includes
+Q5/Q6/Q8 tensors that cannot share one lossless MatMulNBits contract.
 
 ### Native blocks, conversion, and source-file reuse
 

docs/cli_reference.md

@@ -323,9 +323,10 @@ mobius build --model Qwen/Qwen2.5-0.5B --output output_dir/ \
 
 Build an ONNX model from a GGUF file (e.g. from llama.cpp). This is an explicit
 opt-in import path; `mobius build` does not auto-discover or select GGUF files.
-Supported GGUF quantization is preserved by default. This can involve
-byte-preserving native blocks in text-only builds, affine repacking, or
-dequantize/requantize for multimodal and mixed source qtypes.
+Supported GGUF quantization is preserved by default through byte-preserving
+native blocks or value-preserving affine repacking. Mixed source qtypes that
+would require lossy dequantization/requantization, including Q4_K_M presets,
+fail closed and require `--dequantize`.
 
 > **Note**: Requires the optional `gguf` package: `pip install mobius-onnx[gguf]`
 
@@ -377,6 +378,8 @@ contain no quantization to preserve.
 Quantized files containing only qtypes with no supported preservation target
 (for example, pure Q5_K weights) fail instead of silently becoming
 float. Re-run with `--dequantize` to request explicit float conversion.
+The same rule applies when only some projection tensors are incompatible with
+the selected affine target; Mobius does not silently requantize those tensors.
 
 Encoder-only BERT and ModernBERT GGUF backbones auto-select
 `feature-extraction` and output `last_hidden_state`; they do not produce logits

docs/model-catalog.md

@@ -267,11 +267,10 @@ All decoder-only LLMs and MoE models support quantized weight loading:
 | **AWQ** | HuggingFace (e.g. `-AWQ` suffix models) | `build("TheBloke/Llama-2-7B-AWQ")` |
 | **GGUF** | Local `.gguf` files | `build_from_gguf("model.gguf")` |
 
-GGUF import preserves supported quantization by default using affine repacking
-or, for supported text-only inputs, native quantized ops. Multimodal and mixed
-source qtypes can require dequantization/requantization, so this does not imply
-byte preservation for every tensor. Use `--dequantize` for an explicitly float
-model.
+GGUF import preserves supported quantization by default using value-preserving
+affine repacking or, for supported text-only inputs, native quantized ops.
+Projection tensors that would require lossy dequantization/requantization fail
+closed. Use `--dequantize` for an explicitly float model.
 
 Encoder GGUF imports for `bert` and `modern-bert` select
 `feature-extraction` and expose token-level `last_hidden_state` only. Pooling,

src/mobius/functions/matmul_nbits_test.py

@@ -46,7 +46,12 @@ def _const(name: str, arr: np.ndarray) -> ir.Value:
 
 
 def _build_matmulnbits_model(
-    n_out: int, k_in: int, block: int, seed: int = 0
+    n_out: int,
+    k_in: int,
+    block: int,
+    seed: int = 0,
+    *,
+    accuracy_level: int | None = None,
 ) -> tuple[ir.Model, dict, np.ndarray, np.ndarray, np.ndarray]:
     """Build a single-node MatMulNBits model with random 4-bit weights.
 
@@ -67,14 +72,15 @@ def _build_matmulnbits_model(
     sc_c = _const("scales", scales)
     zp_c = _const("zero_points", zero_points)
     y = ir.Value(name="Y")
+    attributes = {"K": k_in, "N": n_out, "bits": 4, "block_size": block}
+    if accuracy_level is not None:
+        attributes["accuracy_level"] = accuracy_level
     node = ir.Node(
         "com.microsoft",
         "MatMulNBits",
         inputs=[a_val, b_c, sc_c, zp_c],
         outputs=[y],
-        attributes=ir.convenience.convert_attributes(
-            {"K": k_in, "N": n_out, "bits": 4, "block_size": block}
-        ),
+        attributes=ir.convenience.convert_attributes(attributes),
     )
     graph = ir.Graph(
         inputs=[a_val],
@@ -118,6 +124,30 @@ def test_inline_matches_native_op(self):
 
         np.testing.assert_allclose(got, reference, rtol=0, atol=0)
 
+    def test_accuracy_four_matches_explicit_asymmetric_dequantization(self):
+        """The CPU kernel consumes Mobius's packed weights and zero points correctly."""
+        model, feeds, packed, scales, packed_zero_points = _build_matmulnbits_model(
+            n_out=8,
+            k_in=64,
+            block=32,
+            seed=11,
+            accuracy_level=4,
+        )
+        got = _run(model, feeds)
+
+        quants = np.empty((8, 2, 32), dtype=np.float32)
+        quants[..., 0::2] = packed & 0x0F
+        quants[..., 1::2] = packed >> 4
+        zero_points = np.empty((8, 2), dtype=np.float32)
+        zero_points[..., 0::2] = packed_zero_points & 0x0F
+        zero_points[..., 1::2] = packed_zero_points >> 4
+        weights = ((quants - zero_points[..., None]) * scales[..., None]).reshape(8, 64)
+        expected = feeds["A"] @ weights.T
+
+        error = got - expected
+        assert float(np.max(np.abs(error))) < 0.1
+        assert float(np.linalg.norm(error) / np.linalg.norm(expected)) < 0.02
+
     def test_inline_matches_native_op_odd_blocks(self):
         """Odd n_blocks exercises the zero-point nibble slice (ceil(nb/2))."""
         # K=96, block=32 → nb=3 (odd)