Add dedicated LFM2MoE GGUF graph support - #606
Merged
Merged
Conversation
Performance Comparison
|
justinchuby
force-pushed
the
justinchuby-finalize-gguf-support-matrix
branch
from
August 25, 2026 16:56
7326da4 to
2e0df5a
Compare
Base automatically changed from
justinchuby-finalize-gguf-support-matrix
to
main
August 25, 2026 16:57
Implement the dense-prefix and correction-biased sigmoid-routed expert graph on top of LFM2's mixed convolution/attention state ABI. Add pinned GGUF metadata and tensor closure, explicit quantized-source dequantization, state threading coverage, synthetic parity, and weight alignment tests while retaining the deferred runtime verdict pending independent real-weight evidence.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Regenerate the authoritative support matrix and move quantization rejection into the architecture registry used by current main. Preserve the routing correction bias in fp32 so fp16 and bf16 exports keep a valid, reference-matching selection graph. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
justinchuby
force-pushed
the
justinchuby-implement-deferred-recurrent-gguf
branch
from
August 25, 2026 17:10
ef635bd to
f09eb0c
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds first-class support for importing and graph-building the LFM2MoE GGUF architecture in mobius, including a dedicated MoE decoder graph, pinned GGUF config/tensor mapping, and expanded synthetic/weight-alignment test coverage. It fits into mobius’ model-registry + GGUF-import pipeline by introducing an lfm2_moe model type and making lfm2moe a supported (but runtime-deferred) GGUF architecture with explicit float import requirements.
Changes:
- Add
Lfm2MoECausalLMModel(sigmoid top‑k routing with selection-only correction bias, dense-prefix → routed MoE transition) plusLfm2MoeConfig. - Extend GGUF integration to map/configure/validate
lfm2moe, including tensor-map extras, postprocessing invariants, mixed-state contract checks, and dequantization-only import behavior. - Add targeted tests for expert weight alignment, synthetic HF-config adaptation, GGUF build/save/load/state semantics, and update GGUF support-matrix documentation.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/weight_alignment_test.py | Adds targeted alignment tests for LFM2MoE expert weight formats (w1/w2/w3 and fused tensors). |
| tests/synthetic_parity_test.py | Extends HF-config adaptation to cover lfm2_moe (including MoE-specific field translation). |
| tests/_test_configs.py | Registers a tiny lfm2_moe test config for shared test parametrizations. |
| src/mobius/models/lfm2.py | Introduces LFM2MoE routing gate, MoE feed-forward, MoE layer composition, and top-level Lfm2MoECausalLMModel. |
| src/mobius/models/lfm2_test.py | Adds a dtype-preservation test ensuring selection-bias math stays float32 under fp16 builds. |
| src/mobius/models/init.py | Exports Lfm2MoECausalLMModel from the public models package. |
| src/mobius/integrations/gguf/_tensor_mapping.py | Adds LFM2MoE extra tensor stems and routes exp_probs_b.bias to an expert_bias parameter via the @ suffix convention. |
| src/mobius/integrations/gguf/_config_mapping.py | Adds lfm2moe hybrid-layout handling and a pinned postprocessor enforcing loader invariants. |
| src/mobius/integrations/gguf/_config_mapping_test.py | Adds tests validating the LFM2MoE postprocessor’s defaults and fail-closed behavior. |
| src/mobius/integrations/gguf/_builder.py | Extends hybrid tensor-contract validation to enforce dense vs routed FFN tensor families per layer for lfm2moe. |
| src/mobius/integrations/gguf/_builder_test.py | Adds end-to-end GGUF build tests for float import, explicit dequantization, state rollback/reorder, and misdispatch rejection. |
| src/mobius/integrations/gguf/_arch_registry.py | Promotes lfm2moe to supported config/tensor-map/graph with runtime still deferred; rejects quantized-preservation import. |
| src/mobius/integrations/gguf/_arch_registry_test.py | Updates census expectations and adds a focused assertion for the new lfm2moe support verdicts. |
| src/mobius/_registry.py | Registers lfm2_moe model type to Lfm2MoECausalLMModel with Lfm2MoeConfig. |
| src/mobius/_configs/_base.py | Adds Lfm2MoeConfig config class for dense-prefix + expert-bias toggles. |
| src/mobius/_configs/init.py | Exports Lfm2MoeConfig from the configs package. |
| docs/api/build_from_gguf.md | Updates the support matrix/census and documents lfm2moe as graph-supported but runtime-deferred with quantized import rejected. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
titaiwangms
added a commit
that referenced
this pull request
Aug 26, 2026
## Summary - align Jais2, CodeShell, Xverse, and Kimi-K3 model `config_class` declarations with registry dispatch - record Kimi-K3 and LFM2-MoE coverage limitations without fabricating golden data - support both tensor and mapping Transformers linear-attention cache ABIs in Falcon-H1 and GatedDeltaNet parity tests - mark the selective-MXFP4 Kimi-K3 checkpoint as a strict graph-only L2 xfail - preserve independent dashboard confidence levels when L2 xfails but L3 passes ## Root causes - #635 introduced the Jais2, CodeShell, and Xverse registry/model contract mismatches - #621 introduced the Kimi-K3 contract and coverage gaps - #606 introduced LFM2-MoE without an L4/L5 coverage disposition - #610 assumed the older tensor cache representation in Falcon-H1 parity coverage - #327 added a dashboard meta-assertion that incorrectly required every L2 xfail to have confidence below L2, contradicting the independent-level truth table ## Validation - dashboard plus original CI regression gates: 90 passed - affected graph and weight-alignment gates: 25 passed - Kimi-K3 L2: 1 passed, 2 strict xfailed --------- Signed-off-by: titaiwang <titaiwang@microsoft.com> Copilot-Session: 2fe48e9f-4ebc-43a9-85f3-4e4f99f3ed5d
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation
The repository-wide non-integration run completed 3,345 passing tests with three unrelated existing failures:
granitemoehybridsynthetic parity and twoglm_moe_dsashape-inference checks. No GitHub CI wait requested.