Add exact MiniMax-01 GGUF support - #616
Merged
Merged
Conversation
Performance Comparison
|
🏗️ Architecture Diff
No architecture changes detected. ✅ Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
justinchuby
force-pushed
the
justinchuby-granitehybrid-moe-gguf
branch
2 times, most recently
from
August 25, 2026 18:33
9cbc655 to
8690dcd
Compare
Implement the pinned MiniMax-01 hybrid architecture, strict GGUF metadata and tensor validation, exact Lightning Attention state handling, and quantized projection preservation. Add synthetic parity, ORT state threading, import, CLI, save/reload, and rejection coverage while deferring released runtime packaging until heterogeneous rollback semantics are representable. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: aedc6984-fda0-45fc-bddb-a6317953da96 Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Include MiniMax-01 in the quantized-import closure set introduced by the squash-merged parent so the pinned capability test matches the reviewed packed projection support. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Construct MiniMax token embeddings through the quantization-aware factory so untied packed tables import and tied packed heads share executable storage. Strengthen exact role and ORT coverage, and correct the runtime-only deferral documentation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
justinchuby
force-pushed
the
justinchuby-minimax-01-gguf
branch
from
August 25, 2026 18:50
3f5fdf1 to
d4957ef
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds first-class, exact GGUF import support for the MiniMax-01 / MiniMax-Text-01 hybrid architecture in Mobius, aligning the exported ONNX graph and GGUF tensor/metadata contract with the pinned llama.cpp behavior while explicitly keeping ORT GenAI “runtime packaging” deferred due to heterogeneous state/rollback limitations tracked in #605.
Changes:
- Introduces a dedicated
MiniMaxConfigplus registry wiring for multiple HFmodel_typespellings, and updates the MiniMax model graph to faithfully represent hybrid Lightning/full-attention scheduling, scaling, and routing behavior. - Adds MiniMax GGUF architecture registration, config/tensor mapping, and strict fail-closed contract validation (metadata + closure + per-tensor shapes) before graph construction.
- Expands test coverage for MiniMax GGUF import (float + keep-quantized), state threading/replay/reorder, tied embeddings/LM head behavior, and schedule derivation; updates docs + pinned reference revision.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/_test_configs.py | Adds MiniMax test config coverage and explicitly selects MiniMaxConfig. |
| testdata/cases/causal-lm/minimax-text-01.yaml | Pins a specific MiniMax reference revision and updates notes to reflect current support/deferment. |
| src/mobius/models/minimax.py | Reworks MiniMax model to match the exact hybrid schedule, scaling, routing gate, and quantized-linear plumbing. |
| src/mobius/integrations/gguf/_tensor_mapping.py | Adds MiniMax GGUF tensor-family → Mobius weight-name mapping. |
| src/mobius/integrations/gguf/_config_mapping.py | Adds MiniMax GGUF metadata mapping, schedule derivation, and a MiniMax postprocessor that restores pinned semantics. |
| src/mobius/integrations/gguf/_config_mapping_test.py | Adds tests for MiniMax default scheduling and interval validation. |
| src/mobius/integrations/gguf/_builder.py | Adds strict MiniMax-01 GGUF contract validation (metadata + closure + shapes) before graph build. |
| src/mobius/integrations/gguf/_builder_test.py | Adds end-to-end MiniMax GGUF build tests (float + quantized), state behavior, tieing, CLI, and fail-closed rejection cases. |
| src/mobius/integrations/gguf/_arch_registry.py | Promotes minimax-01 to supported (config/tensor/graph) while keeping runtime packaging deferred with updated reason. |
| src/mobius/integrations/gguf/_arch_registry_test.py | Updates expected supported-architecture census and includes minimax-01. |
| src/mobius/components/_moe.py | Adds disable_qmoe switch to force the dense MoE path even when QMoE would otherwise be eligible. |
| src/mobius/components/_lightning_attention.py | Updates LightningAttention to MiniMax-01 exact head geometry, masking semantics, slope formula, and quantized-linear plumbing. |
| src/mobius/components/_lightning_attention_test.py | Adds unit tests to pin the new MiniMax-01 decay formula behavior. |
| src/mobius/_registry.py | Registers multiple MiniMax model_type spellings with MiniMaxConfig. |
| src/mobius/_configs/_base.py | Adds MiniMax-specific config fields to ArchitectureConfig and introduces MiniMaxConfig.from_transformers. |
| src/mobius/_configs/_base_test.py | Adds unit test for MiniMaxConfig.from_transformers extraction. |
| src/mobius/_configs/init.py | Exports MiniMaxConfig. |
| docs/api/build_from_gguf.md | Updates the generated GGUF support census and MiniMax-01 row/details. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
justinchuby
added a commit
that referenced
this pull request
Aug 25, 2026
## Summary - implement the pinned Kimi Linear hybrid KDA/NoPE-MLA graph, dense/MoE schedule, and heterogeneous recurrent/KV state ABI - add strict GGUF metadata/tensor closure, layout transforms, compatible quantized MatMul/expert preservation, and malformed-input rejection - add graph, weight, ORT state/replay/reorder/padding, GGUF float/quantized, roundtrip, CLI, and documentation coverage ## Runtime boundary Generic OGA packaging remains deferred to #605 because the released decoder schema cannot represent mixed MLA KV and KDA convolution/matrix states exactly. ## Validation - focused Kimi: 34 passed - full GGUF: 2684 passed - broad non-integration: 7223 passed, 54 skipped - generated GGUF docs check passed - initialized lint completed with only pre-existing Falcon-H1 warnings - two independent reviews completed and findings fixed Stacked on #616 (`justinchuby-minimax-01-gguf`). --------- Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
8d9af256337d1a501250f9bbf4c0859a654bddd6Runtime scope
Released ORT GenAI packaging remains deferred to #605 because MiniMax requires heterogeneous KV/recurrent state slots plus bounded rollback snapshots that the current package schema cannot faithfully represent. Graph-call state threading, deterministic replay, and batch reorder are covered here.
No small public exact checkpoint exists; the approximately 456B public checkpoint is distributed across 413 shards, so real-checkpoint parity and OGA E2E are not locally feasible.
Validation
51 passed, 4 skipped— focused MiniMax config, graph, GGUF closure/import, synthetic parity, weight alignment, Lightning, CLI/roundtrip, and ORT state tests7626 passed, 56 skipped— broad non-integration suitelintrunner -a— cleangit diff --check— cleanRebased linearly onto merged #615 (
cc80c77db0e8e176a04e66611423399270bcfa63).