Add dedicated Falcon-H1 model and GGUF import - #610
Merged
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Performance Comparison
|
🏗️ Architecture Diff
qwen3_5_moe (hybrid-text-generation) / model — 19 change(s)Op summary: 264 → 265 nodes --- base
+++ head
@@ -60,10 +60,11 @@
Constant
Reshape
Reshape
-RMSNormalization
-Cast
-Swish
-Cast
+Cast
+Cast
+RMSNormalization
+Cast
+Swish
Mul
CastLike
ConstantAdded nodes:
Removed nodes:
Modified attributes:
Connectivity changes:
qwen3_5_text (hybrid-text-generation) / model — 11 change(s)Op summary: 126 → 127 nodes --- base
+++ head
@@ -60,10 +60,11 @@
Constant
Reshape
Reshape
+Cast
+Cast
RMSNormalization
Cast
Swish
-Cast
Mul
CastLike
ConstantAdded nodes:
Removed nodes:
Modified attributes:
Connectivity changes:
qwen3_5_vl (hybrid-qwen-vl) / decoder — 12 change(s)Op summary: 149 → 150 nodes --- base
+++ head
@@ -82,10 +82,11 @@
Constant
Reshape
Reshape
+Cast
+Cast
RMSNormalization
Cast
Swish
-Cast
Mul
CastLike
ConstantAdded nodes:
Removed nodes:
Modified attributes:
Connectivity changes:
qwen3_next (hybrid-text-generation) / model — 97 change(s)Op summary: 562 → 565 nodes --- base
+++ head
@@ -60,10 +60,11 @@
Constant
Reshape
Reshape
-RMSNormalization
-Cast
-Swish
-Cast
+Cast
+Cast
+RMSNormalization
+Cast
+Swish
Mul
CastLike
Constant
@@ -203,10 +204,11 @@
Constant
Reshape
Reshape
-RMSNormalization
-Cast
-Swish
-Cast
+Cast
+Cast
+RMSNormalization
+Cast
+Swish
Mul
CastLike
Constant
@@ -346,10 +348,11 @@
Constant
Reshape
Reshape
-RMSNormalization
-Cast
-Swish
-Cast
+Cast
+Cast
+RMSNormalization
+Cast
+Swish
Mul
CastLike
ConstantAdded nodes:
Removed nodes:
Modified attributes:
Connectivity changes:
Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
justinchuby
force-pushed
the
justinchuby-implement-deferred-recurrent-gguf
branch
from
August 25, 2026 17:10
ef635bd to
f09eb0c
Compare
Base automatically changed from
justinchuby-implement-deferred-recurrent-gguf
to
main
August 25, 2026 17:10
Implement the dedicated parallel attention and Mamba2 decoder, four-state dynamic cache ABI, strict pinned GGUF metadata and tensor closure, and fail-closed quantization handling. Add synthetic, stateful, real-checkpoint, CLI, and GGUF regression coverage while keeping downstream runtime support deferred on the heterogeneous-state schema. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
Pin Falcon-H1 in the parent branch's explicit quantized-import policy while retaining dequantize-only fail-closed behavior. Reject SSM group geometries that cannot expand across Mamba heads, add regression coverage, and align the changed files with repository lint rules.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
justinchuby
force-pushed
the
justinchuby-falcon-h1-gguf
branch
from
August 25, 2026 17:23
ea6f21f to
5db3720
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Adds first-class Mobius support for the Falcon-H1 hybrid decoder architecture and its GGUF import path. This fits into Mobius’s “Models → Tasks → Registry/Exporter” stack by introducing a dedicated Falcon-H1 model implementation, a matching task with a four-state ABI, and GGUF metadata/tensor-contract validation plus extensive tests.
Changes:
- Introduce a dedicated Falcon-H1 decoder graph (parallel Attention + Mamba2 per layer) and a task that exposes an ordered 4-state ABI:
(key, value, conv_state, ssm_state). - Add GGUF support for
falcon-h1: strict metadata validation, tensor closure enforcement, config/tensor mapping, and import-time policy enforcement (no static cache; quantized sources require dequantization). - Add comprehensive coverage: unit/integration parity, GGUF builder tests (including failure paths), docs/support census updates, and registration wiring.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/synthetic_parity_test.py | Allows Falcon-H1 HF config creation to override mlp_bias for parity. |
| tests/falcon_h1_test.py | Adds Falcon-H1 graph/parity/state ABI tests + pinned checkpoint integration parity. |
| tests/_test_configs.py | Adds Falcon-H1 tiny config; adjusts CTRL tiny config bias flag usage. |
| testdata/cases/causal-lm/falcon-h1-tiny-90m.yaml | Adds an L4/L5 case entry (skipped due to deferred runtime packaging). |
| src/mobius/tasks/_falcon_h1.py | New Falcon-H1 task defining graph I/O and four-state ABI ordering. |
| src/mobius/tasks/_cache_utils.py | Ensures recurrent function registration recognizes FalconH1Config as Mamba2-capable. |
| src/mobius/tasks/init.py | Exports/registers FalconH1CausalLMTask and task name mapping. |
| src/mobius/models/falcon_h1.py | New Falcon-H1 model implementation (parallel Attention+Mamba2 + SwiGLU). |
| src/mobius/models/init.py | Exports FalconH1ForCausalLM. |
| src/mobius/integrations/gguf/_tensor_processors.py | Treats Falcon-H1 as Mamba2-like for tensor processing rules. |
| src/mobius/integrations/gguf/_tensor_mapping.py | Adds Falcon-H1 GGUF tensor-name → Mobius module-name mapping. |
| src/mobius/integrations/gguf/_config_mapping.py | Adds Falcon-H1 GGUF metadata→config mapping + postprocess validation. |
| src/mobius/integrations/gguf/_builder.py | Adds Falcon-H1 tensor-contract validation + import-time policy checks. |
| src/mobius/integrations/gguf/_builder_test.py | Adds Falcon-H1 GGUF build/save/load/dequantize/bias-family/failure-path tests. |
| src/mobius/integrations/gguf/_arch_registry.py | Marks falcon-h1 graph+import supported (runtime remains deferred). |
| src/mobius/integrations/gguf/_arch_registry_test.py | Updates pinned supported-arch census and asserts Falcon-H1 resolves to dedicated graph. |
| src/mobius/components/_rms_norm.py | Extends PostGatedRMSNorm with optional grouped normalization support. |
| src/mobius/components/_mamba_block.py | Extends Mamba2Block controls (bias flags, multipliers, dt clipping, optional padding mask). |
| src/mobius/components/_attention.py | Adds Falcon-H1 key scaling support via key_multiplier. |
| src/mobius/_registry.py | Registers Falcon-H1 model/task/config and sets test_model_id/family. |
| src/mobius/_configs/_base.py | Introduces FalconH1Config and transformers config extraction/validation. |
| src/mobius/_configs/init.py | Exports FalconH1Config. |
| src/mobius/main.py | Rejects --static-cache for Falcon-H1 model type. |
| docs/api/build_from_gguf.md | Updates GGUF support matrix/census and Falcon-H1 narrative to “supported graph/import, deferred runtime”. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+2729
to
+2731
| def __post_init__(self) -> None: | ||
| self.attn_qkv_bias = self.attention_bias | ||
| self.attn_o_bias = self.attention_bias |
Comment on lines
299
to
303
| "hidden_act": "relu", | ||
| "tie_word_embeddings": True, | ||
| "num_key_value_heads": TINY_HEADS, | ||
| "attn_qkv_bias": True, | ||
| "attn_o_bias": True, | ||
| "attention_bias": True, | ||
| }, |
justinchuby
added a commit
that referenced
this pull request
Aug 25, 2026
## Summary - clean up still-valid typing, documentation, test robustness, error-message, maintainability, and performance findings left by the GGUF PR stack - preserve the behavioral fixes merged in #625-#632 - hash reused multi-GB GGUF sources once, at the final pre-publication integrity gate, while retaining cheap identity checks around staging Exact base: `2db9d33debdc254d879a51b14434c9a81c230f4f` Exact head: `4541ed2bc9d2ab4227484510b4a85b2d9113eb25` ## Reconstructed original 17-item low-priority tranche The persisted audit retained only the totals, so this list was reconstructed from the live threads and current source. All but the already-fixed #596 comment are addressed in this PR. | PR | Comment | Disposition | |---|---:|---| | #550 | 3837144656 | Implemented: correct tuple return annotation | | #552 | 3837389183 | Implemented: multichannel waveform shape docs | | #559 | 3837716261 | Implemented: name-based cache assertions | | #573 | 3854308350 | Implemented: one final GGUF hash, with integrity regression coverage | | #574 | 3854345521 | Implemented: `TensorRole \| None` typing | | #574 | 3854345597 | Implemented: removed obsolete verdict filtering | | #577 | 3854472859 | Implemented: documented SSM sequence length | | #578 | 3843112613 | Implemented: documented F64 passthrough | | #579 | 3854518332 | Implemented: generalized fused-projection error | | #580 | 3854576300 | Implemented: removed brittle node counts | | #583 | 3854681386 | Implemented: documented conditional draft outputs | | #587 | 3854816872 | Implemented: corrected MTP output contract docs | | #596 | 3846110059 | Already fixed on base: unambiguous GQA bias comment | | #600 | 3855174281 | Implemented: metadata-count-only MTP error | | #607 | 3855776048 | Implemented: stable route-field assertions | | #607 | 3855776086 | Implemented: public tensor iterator | | #609 | 3856486136 | Implemented: fail-closed LM-head comment | ## Current unresolved-thread disposition This covers all 48 Copilot threads returned by the reproducible #600-#630 query. The one human #623 thread is excluded. | PR | Comment | Current-main disposition and evidence | |---|---:|---| | #600 | 3855174177 | Already fixed by #629: package cycle and reserved-sidecar validation | | #600 | 3855174238 | Already fixed by #629: explicit MTP sidecar naming/loading | | #600 | 3855174281 | Implemented here: error no longer invents an observed block count | | #602 | 3848155961 | Outside exact stack; already fixed: top-level `expert_dtype` is classified before early return | | #602 | 3848155983 | Outside exact stack; still-valid behavioral block-quant validation, unchanged | | #602 | 3848156000 | Outside exact stack; still-valid truncated-read behavioral finding, unchanged | | #602 | 3848156022 | Outside exact stack; still-valid descriptor byte/dtype validation, unchanged | | #602 | 3848156040 | Outside exact stack; still-valid expert-bank payload validation, unchanged | | #603 | 3855249765 | Already fixed by #630: runtime preflight preserves shard sets | | #603 | 3855249840 | Already fixed by #630: success output follows durable runtime publication | | #604 | 3855343082 | Implemented here: graph-only MTP persistence distinguished from runtime rejection | | #604 | 3855343131 | Already fixed by #630: runtime success messages are atomic | | #607 | 3855776001 | Implemented here: missing generation golden skips before provenance read | | #607 | 3855776048 | Implemented here: only stable route fields are asserted | | #607 | 3855776086 | Implemented here: tensor count uses `tensor_items_raw()` | | #608 | 3856079371 | Still-valid behavioral cache-symlink containment finding; unchanged | | #608 | 3856079415 | Still-valid behavioral lowercase-digest validation finding; unchanged | | #609 | 3856486136 | Implemented here: comment matches value-preserving policy | | #610 | 3855541683 | Already fixed by #628: Falcon bias precedence is explicit | | #610 | 3855541761 | Already fixed by #628: CTRL tiny config exercises projection biases | | #611 | 3856595840 | Implemented here: runtime test resolves the distribution providing the module | | #612 | 3855677383 | Already fixed by #625: supported-version endianness detection | | #612 | 3855677427 | Implemented here: shared `INT64_MAX` sentinel | | #612 | 3855677460 | Implemented here: shared PLaMo2 width inference | | #612 | 3855677486 | Implemented here: accepted PLaMo2 activation spellings are explicit | | #613 | 3855845678 | Implemented here: canonical issue URL | | #613 | 3855845757 | Implemented here: Mamba-1 function-registration docs | | #613 | 3855845806 | Implemented here: test expects the canonical issue URL | | #614 | 3855988545 | Implemented here: removed stale Nemotron-H divergence comments | | #614 | 3855988597 | Already fixed by #628: zero-head geometry raises actionable `ValueError` | | #615 | 3856082290 | Already fixed by #626: dense GraniteHybrid bias closure | | #618 | 3856729330 | Implemented here: required routes filter ORT GenAI evidence | | #618 | 3856729409 | Implemented here: env-selected runtime version is authoritative | | #618 | 3856729490 | Stale/N/A: PR-description-only matrix claim; repository workflow claims one pinned version | | #618 | 3856729563 | Implemented here: schema tail restored to normal indentation | | #619 | 3856342484 | Already fixed on base: Kimi Linear uses `/issues/605` | | #619 | 3856342532 | Already fixed by #628: config rejects convolution kernels below 2 | | #619 | 3856342580 | Already fixed by #628: GGUF contract rejects convolution kernels below 2 | | #620 | 3855717041 | Already fixed by #627: tied LM-head-only checkpoints are retained | | #621 | 3856722324 | Already fixed by #628: Kimi-K3 required metadata is complete | | #623 | 3855931210 | N/A to current main: comment belongs to open, unmerged #623 | | #623 | 3855939302 | N/A to current main: comment belongs to open, unmerged #623 | | #623 | 3855939358 | N/A to current main: comment belongs to open, unmerged #623 | | #623 | 3855939394 | N/A to current main: comment belongs to open, unmerged #623 | | #624 | 3856777152 | Implemented here: runtime compatibility reuses the emitted model type | | #625 | 3857049733 | Implemented here: unsupported header reports both endian candidates | | #629 | 3857313182 | Newer post-audit behavioral sidecar-symlink cleanup finding; unchanged | | #629 | 3857313251 | Newer post-audit cross-platform path-safety finding; unchanged | ## Validation - affected GGUF/package/ORT GenAI/model/schema tests: 1,040 passed - broad non-integration suite: 7,851 passed, 56 skipped, 1 subtest passed - generated GGUF docs checks: 7 passed - initialized `lintrunner`; full lint/format passed - GPT-5.6 Sol medium review: one integrity finding fixed; re-review found no significant issues Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
titaiwangms
added a commit
that referenced
this pull request
Aug 26, 2026
## Summary - align Jais2, CodeShell, Xverse, and Kimi-K3 model `config_class` declarations with registry dispatch - record Kimi-K3 and LFM2-MoE coverage limitations without fabricating golden data - support both tensor and mapping Transformers linear-attention cache ABIs in Falcon-H1 and GatedDeltaNet parity tests - mark the selective-MXFP4 Kimi-K3 checkpoint as a strict graph-only L2 xfail - preserve independent dashboard confidence levels when L2 xfails but L3 passes ## Root causes - #635 introduced the Jais2, CodeShell, and Xverse registry/model contract mismatches - #621 introduced the Kimi-K3 contract and coverage gaps - #606 introduced LFM2-MoE without an L4/L5 coverage disposition - #610 assumed the older tensor cache representation in Falcon-H1 parity coverage - #327 added a dashboard meta-assertion that incorrectly required every L2 xfail to have confidence below L2, contradicting the independent-level truth table ## Validation - dashboard plus original CI regression gates: 90 passed - affected graph and weight-alignment gates: 25 passed - Kimi-K3 L2: 1 passed, 2 strict xfailed --------- Signed-off-by: titaiwang <titaiwang@microsoft.com> Copilot-Session: 2fe48e9f-4ebc-43a9-85f3-4e4f99f3ed5d
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
8d9af256337d1a501250f9bbf4c0859a654bddd6Validation
python -m pytest tests/falcon_h1_test.py tests/build_graph_test.py tests/weight_alignment_test.py tests/synthetic_parity_test.py -q -k 'falcon_h1' --tb=short— 11 passedpython -m pytest tests/falcon_h1_test.py -m integration -k pinned -q --tb=short— pinnedtiiuae/Falcon-H1-Tiny-90M-Base@7994372e93b62822ae25f8bfb19f653649cea3a3full-logit parity passedpython -m pytest src/mobius/integrations/gguf -q --tb=short -n auto— 2,583 passedpython -m pytest tests/yaml_schema_test.py tests/falcon_h1_test.py tests/build_graph_test.py tests/weight_alignment_test.py tests/synthetic_parity_test.py -q -k 'falcon_h1 or yaml_schema' --tb=short -n auto— 261 passedpython -m pytest tests/build_graph_test.py tests/cli_test.py src/ -q -k 'not phi4mm and not apply_weights_unknown and not glm_moe_dsa' --tb=short -n auto— 7,056 passed, 52 skippedpython scripts/generate_gguf_support_docs.py --check— passedlintrunner f --output oneline --all-files && lintrunner -a— clean after initialized lintrunnerReviews
Runtime and quantization verdict
Graph construction and GGUF import are supported. Runtime packaging remains DEFERRED because the downstream package schema cannot represent heterogeneous K/V plus convolution/SSM states; tracked by #605. No ORT GenAI runtime-support claim is made.
Quantized Falcon-H1 sources can be imported through explicit dequantization.
keep_quantized=Truefails closed because the current route cannot preserve only compatible attention/FFN MatMul roles while safely dequantizing recurrent/state-sensitive tensors and retaining every quantization sidecar.Waivers