Add specialized GGUF encoder architectures - #634
Merged
Conversation
Performance Comparison
|
🏗️ Architecture Diff
No architecture changes detected. ✅ Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds four specialized stateless GGUF encoder architectures (EuroBERT, NeoBERT, dense NomicBERT, JinaBERT-v2) to mobius’ GGUF import pipeline, including a dedicated feature-extraction task that matches the GGUF pooling ABI and strict tensor-closure validation.
Changes:
- Add
gguf_encoders.pyencoder model implementations and register them in the mobius model registry. - Introduce
GGUFEncoderFeatureExtractionTaskand wire it through task registration and GGUF builder dispatch. - Extend GGUF architecture registry, config postprocessing, tensor mapping, upstream pin inventory, docs, and add targeted tests for the new encoders.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/_test_configs.py | Adds tiny-config coverage entries for the new GGUF encoder module types. |
| src/mobius/tasks/_feature_extraction.py | Allows task-defined output naming; adds GGUFEncoderFeatureExtractionTask for GGUF pooling ABI. |
| src/mobius/tasks/init.py | Exports/registers GGUFEncoderFeatureExtractionTask and task-name mapping. |
| src/mobius/models/gguf_encoders.py | New specialized encoder model graphs (RoPE/ALiBi variants + pooling implementation). |
| src/mobius/models/init.py | Exports new GGUF encoder model classes. |
| src/mobius/integrations/gguf/_upstream_data/llamacpp_pin.json | Adds pinned tensor inventories for the newly promoted encoder architectures. |
| src/mobius/integrations/gguf/_tensor_mapping.py | Adds GGUF→HF tensor stem mappings for the new encoders. |
| src/mobius/integrations/gguf/_specialized_encoders_test.py | New tests validating config transforms, closure, mapping, pooling ABI, and graph properties. |
| src/mobius/integrations/gguf/_config_mapping.py | Adds specialized-encoder postprocess resolving optional tensor-selected variants and pooling type. |
| src/mobius/integrations/gguf/_builder.py | Wires specialized closure validation + dispatches to the GGUF encoder feature-extraction task. |
| src/mobius/integrations/gguf/_arch_registry.py | Promotes four encoder architectures to supported graph import with specialized registry entries. |
| src/mobius/integrations/gguf/_arch_registry_test.py | Updates expected supported counts and registry/task resolution assertions. |
| src/mobius/_registry.py | Registers new module types to models + default GGUF encoder feature-extraction task. |
| src/mobius/_configs/_base.py | Adds encoder-specific config fields needed for specialized GGUF variants. |
| docs/api/build_from_gguf.md | Updates support matrix and docs to reflect promoted encoder architectures and quantized-import policy. |
Suppressed comments (1)
src/mobius/integrations/gguf/_builder.py:3831
- For specialized encoder GGUF architectures,
build_from_gguf()validatestaskoverrides against only"feature-extraction", but the code later auto-selectsGGUFEncoderFeatureExtractionTaskand the task is publicly registered as"gguf-encoder-feature-extraction". As a result, passingtask="gguf-encoder-feature-extraction"will be rejected even though it’s the intended task for these architectures.
expected_task = (
"t5-text-encoding" if model_type == "t5encoder" else "feature-extraction"
)
if task is not None and task != expected_task:
raise ValueError(
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Implement exact float GGUF import paths for EuroBERT, NeoBERT, dense NomicBERT, and JinaBERT-v2 with strict config and tensor closure validation. Keep quantized import and runtime packaging fail-closed until real artifact evidence exists.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Copilot-Session: c9f552d7-041b-482e-b1a6-7227d980ed5f
Correct NeoBERT per-head QKV packing and interleaved RoPE, use JinaBERT-v2 tanh GELU, reject unsupported pooling, and preserve established GGUF route fingerprints.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Copilot-Session: c9f552d7-041b-482e-b1a6-7227d980ed5f
Apply the initialized repository formatter after the post-rebase validation pass.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Copilot-Session: c9f552d7-041b-482e-b1a6-7227d980ed5f
Implement GGUF NONE/MEAN/CLS pooling through a dedicated encoder task and cast JinaBERT-v2 ALiBi bias to the model dtype for FP16/BF16 graph loading.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Copilot-Session: c9f552d7-041b-482e-b1a6-7227d980ed5f
Use contiguous full-width Q/K/V views from pinned build_qkv and select the first valid sequence token for CLS pooling, including a left-padding value test.\n\nCo-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Copilot-Session: c9f552d7-041b-482e-b1a6-7227d980ed5f
justinchuby
force-pushed
the
justinchuby-special-gguf-architectures
branch
from
August 25, 2026 22:44
0960104 to
780267a
Compare
Apply the repository-pinned Ruff formatter to the Xverse permutation helper added by #635 so all-files lint is idempotent on the specialized branch. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c9f552d7-041b-482e-b1a6-7227d980ed5f Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Promotes four stateless specialized GGUF encoder architectures from the pinned llama.cpp census:
eurobert: pre-RMSNorm split-QKV RoPE encoder with parallel SwiGLUneo-bert: fused contiguous QKV, interleaved RoPE, fused SwiGLU, and CLS poolingnomic-bert(dense only): post-norm RoPE encoder with gated FFNjina-bert-v2: ALiBi encoder with optional Q/K norms, optional second attention norm, and fused/separate GeGLUAll four use a dedicated GGUF feature-extraction task supporting
NONE,MEAN, andCLSpooling. Tensor closure and shape validation are fail-closed. Quantized import is rejected until complete fused/split packed-weight routes exist; runtime packaging remains deferred pending representative real-weight evidence.Authoritative evidence
8d9af256337d1a501250f9bbf4c0859a654bddd6EuroBERT/EuroBERT-210m@39b51e15dd1f1a06f58b5cbf6a8a188cec60bd0echandar-lab/NeoBERT@5424c8efeea6491b151d62dee55a752165407430nomic-ai/nomic-embed-text-v1.5@e9b6763023c676ca8431644204f50c2b100d9aabjinaai/jina-embeddings-v2-small-en@44e7d1d6caec8c883c2d4b207588504d519788d0Tests cover tiny graph construction, pinned config transforms, exact tensor mapping/closure, contiguous NeoBERT QKV and interleaved RoPE, tanh GeGLU, FP16/BF16 Jina ALiBi graph loading, pooling ABI, left-padded CLS value selection, and float-only quantization policy.
Residual deferred architectures
jina-bert-v3: released artifacts require adapter handling; MoE scheduling is not coverednomic-bert-moe: encoder MoE schedule/experts remain unsupportedtalkie: dedicated gain/skip/logit-scale graph and value transforms requiredwavtokenizer-dec: requires a code-token-to-ISTFT-feature task and waveform processorrwkv6,rwkv6qwen2,rwkv7,arwkv7: token-shift plus matrix-state ABI is not KV/Mamba statebailingmoe3: mixed KDA convolution/matrix state plus MLA cachedeepseek4: compressed cache/history/rollback ABI is not represented by the fallback graphpockettts,qwen3tts: primary GGUFs require mandatory sidecar bundles and non-text orchestrationValidation
lintrunner f --output oneline --all-files: success and zero worktree changes after commit4e1059eb3b2c9cd624c5f1e58a5948fb7e6af6453ea64ff2(Add conventional dense GGUF architectures #635)4e1059eb6516ba40d7e059a799ab47d1f420835eThe post-format monolithic shared rerun was attempted twice, but the host dropped to 100–115 MiB free and pytest could no longer create temporary files (
FileNotFoundError: No usable temporary directory; OpenMP/tmpallocation failures). The formatting-only affected tests pass, and the identical shared suite passed immediately before the semantic no-op formatting commit.Waivers
model_typeroute. Pinned HF-to-GGUF config semantics and synthetic ORT parity are covered by_specialized_encoders_test.py; misleading generictest_model_idclaims are intentionally omitted.keep_quantized=True.kimi_k3andlfm2_moealready have test IDs without YAML cases on Add conventional dense GGUF architectures #635; the shared run excludes those two cases and their aggregate sentinel. Specialized coverage checks pass.3ca3…produced vsb999…pinned) that reproduces unchanged on untouched Fix atomic GGUF runtime CLI publication #630; this branch preserves the baseline route fingerprint and does not update evidence.