Validate real PLaMo2 GGUF import - #617
Merged
Merged
Conversation
Performance Comparison
|
justinchuby
force-pushed
the
justinchuby-plamo2-gguf
branch
from
August 25, 2026 17:31
64aa8fb to
7743ca4
Compare
Accept the exact legacy metadata emitted by the public PLaMo2 F32 conversion, add an opt-in full-artifact round-trip test, and record immutable provenance plus the observed ORT and ORT GenAI runtime blockers. L4/L5 remain explicitly deferred because cached decode diverges from the independent reference. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
justinchuby
force-pushed
the
justinchuby-validate-real-plamo2
branch
from
August 25, 2026 17:39
1315932 to
52fa76a
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR tightens Mobius’ GGUF import validation for the real pinned public PLaMo2 1B F32 artifact by accepting legacy tokenizer/head-count metadata variations seen in that conversion, while adding targeted tests and an immutable evidence record to document the current runtime limitations.
Changes:
- Accept legacy scalar
head_count/head_count_kvGGUF metadata for PLaMo2 by inferring the per-layer schedule from tensor-family presence. - Allow (and explicitly validate) the legacy inert tokenizer
tokenizer.ggml.pre="default"formodel="plamo2", while still deferring tokenizer materialization. - Add an explicit opt-in, slow integration round-trip test for importing/saving/reloading the pinned 5.16GB GGUF, and commit a detailed evidence JSON record plus updated test-case notes.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/integration_test.py | Adds an env-gated slow integration test that imports the pinned real PLaMo2 GGUF, round-trips through ModelPackage.save/load, and asserts key ABI expectations. |
| testdata/evidence/causal-lm/plamo2-1b-real.json | Adds an immutable evidence record documenting exact pinned inputs, tensor comparisons, ORT parity results, and current ORT GenAI blockers. |
| testdata/cases/causal-lm/plamo2-1b.yaml | Updates notes/skip rationale to point at the new evidence and clarify why L4/L5 remain deferred. |
| src/mobius/integrations/gguf/_tokenizer.py | Accepts the legacy PLaMo2 tokenizer.ggml.pre="default" value (validated) while keeping tokenizer routing deferred. |
| src/mobius/integrations/gguf/_tokenizer_test.py | Adds a unit test verifying the PLaMo2 legacy pre="default" path is accepted and remains deferred. |
| src/mobius/integrations/gguf/_builder.py | Extends strict PLaMo2 GGUF tensor contract validation to handle legacy scalar head-count metadata by inferring the layer schedule from tensor presence. |
| src/mobius/integrations/gguf/_builder_test.py | Adds a unit test covering legacy scalar head-count metadata inference for PLaMo2. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
justinchuby
added a commit
that referenced
this pull request
Aug 25, 2026
## Summary - restore PLaMo2's published `1e4` local RoPE base for the legacy public GGUF profile whose metadata incorrectly declares `1e6` - preserve `A_log` and evaluate `-exp(A_log)` in the ONNX graph, including correct HF/GGUF and effectively-tied weight loading - align PLaMo2's public recurrent state ABI with ORT GenAI 0.15.2 and add durable real-model L4/L5 evidence ## Root cause The pinned GGUF declares `plamo2.rope.freq_base=1e6`, while the pinned HF architecture has `full_attention_idx=[]` and uses `rope_local_theta=1e4` for every attention layer. Trusting the contradictory GGUF field caused the first divergence in layer 1 attention during cached decode; later recurrent layers amplified it. Mamba state update, convolution orientation, attention QKV/QK norm, residual ordering, cache indexing, fusion, and rollback/replay were not the source. ## Real-model validation Pinned revisions: - HF `pfnet/plamo-2-1b`: `92c75fd6eea9018bcb9c33ee8921589febe071fa` - F32 GGUF: `184326afcef7232d94df09a33279598255890325` Raw ORT 1.29 CPU after the fix: - prefill max abs error: `5.34057617e-05` - first cached decode max abs error: `2.47955322e-05` - maximum over 19 cached steps: `4.62532043e-05` - exact 20/20 greedy token match - rollback/replay exact ORT GenAI 0.15.2 source independently discovers sparse KV indices and sparse `conv_state` indices, then derives the paired `recurrent_state` names. After renaming PLaMo2's semantic SSM state to that public ABI, a CPU-specialized package with all eight attention layers lowered to GQA loaded and generated the exact verified 20-token continuation from prompt IDs `[1, 6721]`. Unspecialized `Attention` plus recurrent state is still rejected during package config generation. The immutable tokenizer source was audited against the GGUF: all 100,000 ordered tokens, token types, float32 scores, and special IDs match exactly, with no normalizer or pre-tokenizer. Text tokenization remains deferred because ORT GenAI requires `tokenizer.json`, and standard Unigram byte fallback is not identity-preserving: the official tokenizer encodes literal `<0xFA>` as `[60, 48, 120, 5109, 62]`, while the standard tokenizer emits byte token `[250]`. ## Branch reconstruction Parent #617 was admin squash-merged as `79efe2c4512a40900ac5af5e794d057ea39c8f5b`. This branch was reconstructed directly on that commit and retains only the two reviewed #620 commits, preserving #617 as the explicit parent while removing its duplicated pre-squash history. ## Tests - focused PLaMo2 raw ORT state/rollback, GGUF, OGA metadata, evidence, tokenizer, and graph tests: 53 passed - PLaMo2 synthetic parity/weight selection: 2 passed, 1 expected skip - YAML model-case schema validation: 252 passed - broad non-integration suite: 7,525 passed, 54 skipped, 1 subtest passed - initialized pinned lintrunner tools; formatting and lint clean - GPT-5.6-sol medium post-rebase review: no actionable findings - durable real-model L4 passed and L5 matched 20/20 tokens; the removed 19 GB validation cache was not redownloaded --------- Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Historical evidence
testdata/evidence/causal-lm/plamo2-1b-real.jsonMOBIUS_PLAMO2_REAL_GGUFPost-rebase validation
c98ee9eafter Add dedicated PLaMo2 model and GGUF import #612 was squash-mergedlintrunnertools: clean