Skip to content

Clean up GGUF review findings - #633

Merged
justinchuby merged 1 commit into
mainfrom
justinchuby-gguf-review-cleanup
Aug 25, 2026
Merged

Clean up GGUF review findings#633
justinchuby merged 1 commit into
mainfrom
justinchuby-gguf-review-cleanup

Conversation

@justinchuby

Copy link
Copy Markdown
Member

Summary

Exact base: 2db9d33debdc254d879a51b14434c9a81c230f4f

Exact head: 4541ed2bc9d2ab4227484510b4a85b2d9113eb25

Reconstructed original 17-item low-priority tranche

The persisted audit retained only the totals, so this list was reconstructed
from the live threads and current source. All but the already-fixed #596
comment are addressed in this PR.

PR Comment Disposition
#550 3837144656 Implemented: correct tuple return annotation
#552 3837389183 Implemented: multichannel waveform shape docs
#559 3837716261 Implemented: name-based cache assertions
#573 3854308350 Implemented: one final GGUF hash, with integrity regression coverage
#574 3854345521 Implemented: TensorRole | None typing
#574 3854345597 Implemented: removed obsolete verdict filtering
#577 3854472859 Implemented: documented SSM sequence length
#578 3843112613 Implemented: documented F64 passthrough
#579 3854518332 Implemented: generalized fused-projection error
#580 3854576300 Implemented: removed brittle node counts
#583 3854681386 Implemented: documented conditional draft outputs
#587 3854816872 Implemented: corrected MTP output contract docs
#596 3846110059 Already fixed on base: unambiguous GQA bias comment
#600 3855174281 Implemented: metadata-count-only MTP error
#607 3855776048 Implemented: stable route-field assertions
#607 3855776086 Implemented: public tensor iterator
#609 3856486136 Implemented: fail-closed LM-head comment

Current unresolved-thread disposition

This covers all 48 Copilot threads returned by the reproducible #600-#630
query. The one human #623 thread is excluded.

PR Comment Current-main disposition and evidence
#600 3855174177 Already fixed by #629: package cycle and reserved-sidecar validation
#600 3855174238 Already fixed by #629: explicit MTP sidecar naming/loading
#600 3855174281 Implemented here: error no longer invents an observed block count
#602 3848155961 Outside exact stack; already fixed: top-level expert_dtype is classified before early return
#602 3848155983 Outside exact stack; still-valid behavioral block-quant validation, unchanged
#602 3848156000 Outside exact stack; still-valid truncated-read behavioral finding, unchanged
#602 3848156022 Outside exact stack; still-valid descriptor byte/dtype validation, unchanged
#602 3848156040 Outside exact stack; still-valid expert-bank payload validation, unchanged
#603 3855249765 Already fixed by #630: runtime preflight preserves shard sets
#603 3855249840 Already fixed by #630: success output follows durable runtime publication
#604 3855343082 Implemented here: graph-only MTP persistence distinguished from runtime rejection
#604 3855343131 Already fixed by #630: runtime success messages are atomic
#607 3855776001 Implemented here: missing generation golden skips before provenance read
#607 3855776048 Implemented here: only stable route fields are asserted
#607 3855776086 Implemented here: tensor count uses tensor_items_raw()
#608 3856079371 Still-valid behavioral cache-symlink containment finding; unchanged
#608 3856079415 Still-valid behavioral lowercase-digest validation finding; unchanged
#609 3856486136 Implemented here: comment matches value-preserving policy
#610 3855541683 Already fixed by #628: Falcon bias precedence is explicit
#610 3855541761 Already fixed by #628: CTRL tiny config exercises projection biases
#611 3856595840 Implemented here: runtime test resolves the distribution providing the module
#612 3855677383 Already fixed by #625: supported-version endianness detection
#612 3855677427 Implemented here: shared INT64_MAX sentinel
#612 3855677460 Implemented here: shared PLaMo2 width inference
#612 3855677486 Implemented here: accepted PLaMo2 activation spellings are explicit
#613 3855845678 Implemented here: canonical issue URL
#613 3855845757 Implemented here: Mamba-1 function-registration docs
#613 3855845806 Implemented here: test expects the canonical issue URL
#614 3855988545 Implemented here: removed stale Nemotron-H divergence comments
#614 3855988597 Already fixed by #628: zero-head geometry raises actionable ValueError
#615 3856082290 Already fixed by #626: dense GraniteHybrid bias closure
#618 3856729330 Implemented here: required routes filter ORT GenAI evidence
#618 3856729409 Implemented here: env-selected runtime version is authoritative
#618 3856729490 Stale/N/A: PR-description-only matrix claim; repository workflow claims one pinned version
#618 3856729563 Implemented here: schema tail restored to normal indentation
#619 3856342484 Already fixed on base: Kimi Linear uses /issues/605
#619 3856342532 Already fixed by #628: config rejects convolution kernels below 2
#619 3856342580 Already fixed by #628: GGUF contract rejects convolution kernels below 2
#620 3855717041 Already fixed by #627: tied LM-head-only checkpoints are retained
#621 3856722324 Already fixed by #628: Kimi-K3 required metadata is complete
#623 3855931210 N/A to current main: comment belongs to open, unmerged #623
#623 3855939302 N/A to current main: comment belongs to open, unmerged #623
#623 3855939358 N/A to current main: comment belongs to open, unmerged #623
#623 3855939394 N/A to current main: comment belongs to open, unmerged #623
#624 3856777152 Implemented here: runtime compatibility reuses the emitted model type
#625 3857049733 Implemented here: unsupported header reports both endian candidates
#629 3857313182 Newer post-audit behavioral sidecar-symlink cleanup finding; unchanged
#629 3857313251 Newer post-audit cross-platform path-safety finding; unchanged

Validation

  • affected GGUF/package/ORT GenAI/model/schema tests: 1,040 passed
  • broad non-integration suite: 7,851 passed, 56 skipped, 1 subtest passed
  • generated GGUF docs checks: 7 passed
  • initialized lintrunner; full lint/format passed
  • GPT-5.6 Sol medium review: one integrity finding fixed; re-review found no significant issues

Resolve the still-valid low-priority typing, documentation, test robustness, error-message, maintainability, and reuse-performance findings left across the GGUF PR stack. Preserve the merged behavioral contracts while reducing GGUF save hashing to one final integrity check.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchuby@users.noreply.github.com>
@justinchuby
justinchuby requested review from a team and a lite review from Copilot August 25, 2026 21:40
@justinchuby
justinchuby merged commit 8513774 into main Aug 25, 2026
15 of 24 checks passed
@justinchuby
justinchuby deleted the justinchuby-gguf-review-cleanup branch August 25, 2026 21:40
@github-actions

Copy link
Copy Markdown

Performance Comparison

Comparing 2db9d334541ed2

Model Metric Baseline Current Delta
bert (feature-extraction) model_size_bytes 359 KB 359 KB +0.0%
bert (feature-extraction) num_nodes 68 68 +0.0%
falcon model_size_bytes 364 KB 364 KB +0.0%
falcon num_nodes 66 66 +0.0%
gemma2 model_size_bytes 428 KB 428 KB +0.0%
gemma2 num_nodes 105 105 +0.0%
gpt2 model_size_bytes 388 KB 388 KB +0.0%
gpt2 num_nodes 54 54 +0.0%
llama model_size_bytes 425 KB 425 KB +0.0%
llama num_nodes 60 60 +0.0%
llama (static-cache) model_size_bytes 425 KB 425 KB +0.0%
llama (static-cache) num_nodes 56 56 +0.0%
mamba (ssm-text-generation) model_size_bytes 296 KB 296 KB +0.0%
mamba (ssm-text-generation) num_nodes 94 94 +0.0%
phi3 model_size_bytes 421 KB 421 KB +0.0%
phi3 num_nodes 58 58 +0.0%
phi3 (static-cache) model_size_bytes 421 KB 421 KB +0.0%
phi3 (static-cache) num_nodes 54 54 +0.0%
qwen2 model_size_bytes 425 KB 425 KB +0.0%
qwen2 num_nodes 60 60 +0.0%
qwen2 (static-cache) model_size_bytes 425 KB 425 KB +0.0%
qwen2 (static-cache) num_nodes 56 56 +0.0%
qwen3_5_moe (hybrid-text-generation) model_size_bytes 506 KB 506 KB +0.0%
qwen3_5_moe (hybrid-text-generation) num_nodes 265 265 +0.0%
qwen3_5_text (hybrid-text-generation) model_size_bytes 458 KB 458 KB +0.0%
qwen3_5_text (hybrid-text-generation) num_nodes 127 127 +0.0%
qwen3_5_vl (hybrid-qwen-vl) model_size_bytes 977 KB 977 KB +0.0%
qwen3_5_vl (hybrid-qwen-vl) num_nodes 429 429 +0.0%
t5 (seq2seq) model_size_bytes 836 KB 836 KB +0.0%
t5 (seq2seq) num_nodes 176 176 +0.0%
whisper (speech-to-text) model_size_bytes 1008 KB 1008 KB +0.0%
whisper (speech-to-text) num_nodes 128 128 +0.0%

No performance regressions.

@github-actions

Copy link
Copy Markdown

🏗️ Architecture Diff

Comparing 2db9d334541ed2

Model Sub-model Changes Status
bert (feature-extraction) model 0
falcon model 0
gemma2 model 0
gemma4 (gemma4) decoder 0
gemma4 (gemma4) embedding 0
gemma4 (gemma4) vision_encoder 0
gemma4_text model 0
gpt2 model 0
llama model 0
llama (static-cache) model 0
mamba (ssm-text-generation) model 0
phi3 model 0
phi3 (static-cache) model 0
qwen model 0
qwen (static-cache) model 0
qwen2 model 0
qwen2 (static-cache) model 0
qwen2_moe model 0
qwen2_moe (static-cache) model 0
qwen3 model 0
qwen3 (static-cache) model 0
qwen3_5_moe (hybrid-text-generation) model 0
qwen3_5_text (hybrid-text-generation) model 0
qwen3_5_vl (hybrid-qwen-vl) decoder 0
qwen3_5_vl (hybrid-qwen-vl) embedding 0
qwen3_5_vl (hybrid-qwen-vl) vision_encoder 0
qwen3_moe model 0
qwen3_moe (static-cache) model 0
qwen3_next (hybrid-text-generation) model 0
t5 (seq2seq) decoder 0
t5 (seq2seq) encoder 0
whisper (speech-to-text) decoder 0
whisper (speech-to-text) encoder 0

No architecture changes detected.


Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR cleans up remaining low-priority review findings from the GGUF PR stack while preserving the already-merged behavioral fixes. The changes primarily tighten documentation, improve test robustness, and refine GGUF reuse/package integrity checks so multi-GB sources are hashed once at the final verification gate.

Changes:

  • Harden GGUF reuse/publishing flows and related tests (single final SHA-256 verification; cheaper identity checks during staging).
  • Improve test robustness by removing brittle assertions (node counts, full-route equality) and tightening runtime-evidence filtering.
  • Documentation/typing cleanups across tasks, GGUF contract validation, and model components.

Reviewed changes

Copilot reviewed 29 out of 30 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/yaml_schema_test.py Filter required runtime routes to ORT GenAI evidence only.
tests/synthetic_parity_test.py Remove stale divergence commentary for newly registered models.
tests/ort_genai_e2e_test.py Make expected ORT GenAI version env override default to pinned value.
tests/gguf_small_model_runtime_integration_test.py Stabilize import-route assertions; improve ORT GenAI version resolution.
tests/e2e_golden_test.py Avoid private GGUF reader internals; make tensor/qtype checks API-driven.
testdata/cases/schema.json Restore schema tail formatting/indentation consistency.
src/mobius/tasks/_ssm_causal_lm.py Clarify sequence_length semantics for Mamba-1 vs Mamba2.
src/mobius/tasks/_dflash.py Update DFlash outputs contract docs for conditional draft logits/hidden.
src/mobius/tasks/_causal_lm.py Fix canonical issue URL format for runtime-support metadata.
src/mobius/tasks/_cache_utils.py Expand function-registration docs to include Mamba-1 layers.
src/mobius/models/t5_test.py Remove brittle node-count assertions from graph contract test.
src/mobius/models/qwen3_tts_tokenizer.py Correct waveform shape docs for multichannel audio.
src/mobius/models/plamo2.py Replace hard-coded INT64 sentinel with shared INT64_MAX.
src/mobius/models/glm_moe_dsa_test.py Make cache IO selection robust by matching port name prefixes.
src/mobius/models/deepseek_v4_test.py Correct helper return typing for layer runner.
src/mobius/integrations/ort_genai/auto_export.py Reuse emitted model type directly for compatibility metadata.
src/mobius/integrations/onnx_genai/inference_metadata.py Clarify MTP proposer execution/doc distinctions for logits vs hidden.
src/mobius/integrations/gguf/_reuse.py Shift to cheap identity checks during staging; hash once at final verify gate.
src/mobius/integrations/gguf/_reader_test.py Add regression test for endian-aware unsupported-header reporting.
src/mobius/integrations/gguf/_quant_registry.py Clarify lm_head quantization preservation policy docs.
src/mobius/integrations/gguf/_preflight.py Document F64 passthrough alongside other float types.
src/mobius/integrations/gguf/_mtp.py Improve MTP contract error message to reference exact metadata key/value.
src/mobius/integrations/gguf/_mtp_test.py Update MTP error-message matching to new key-based wording.
src/mobius/integrations/gguf/_header.py Improve unsupported GGUF version error to report both endian candidates.
src/mobius/integrations/gguf/_config_mapping.py Refactor PLaMo2 attention width inference into shared helper.
src/mobius/integrations/gguf/_builder.py Tighten TensorRole typing; reuse shared PLaMo2 width inference; clarify errors.
src/mobius/integrations/gguf/_builder_test.py Add coverage for “hash once” reuse behavior + final publish-time verification.
src/mobius/integrations/gguf/_arch_registry_test.py Remove obsolete verdict filtering in expected rows.
docs/cli_reference.md Clarify graph-only vs runtime packaging behavior for MTP sidecars.
.agents/skills/attention-optimization/SKILL.md Update kernel reference link to a stable GitHub URL.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +341 to +349
def _source_identity(path: Path) -> tuple[int, int, int, int, int]:
source_stat = path.stat()
return (
source_stat.st_dev,
source_stat.st_ino,
source_stat.st_size,
source_stat.st_mtime_ns,
source_stat.st_ctime_ns,
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants