[QNN EP] Sign-extend sub-byte weights when folding a constant DequantizeLinear

[qti-yuduo]

Problem

UnpackInitializerData() expands a sub-byte initializer to one byte per element and masks off the unused high bits (UnpackInt4ToInt8(): dst[i] &= 0x0F). That mask works around a QNN INT4 accuracy bug, so it has to stay.

GetEffectivelyConstantTensorBytes() hands those bytes to callers that read them as int8_t. FoldConstantDequantizeLinear() → DequantizePerChannel() therefore decodes every negative 4-bit weight as q + 16 (-1 → 15, -7 → 9), leaving the folded FLOAT_32 result with no negative values at all. The bit width is unrecoverable there: CreateMapQuantize collapses INT4/INT2 into SFIXED_POINT_8 on non-GPU backends.

Fix

New utils::SignExtendUnpackedSubByteData() for INT4/INT2, over the existing Int4x2::SignExtendLower4Bits / Int2x4::SignExtendLower2Bits. It masks before extending, so it is well-defined for any input byte and idempotent. GetEffectivelyConstantTensorBytes() calls it on the initializer branch, which also covers the same defect in BatchNormalization's folded-scale path.

UINT4/UINT2 need nothing — the masked byte is already the value. The folded-constant branch needs nothing either: those bytes are fp32 or 8/16/32-bit, since QuantizeData has no sub-byte case.

Deliberately not fixed by dropping the mask in UnpackInt4ToInt8 (re-exposes the QNN bug across unaudited call sites), nor by teaching DequantizePerChannel about ONNX types (misses BatchNormalization).

Where it shows up

A w4a16 Qwen3 0.6B QDQ model: 32 KV-producing ops take the float fallback — DQ input uint16, Q output uint8, so CheckQDQNodes() declines the node unit — and fold their weights through this path. The fallback is correct behaviour; only its int4 decode was wrong.

Validation

Check	Before	After
16 SpinQuant R3 Hadamard weights (analytically ±1/√128 = ±0.0883883)	all positive; spurious 0.113642 = 0.0883883 × 9/7, i.e. −7 read as 9	exactly ±0.0883883
PSNR vs AIMET QuantSim, min/mean	prompt −7.66 dB	prompt 14.48 / 26.59 dB, token 16.73 / 34.30 dB
Windowed wikitext perplexity, 2032 scored tokens	baseline 223.98 (unquantized ref 203.47)	208.72, ΔCE −0.0706 ± 0.0270

All 57 part2 outputs match; inputs byte-identical across arms. DLCs are otherwise structurally identical — same op and tensor counts, same dtype histograms — so the change is numeric only.

Perplexity caveats: SpinQuant R2+R3 only (no AdaScale/SeqMSE), so the absolute PPL is not recipe-faithful, and the sweep is windowed (disjoint 128-token windows, empty KV). All arms consume byte-identical windows, so the comparison holds.

Tests

• Convf32_PerChannelInt4DQConstWeight_SignRegression (conv_test.cc, CPU and HTP): folds a per-channel INT4 DQ with weights spanning −8..7 against the CPU EP. Both variants fail without the fix, while the neighbouring int8 chain tests keep passing.
• qnn_utils_test.cc: every INT4 and INT2 value, the types the helper must leave alone, the empty span.
• qdq_constant_folding_test.cc (new): mocked INT4 / UINT4 / UINT8 initializers plus the not-found path.

627 tests pass. Negative control: neutering the sign extension fails exactly the 2 graph tests and 3 sign-sensitive unit tests, nothing else.

Notes

• [QNN EP] Sign-extend sub-byte weights when pre-dequantizing a per-channel integer-op weight #729 is stacked on this and fixes the same class of defect in the DQ-IntegerOp fusion helper (a latent path). This PR stands alone.
• Possible follow-up: UnpackZeroPoints() hand-rolls the same unmasking inline, and [QNN EP] Sign-extend sub-byte weights when pre-dequantizing a per-channel integer-op weight #729 adds a third consumer. Converging them — or letting UnpackInitializerData() sign-extend on request, where the element type is already in hand — would remove the recurring trap for the ~80 call sites sharing the implicit "bytes are masked" convention.

[yath1]

Very nice find! LGTM.