Skip to content

[QNN EP] Handle broadcast Gemm C with FC and Add - #741

Merged
qti-yuduo merged 1 commit into
onnxruntime:mainfrom
qti-yuduo:fix/gemm-column-broadcast-c
Aug 21, 2026
Merged

qti-yuduo merged 1 commit into
onnxruntime:mainfrom
qti-yuduo:fix/gemm-column-broadcast-c

Conversation

@qti-yuduo

@qti-yuduo qti-yuduo commented Aug 17, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Lower valid Gemm C broadcasts that cannot be represented as a QNN FullyConnected bias to FullyConnected + ElementWiseAdd.
  • Share the FC-compatible bias-shape rule and keep the absorbed-Reshape selector aligned with the builder.
  • Add direct, MatMulAddFusion, and selector regression coverage.

Root cause

ORT MatMulAddFusion can produce Gemm with C=[M,1] from MatMul followed by a column-broadcast Add. ONNX permits C to broadcast to [M,N], but QNN FullyConnected can consume only one bias value per output channel. Validation rejected C before the existing FC + Add lowering could run.

The fix keeps scalar C for N=1, [N], and [1,N] on the direct FC path. Other valid broadcasts use a separate Add; block-quantized Gemm remains strict.

@qti-yuduo
qti-yuduo force-pushed the fix/gemm-column-broadcast-c branch from 36f8abc to 4262793 Compare August 17, 2026 21:37
@qti-yuduo qti-yuduo changed the title [QNN EP] Handle Gemm column-broadcast C [QNN EP] Handle broadcast Gemm C with FC and Add Aug 17, 2026
@qti-yuduo
qti-yuduo force-pushed the fix/gemm-column-broadcast-c branch from 4262793 to 8e6068f Compare August 17, 2026 22:17
@qti-yuduo
qti-yuduo marked this pull request as ready for review August 17, 2026 22:19

@qti-ashwshan qti-ashwshan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! we are correctly gating ExplictOpCheck to let decomposable bias shapes flow through to the existing FC+Add path instead of rejecting them outright.

@qti-yuduo
qti-yuduo merged commit c2c3a4a into onnxruntime:main Aug 21, 2026
44 checks passed
@qti-yuduo
qti-yuduo deleted the fix/gemm-column-broadcast-c branch September 8, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants