Skip to content

[Hexagon:Feature] Support Qwen3-VL with symmetric and asymmetric block-wise quantization - #4774

Open
yydhYYDH wants to merge 4 commits into
alibaba:masterfrom
yydhYYDH:feature/qwen3-vl-hexagon-blockwise-quant
Open

[Hexagon:Feature] Support Qwen3-VL with symmetric and asymmetric block-wise quantization#4774
yydhYYDH wants to merge 4 commits into
alibaba:masterfrom
yydhYYDH:feature/qwen3-vl-hexagon-blockwise-quant

Conversation

@yydhYYDH

Copy link
Copy Markdown
Contributor

Description

This change adds two main capabilities to the Android Hexagon backend:

  • Qwen3-VL Vision model execution;
  • symmetric and asymmetric block-wise INT4/INT8 quantization for Vision and language-model weights.

Performance

llm_bench -p N -n 128 -rep 3 on OnePlus 13 / Snapdragon 8 Elite / Hexagon v79, tokens/s:

Weight format Prefill 128 Prefill 256 Prefill 512 Decode avg
INT4 asymmetric block64 893.07 1257.30 1417.40 18.06
INT8 symmetric block64 631.46 713.35 732.10 20.86
INT8 asymmetric block64 597.42 691.14 730.40 11.57
FP16 350.42 502.68 609.72 4.02

Module

Hexagon

Type

  • Feature
  • Bugfix
  • Perf
  • Refact
  • Style
  • Doc
  • Test
  • Chore

Checklist

  • Commit message follows [Module:Type] Description format
  • Code compiles without errors
  • Tested on relevant platform(s)
  • No unrelated format or style changes included

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant