Skip to content

Vulkan:Speed: Speed up vulkan for corp-matrix, complete segment ops - #4575

Open
jxt1234 wants to merge 1 commit into
alibaba:masterfrom
jxt1234:feature/vulkan_opt
Open

Vulkan:Speed: Speed up vulkan for corp-matrix, complete segment ops#4575
jxt1234 wants to merge 1 commit into
alibaba:masterfrom
jxt1234:feature/vulkan_opt

Conversation

@jxt1234

@jxt1234 jxt1234 commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator

segment Vulkan buffer 2194.10 tok/s 86.32 tok/s
non-segment Vulkan buffer 1100 tok/s 55 tok/s

Description

Module

Type

  • Feature
  • Bugfix
  • Perf
  • Refact
  • Style
  • Doc
  • Test
  • Chore

Checklist

  • Commit message follows [Module:Type] Description format
  • Code compiles without errors
  • Tested on relevant platform(s)
  • No unrelated format or style changes included

@jxt1234
jxt1234 force-pushed the feature/vulkan_opt branch from 9a2fc16 to 20ede0c Compare June 29, 2026 02:49
@wangzhaode

Copy link
Copy Markdown
Collaborator

Thanks for the substantial Vulkan buffer optimization. The reported gains (prefill 1100 → 2194.10 tok/s, decode 55 → 86.32 tok/s) look promising.

Could you please make a few updates before we merge?

  1. Please rebase onto the latest master and resolve the current conflicts.
  2. Please keep the shader-generation changes focused. The current diff changes subprocess.run back to os.popen in three makeshader.py files and also includes changes under skills/. Please revert these unless they are required for this optimization, or briefly explain why they are needed.
  3. Please add the basic benchmark context (device/GPU, model, quantization, and build flags) together with a simple CPU vs Vulkan correctness result.
  4. Please confirm that changing the MNN_VULKAN_IMAGE default to OFF and unconditionally enabling MNN_GPU_RECORD_BATCH are intentional. The embedded SPIR-V size grows from about 1.41 MB to 1.94 MB (about +527 KB), so a brief binary-size note would also help.

No exhaustive device matrix is required; one representative real-device correctness/performance result is enough. Thanks!

@wangzhaode wangzhaode added the awaiting contributor Waiting for contributor to address review comments or rebase label Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting contributor Waiting for contributor to address review comments or rebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants