Repository navigation

Single-node vLLM op-attribution profiling (NVIDIA + AMD): per-kernel op/module, clocks, MoE routing - #3639

Draft

hbarclay wants to merge 54 commits into

mainfrom

hbarclay/profiling

Commits on Sep 28, 2026
- profiling: op-attributed vLLM torch traces for srt-slurm AgentX points
  
  hbarclay
  and
  claude
  committed
  3f6fafb
- profiling: extractor joining device kernels to their launching ops
  
  hbarclay
  and
  claude
  committed
  2c0a89d
- profiling: stack-free replay windows with module and piece markers
  
  hbarclay
  and
  claude
  committed
  b161fff
- profiling: time windows from the end of warmup and replay only as long as they need
  
  hbarclay
  and
  claude
  committed
  d057ab6
- profiling: survive window exports on offload-sized DEP8 hosts
  
  hbarclay
  and
  claude
  committed
  3287eab
- profiling: module markers carry their tensor inputs' shapes and dtypes
  
  hbarclay
  and
  claude
  committed
  d9a2a8f
- profiling: name the Python launcher and vLLM callers of every op-less kernel
  
  hbarclay
  and
  claude
  committed
  7325a8b
- profiling: per-step profile artifact alongside the full one
  
  hbarclay
  and
  claude
  committed
  c08cf57
- profiling: ops are torch ops only; drop the compile-cache overrides
  
  hbarclay
  and
  claude
  committed
  5fcd677
- profiling: keep host headroom for trace buffers on CPU-offload recipes
  
  hbarclay
  and
  claude
  committed
  a4ad296
- profiling: document dispatch, settings, artifacts and attribution
  
  hbarclay
  and
  claude
  committed
  8a46bcf
- profiling: move under inferencex-e2e with the rest of the benchmark tree
  
  hbarclay
  and
  claude
  committed
  3bf0a16
- profiling: count idle DP ranks' dummy forwards as profiler steps
  
  hbarclay
  and
  claude
  committed
  e382166
- profiling: quote-free marker names; parser repairs and fails loudly
  
  hbarclay
  and
  claude
  committed
  979a30e
- profiling: document the quote-free marker formats
  
  hbarclay
  and
  claude
  committed
  9945ed9
- profiling: drop the legacy-trace repair
  
  hbarclay
  and
  claude
  committed
  a6a428e
- profiling: open windows on srt-slurm's logical worker endpoints
  
  hbarclay
  and
  claude
  committed
  ebb34f2
- profiling: second default window in the decode steady state
  
  hbarclay
  and
  claude
  committed
  9cff759
- profiling: anchor windows on aiperf's phases; stop right after the last one
  
  hbarclay
  and
  claude
  committed
  91b7a6b
- Profile steady decode by engine state and stop the replay after the windows
  
  hbarclay
  and
  claude
  committed
  f683664
- Open the prefill profile window at the warmup start
  
  hbarclay
  and
  claude
  committed
  cd7f626
- Find aiperf's controller by its process title to stop the replay
  
  hbarclay
  and
  claude
  committed
  134931f
- Attribute CPU KV-offload memcpys; let a stopped profile replay pass
  
  hbarclay
  and
  claude
  committed
  8353c61
- Record measured node time and the DEP8 c192 warmup-window fault
  
  hbarclay
  and
  claude
  committed
  20fa538
Commits on Sep 29, 2026
- Record every GPU clock over each profiled kernel's lifetime
  
  hbarclay
  and
  claude
  committed
  a1c3c83
- Poll each GPU's clocks from its own thread and stamp both edges of a poll
  
  hbarclay
  and
  claude
  committed
  17e0234
Commits on Sep 30, 2026
- Sample a window's clocks until its GPU work finishes, not its logging
  
  hbarclay
  and
  claude
  committed
  8443c20
- Stop a window's clock sampling on the engines' own profiler stops
  
  hbarclay
  and
  claude
  committed
  0768ded
- Update the DEP8 c192 warmup-window fault note to two of three runs
  
  hbarclay
  and
  claude
  committed
  353a799
Commits on Oct 1, 2026
- Record each profiled step's MoE routing
  
  hbarclay
  and
  claude
  committed
  4a15363
- Merge origin/main: port profile staging to the Python launcher
  
  hbarclay
  and
  claude
  committed
  6b2240a
- Add example decode and prefill step timelines to the profiling README
  
  hbarclay
  and
  claude
  committed
  309d6f7
- Bind the patch's own MoE routing capturer
  
  hbarclay
  and
  claude
  committed
  b2d7497
Commits on Oct 2, 2026
- Add a synthetic-shape profiling mode and AMD clock sampling
  
  hbarclay
  and
  claude
  committed
  398510e
- Support vLLM's V1 GPUModelRunner in the profiling patch
  
  hbarclay
  and
  claude
  committed
  1722096
- Load only the replayed graphs from the capture trace
  
  hbarclay
  and
  claude
  committed
  15108c8
- Give profiled runs host headroom from embedded Mooncake stores too
  
  hbarclay
  and
  claude
  committed
  c308ca5
Commits on Oct 5, 2026
- Override the whole services list for Mooncake headroom
  
  hbarclay
  and
  claude
  committed
  51221f0
- Join HIP graph replays to their capture by stream interleaving
  
  hbarclay
  and
  claude
  committed
  d85322f
- Align HIP graph replays on any number of streams by beam search
  
  hbarclay
  and
  claude
  committed
  6db0880
- Mark launchers in every vllm.third_party and flashinfer module
  
  hbarclay
  and
  claude
  committed
  ce9942c
- List HIP graph nodes at capture so replays join by kernel name
  
  hbarclay
  and
  claude
  committed
  4a20ca1
- Mark modules inside Inductor-compiled pieces
  
  hbarclay
  and
  claude
  committed
  45ff519
- Force keep_graph through CUDAGraph.__init__; mark modules in subgraph wrappers
  
  hbarclay
  and
  claude
  committed
  fb6bff5
- Join HIP graph replays exactly through capture-time stream marks
  
  hbarclay
  and
  claude
  committed
  2340b82
- Raise vLLM 0.31's engine-ready and execute-model timeouts for profiled runs
  
  hbarclay
  and
  claude
  committed
  56010d6
- Name compiled kernels' modules from the weights they read
  
  hbarclay
  and
  claude
  committed
  b43586f
Commits on Oct 6, 2026
- Note why compiled kernels found no weight module
  
  hbarclay
  and
  claude
  committed
  515c0f8
- Mark eager modules in every compilation mode
  
  hbarclay
  and
  claude
  committed
  68fda9d
- Profile graph capture without Python stacks
  
  hbarclay
  and
  claude
  committed
  edc4189
- Mark private and compiled-extension launchers in kernel packages
  
  hbarclay
  and
  claude
  committed
  a774fd2
- Pair a KV-offload copy with the memcpys it was split into
  
  hbarclay
  and
  claude
  committed
  8b6295f
- Note callables the kernel-package launcher hook leaves unwrapped
  
  hbarclay
  and
  claude
  committed
  bf934b6
- Wrap kernel-package code imported from outside its module namespace
  
  hbarclay
  and
  claude
  committed
  3fb59e4

Commits

Commits on Sep 28, 2026

Commits on Sep 29, 2026

Commits on Sep 30, 2026

Commits on Oct 1, 2026

Commits on Oct 2, 2026

Commits on Oct 5, 2026

Commits on Oct 6, 2026