[cuTile] Add v3 cuTile backends for 8 Liger ops - #1321
Open
xjmxyt wants to merge 2 commits into
Open
Conversation
Add cuTile (nvidia-cuda-tile) backends, dispatched via LIGER_KERNEL_IMPL=cutile, for eight ops — each drops into the existing LigerXxxFunction autograd interface so the backend swap is transparent: rms_norm, swiglu (SiLUMul), dyt, softmax, poly_norm, fused_add_rms_norm, fused_linear_cross_entropy, grpo_loss (token- and sequence-level importance sampling, GRPO/CISPO/SAPO/VESPO) Correctness is exercised by the existing transformer suites run under LIGER_KERNEL_IMPL=cutile (which routes each op to its cuTile implementation), and test_cutile_backend.py verifies that routing. Notable correctness fixes made while validating on B200 (sm_100): - grpo_loss: round BLOCK_N up to a power of 2 (ct.arange requires it) and flatten the rank-2 vllm_is_ratio to 1-D for the flat gather — both only bit non-power-of-2 vocab sizes. - fused_add_rms_norm: cast the LLAMA/NONE Y and dX stores back to the output dtype, so an fp32 norm weight with bf16 activations no longer fails to compile. Benchmarks: Triton-vs-cuTile speed+memory data on NVIDIA B200 for all eight ops, plus a benchmark-harness fix so multiple Liger providers in one script are tagged distinctly (LIGER_BENCH_PROVIDER_TAG rewrites the leading "liger" of each provider). Note: tvd and fused_neighborhood_attention cuTile ports were prototyped but are NOT included — the open-source tileiras 13.3.36 miscompiles their kernels; they fall back to the Triton implementations until the compiler fix ships. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
xjmxyt
changed the title
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
xjmxyt
changed the title
Contributor
Author
|
@vaibhavjindal @Tcc0403 Could you help to review this PR? Thanks a lot. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add cuTile (nvidia-cuda-tile) backends, dispatched via LIGER_KERNEL_IMPL=cutile, for eight ops — each drops into the existing LigerXxxFunction autograd interface so the backend swap is transparent:
rms_norm, swiglu (SiLUMul), dyt, softmax, poly_norm,
fused_add_rms_norm, fused_linear_cross_entropy,
grpo_loss (token- and sequence-level importance sampling, GRPO/CISPO/SAPO/VESPO)
Correctness is exercised by the existing transformer suites run under LIGER_KERNEL_IMPL=cutile (which routes each op to its cuTile implementation), and test_cutile_backend.py verifies that routing.
Benchmarks: Triton-vs-cuTile speed+memory data on NVIDIA B200 for all eight ops
Summary
Testing Done
make testto ensure correctnessmake checkstyleto ensure code stylemake test-convergenceto ensure convergence