[NV] Kimi K2.5 NVFP4 B200 disaggregated TRT-LLM (Dynamo) recipe#2249
[NV] Kimi K2.5 NVFP4 B200 disaggregated TRT-LLM (Dynamo) recipe#2249richardhuo-nv wants to merge 2 commits into
Conversation
…LLM (Dynamo) recipe
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29528243378 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29528755773 |
3 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29528755773 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29528755773 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29528755773 |
1 participant
[NV] kimik2.5-fp4-b200-dynamo-trt: Kimi K2.5 NVFP4 B200 disaggregated TRT-LLM (Dynamo) recipe
Summary
Adds a new multi-node benchmark config
kimik2.5-fp4-b200-dynamo-trt— Kimi K2.5 NVFP4 on B200, disaggregated prefill/decode served via Dynamo + TensorRT-LLM. This is the B200 sibling of the existingkimik2.5-fp4-gb200-dynamo-trtentry, with topologies and engine configs sourced from the NVIDIAsrt-slurmB200 recipes.Scenario: ISL 8192 / OSL 1024, 11 STP (non-MTP) Pareto points.
Topologies
1p4d-dep4-tep81p5d-dep4-tep41p1d-dep4-dep82p1d-dep4-dep83p1d-dep4-dep84p1d-dep4-dep8nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc214025cb6a8992d19fd19eead36065e08ea5301f35),nixlKV transfer, TCP request planeb200-multinode(b200-dgxc-slurm)Changes
benchmarks/multi_node/srt-slurm-recipes/trtllm/kimi-k2.5/disagg/trtllm_dynamo/b200-fp4/— 11 flat recipe YAMLs copied fromsrt-slurm, withmodel.path/model.containerrewritten to the cluster-resolved aliases (kimik2.5-fp4/dynamo-trtllm)configs/nvidia-master.yaml— newkimik2.5-fp4-b200-dynamo-trtentry (all 11 configs, each pointing at its localCONFIG_FILE)runners/launch_b200-dgxc.sh— added adynamo-trt+kimik2.5+fp4branch that clonessrt-slurmand copies the local B200 recipes intorecipes/trtllm/kimi-k25-nvfp4/b200-fp4/perf-changelog.yaml— new changelog entry (selects this config for the sweep)