[Klaud Cold] Deprecate GLM-5 / GLM-5.1 benchmark configs / 弃用 GLM-5 / GLM-5.1 基准测试配置#2276
Conversation
Following the MiniMax M2.5/M2.7 (#1874) and gpt-oss-120b (#2101) deprecation pattern: all 31 glm5/glm5.1 config entries (23 nvidia, 8 amd) removed from the active master configs and archived verbatim under configs/deprecated/{nvidia,amd}-glm5-glm5.1-master.yaml; the 17 single-node scripts and the mi355x sglang-disagg multi-node script moved into deprecated/ subdirectories with benchmark_lib.sh source paths corrected for the extra directory level (the #1860 bug class, matching #2101). Launchers, srt-slurm recipe trees, eval thresholds, and perf-changelog history are left untouched, matching both precedents. GLM-5.2 (glm52 speedbench collector, open agentic PRs) is out of scope. 中文:按 MiniMax M2.5/M2.7(#1874)与 gpt-oss-120b(#2101)弃用模式 弃用 GLM-5 / GLM-5.1 基准测试配置——31 个配置条目从活动 master 配置 移除并原样归档至 configs/deprecated/,18 个脚本移入 deprecated/ 子目 录且已修正 benchmark_lib.sh 相对路径;launcher、srt-slurm 配方、评估 阈值与 perf-changelog 历史保持不变,GLM-5.2 不在本次范围内。 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 如需进行 PR 验证,请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
There was a problem hiding this comment.
Looks good — this is a mechanical, precedent-following deprecation (mirrors #1874 and #2101) with no logic changes. Checked the one inline finding against pre-PR history: the flagged 'EAGLE MTP' comment mislabeling is pre-existing (present at the same location before this PR, confirmed via git show 7dab98e^) and untouched by this diff, so it's not attributable to this merge.
Extended reasoning...
Overview
This PR removes 31 GLM-5/GLM-5.1 config entries (23 NVIDIA, 8 AMD) from the active master YAML configs and archives them verbatim into new configs/deprecated/*.yaml files, and moves the corresponding 18 benchmark scripts into deprecated/ subdirectories with corrected benchmark_lib.sh relative source paths. It follows an established, previously-merged pattern (#1874, #2101) with no new logic, only relocation/removal of static config and shell scripts.
Security risks
None. This is pure YAML/config data relocation and shell script path fixups (source "$(dirname "$0")/../../../benchmark_lib.sh"). No user input, no auth/crypto paths, no new executable logic.
Level of scrutiny
Low-to-moderate is appropriate — the change is large in line count but mechanical in nature (deletions + verbatim archival + relative-path adjustments). I independently verified the one flagged finding: the mislabeled '1k1k high-throughput ... EAGLE MTP' comment above qwen3.5-fp8-b300-sglang-agentic-hicache in configs/nvidia-master.yaml is present unchanged at the same location in the pre-PR blob (git show 7dab98e^), confirming it predates this PR and is not a regression it introduced. YAML comments have no functional effect on sweep generation either way.
Other factors
PR description includes concrete validation (sweep job diff showing exactly 240 glm5 jobs removed, 0 non-glm5 removals/additions; byte-identical config diffs outside the glm5 blocks; pytest utils/matrix_logic/ 224 passed; bash -n on all moved scripts). No outstanding reviewer comments in the timeline, and no prior review from me on this PR.
| ep: 4 | ||
| dp-attn: true | ||
| additional-settings: | ||
| - "CONFIG_FILE=recipes/sglang/glm5.1/gb300-fp8/8k1k/disagg/mtp/8k1k_mtp_lowlat_1.yaml" | ||
| - "CONFIG_FILE=recipes/sglang/qwen3.5/gb300-fp4/8k1k/disagg/stp/8k1k_stp_maxtpt_2.yaml" | ||
| decode: | ||
| num-worker: 17 | ||
| tp: 4 | ||
| ep: 1 | ||
| dp-attn: false | ||
| - spec-decoding: "mtp" | ||
| conc-list: [24] | ||
| prefill: | ||
| num-worker: 1 | ||
| tp: 4 | ||
| ep: 1 | ||
| tp: 16 | ||
| ep: 16 | ||
| dp-attn: true | ||
| additional-settings: | ||
| - "CONFIG_FILE=recipes/sglang/glm5.1/gb300-fp8/8k1k/disagg/mtp/8k1k_mtp_lowlat_2.yaml" | ||
| decode: | ||
| num-worker: 17 | ||
| tp: 4 | ||
| ep: 1 | ||
| dp-attn: false | ||
|
|
||
| # ---------- 1k1k high-throughput (wide-EP decode, EAGLE MTP) ---------- | ||
| qwen3.5-fp8-b300-sglang-agentic-hicache: | ||
| image: lmsysorg/sglang:nightly-dev-cu13-20260520-425dffbd |
There was a problem hiding this comment.
🟣 The section-header comment at this location ('# ---------- 1k1k high-throughput (wide-EP decode, EAGLE MTP) ----------') mislabels the qwen3.5-fp8-b300-sglang-agentic-hicache entry that follows it, but this is a pre-existing documentation defect that predates this PR — diffing against the parent commit shows this exact comment already sat immediately above this exact entry before any glm5 removal happened, and the PR left this line completely untouched (it's unchanged diff context, not a deletion artifact). The claim that this comment was 'dropped entirely' is also incorrect: it is still present verbatim in the current file.
Extended reasoning...
I verified this finding by diffing configs/nvidia-master.yaml between 7dab98e^ (pre-PR) and 7dab98e (post-PR, current HEAD) and by re-reading the current file directly.
The comment at this exact location predates the PR and was not touched by it. At the anchored location (current line 6330), the comment # ---------- 1k1k high-throughput (wide-EP decode, EAGLE MTP) ---------- sits directly above qwen3.5-fp8-b300-sglang-agentic-hicache:. Checking the pre-PR file (git show 7dab98e^:configs/nvidia-master.yaml), the identical comment already sat immediately above the identical entry at line 8196, well before this PR made any change. Running diff -u between the pre- and post-PR files confirms this line is an unchanged context line in the diff — it carries no +/- marker, meaning the PR's hunks neither added, moved, nor removed it. The mislabeling (a "1k1k"/"EAGLE MTP" header sitting over an agentic-coding scenario entry that has nothing to do with either) is real, but it is stale documentation cruft that has existed since at least the previous commit, unrelated to the glm5/glm5.1 removal this PR performs.
The "dropped entirely" claim is also factually incorrect. The synthesis description states this comment "was also dropped entirely — neither cleanly removed nor preserved in the archive." That's not accurate: grep on the current configs/nvidia-master.yaml shows the comment is still present verbatim at line 6330. There is a different comment with similar wording — # ---------- 8k1k low-latency (per-node TP=4 decode workers, EAGLE MTP) ---------- — that genuinely was part of a deleted glm5 block (glm5-fp8-gb300-dynamo-sglang-mtp), and that one was correctly carried into configs/deprecated/nvidia-glm5-glm5.1-master.yaml (verified present at line 1851 there). The synthesis conflated these two distinct "EAGLE MTP"-named comments.
Step-by-step proof:
git show 7dab98e^:configs/nvidia-master.yaml | grep -n "EAGLE MTP"→ line 8196 has this exact comment, immediately followed byqwen3.5-fp8-b300-sglang-agentic-hicache:(verified by reading lines 8196-8198 of that pre-PR blob).diff -u <pre> <post>→ the corresponding line in the diff output has no+/-prefix, i.e. it is unmodified context — proof the PR did not act on this line at all.grep -n "EAGLE MTP" configs/nvidia-master.yaml(current tree) → the comment is present at line 6330, directly refuting "dropped entirely."grep -n "EAGLE MTP" configs/deprecated/nvidia-glm5-glm5.1-master.yaml→ shows the other "8k1k low-latency ... EAGLE MTP" comment (the one that actually belonged to a deleted glm5 block) was correctly archived.
One part of the broader merged finding does hold up, for completeness even though it's a different file than this anchor: in configs/amd-master.yaml, the # === Net-new agentic recipes from chore/agentx-v0.3 === header genuinely was present pre-PR (git show 7dab98e^:configs/amd-master.yaml line 1510) and is genuinely absent post-PR — that one actually is a PR-diff artifact (the comment sat adjacent to the removed glm5.1-fp4-mi355x-atom block and was swept away with it), and it was also duplicated into the new configs/deprecated/amd-glm5-glm5.1-master.yaml where it now dangles at EOF. That is a real, minor, PR-caused documentation nit — but it lives in a different file than this bug's anchor and isn't the claim this specific inline comment should make.
Impact: none functionally either way — YAML comments are ignored by config parsing and sweep generation. For the specific location this report is anchored to, the mislabeled comment is real but pre-existing and unrelated to this PR's changes, so it should not block or be attributed to this merge.
Summary
Deprecates GLM-5 and GLM-5.1 following the MiniMax M2.5/M2.7 pattern (#1874) and the gpt-oss-120b pattern (#2101):
glm5-{fp8,fp4}-{b200,b300}-sglang[-mtp],glm5-fp8-h200-sglang[-mtp],glm5-{fp8,fp4}-{gb200,gb300,b200}-dynamo-{sglang,trt}[-mtp]) →configs/deprecated/nvidia-glm5-glm5.1-master.yamlglm5-fp8-mi355x-{sglang,sglang-mtp,sglang-disagg,atom},glm5.1-fp4-mi355x-{sglang,atom},glm5-fp8-mi325x-sglang[-mtp]) →configs/deprecated/amd-glm5-glm5.1-master.yamldeprecated/subdirectories (17 ×single_node/fixed_seq_len/, 1 ×multi_node/), withbenchmark_lib.shsource paths corrected for the extra directory level (fix: move dsv4-fp4-mi355x MTP scripts into fixed_seq_len/ #1860 bug class), matching the [on june 22, 2026, current gptoss submiters were given notice that it will be deprepcated on july 6th, and were given an chance for comment period and everyone reacted notice] Deprecate gpt-oss-120b benchmark configs / 弃用 gpt-oss-120b 基准测试配置 #2101 improvement over Deprecate MiniMax M2.5 / MiniMax M2.7 model architecture benchmark configs #1874.perf-changelog.yamlentry, matching both precedents (removals don't trigger sweeps).Deliberately left untouched, matching the #1874/#2101 precedent:
runners/launch_{gb200,gb300,b300}-nv.sh,launch_b200-dgxc.sh) — dead but harmless, same as the gptoss/minimax branches still present there.benchmarks/multi_node/srt-slurm-recipes/{sglang,trtllm}/glm5*recipe trees — only reachable via the removed multinode config entries (M2.5 recipe trees were likewise left in place by Deprecate MiniMax M2.5 / MiniMax M2.7 model architecture benchmark configs #1874).utils/evals/thresholds.yamlglm5/glm5.1 entries andperf-changelog.yamlhistory (gptoss/minimaxm2.5 entries remain there too).Out of scope: GLM-5.2 (
glm52SPEED-Bench collector,speedbench/glm5_fp4_b300_vllm.shmanual collectors, and the open GLM-5.2 agentic PRs #2268/#2272/#2273/#2275 — none touch the removed entries).Validation
generate_sweep_configs.py full-sweepon the modified masters: 2477 → 2237 jobs; all 240 removed jobs are glm5, 0 non-glm5 removals, 0 additions (verified against a HEAD baseline).pytest utils/matrix_logic/— 224 passed.bash -non all 18 moved scripts; corrected source paths resolve tobenchmarks/benchmark_lib.sh.中文说明
configs/deprecated/(含注释),扫描生成不再选中它们;全部 18 个脚本移入deprecated/子目录。benchmark_lib.sh的相对路径(fix: move dsv4-fp4-mi355x MTP scripts into fixed_seq_len/ #1860 同类问题)。perf-changelog.yaml条目;launcher 路由分支、srt-slurm 配方树、评估阈值与 changelog 历史保持不变。glm52SPEED-Bench 收集器及开放中的 GLM-5.2 agentic PR)。pytest utils/matrix_logic/224 项全部通过。🤖 Generated with Claude Code