Skip to content

[Klaud Cold] Deprecate GLM-5 / GLM-5.1 benchmark configs / 弃用 GLM-5 / GLM-5.1 基准测试配置#2276

Merged
functionstackx merged 1 commit into
mainfrom
klaud-cold/deprecate-glm5-glm5.1
Jul 18, 2026
Merged

[Klaud Cold] Deprecate GLM-5 / GLM-5.1 benchmark configs / 弃用 GLM-5 / GLM-5.1 基准测试配置#2276
functionstackx merged 1 commit into
mainfrom
klaud-cold/deprecate-glm5-glm5.1

Conversation

@functionstackx

Copy link
Copy Markdown
Collaborator

Summary

Deprecates GLM-5 and GLM-5.1 following the MiniMax M2.5/M2.7 pattern (#1874) and the gpt-oss-120b pattern (#2101):

Deliberately left untouched, matching the #1874/#2101 precedent:

  • Launcher routing branches (runners/launch_{gb200,gb300,b300}-nv.sh, launch_b200-dgxc.sh) — dead but harmless, same as the gptoss/minimax branches still present there.
  • benchmarks/multi_node/srt-slurm-recipes/{sglang,trtllm}/glm5* recipe trees — only reachable via the removed multinode config entries (M2.5 recipe trees were likewise left in place by Deprecate MiniMax M2.5 / MiniMax M2.7 model architecture benchmark configs #1874).
  • utils/evals/thresholds.yaml glm5/glm5.1 entries and perf-changelog.yaml history (gptoss/minimaxm2.5 entries remain there too).

Out of scope: GLM-5.2 (glm52 SPEED-Bench collector, speedbench/glm5_fp4_b300_vllm.sh manual collectors, and the open GLM-5.2 agentic PRs #2268/#2272/#2273/#2275 — none touch the removed entries).

Validation

中文说明

🤖 Generated with Claude Code

Following the MiniMax M2.5/M2.7 (#1874) and gpt-oss-120b (#2101)
deprecation pattern: all 31 glm5/glm5.1 config entries (23 nvidia,
8 amd) removed from the active master configs and archived verbatim
under configs/deprecated/{nvidia,amd}-glm5-glm5.1-master.yaml; the
17 single-node scripts and the mi355x sglang-disagg multi-node script
moved into deprecated/ subdirectories with benchmark_lib.sh source
paths corrected for the extra directory level (the #1860 bug class,
matching #2101).

Launchers, srt-slurm recipe trees, eval thresholds, and
perf-changelog history are left untouched, matching both precedents.
GLM-5.2 (glm52 speedbench collector, open agentic PRs) is out of
scope.

中文:按 MiniMax M2.5/M2.7(#1874)与 gpt-oss-120b(#2101)弃用模式
弃用 GLM-5 / GLM-5.1 基准测试配置——31 个配置条目从活动 master 配置
移除并原样归档至 configs/deprecated/,18 个脚本移入 deprecated/ 子目
录且已修正 benchmark_lib.sh 相对路径;launcher、srt-slurm 配方、评估
阈值与 perf-changelog 历史保持不变,GLM-5.2 不在本次范围内。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

@functionstackx
functionstackx merged commit a5bfbe8 into main Jul 18, 2026
13 checks passed
@functionstackx
functionstackx deleted the klaud-cold/deprecate-glm5-glm5.1 branch July 18, 2026 20:31

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good — this is a mechanical, precedent-following deprecation (mirrors #1874 and #2101) with no logic changes. Checked the one inline finding against pre-PR history: the flagged 'EAGLE MTP' comment mislabeling is pre-existing (present at the same location before this PR, confirmed via git show 7dab98e^) and untouched by this diff, so it's not attributable to this merge.

Extended reasoning...

Overview

This PR removes 31 GLM-5/GLM-5.1 config entries (23 NVIDIA, 8 AMD) from the active master YAML configs and archives them verbatim into new configs/deprecated/*.yaml files, and moves the corresponding 18 benchmark scripts into deprecated/ subdirectories with corrected benchmark_lib.sh relative source paths. It follows an established, previously-merged pattern (#1874, #2101) with no new logic, only relocation/removal of static config and shell scripts.

Security risks

None. This is pure YAML/config data relocation and shell script path fixups (source "$(dirname "$0")/../../../benchmark_lib.sh"). No user input, no auth/crypto paths, no new executable logic.

Level of scrutiny

Low-to-moderate is appropriate — the change is large in line count but mechanical in nature (deletions + verbatim archival + relative-path adjustments). I independently verified the one flagged finding: the mislabeled '1k1k high-throughput ... EAGLE MTP' comment above qwen3.5-fp8-b300-sglang-agentic-hicache in configs/nvidia-master.yaml is present unchanged at the same location in the pre-PR blob (git show 7dab98e^), confirming it predates this PR and is not a regression it introduced. YAML comments have no functional effect on sweep generation either way.

Other factors

PR description includes concrete validation (sweep job diff showing exactly 240 glm5 jobs removed, 0 non-glm5 removals/additions; byte-identical config diffs outside the glm5 blocks; pytest utils/matrix_logic/ 224 passed; bash -n on all moved scripts). No outstanding reviewer comments in the timeline, and no prior review from me on this PR.

Comment on lines +6320 to 6332
ep: 4
dp-attn: true
additional-settings:
- "CONFIG_FILE=recipes/sglang/glm5.1/gb300-fp8/8k1k/disagg/mtp/8k1k_mtp_lowlat_1.yaml"
- "CONFIG_FILE=recipes/sglang/qwen3.5/gb300-fp4/8k1k/disagg/stp/8k1k_stp_maxtpt_2.yaml"
decode:
num-worker: 17
tp: 4
ep: 1
dp-attn: false
- spec-decoding: "mtp"
conc-list: [24]
prefill:
num-worker: 1
tp: 4
ep: 1
tp: 16
ep: 16
dp-attn: true
additional-settings:
- "CONFIG_FILE=recipes/sglang/glm5.1/gb300-fp8/8k1k/disagg/mtp/8k1k_mtp_lowlat_2.yaml"
decode:
num-worker: 17
tp: 4
ep: 1
dp-attn: false

# ---------- 1k1k high-throughput (wide-EP decode, EAGLE MTP) ----------
qwen3.5-fp8-b300-sglang-agentic-hicache:
image: lmsysorg/sglang:nightly-dev-cu13-20260520-425dffbd

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟣 The section-header comment at this location ('# ---------- 1k1k high-throughput (wide-EP decode, EAGLE MTP) ----------') mislabels the qwen3.5-fp8-b300-sglang-agentic-hicache entry that follows it, but this is a pre-existing documentation defect that predates this PR — diffing against the parent commit shows this exact comment already sat immediately above this exact entry before any glm5 removal happened, and the PR left this line completely untouched (it's unchanged diff context, not a deletion artifact). The claim that this comment was 'dropped entirely' is also incorrect: it is still present verbatim in the current file.

Extended reasoning...

I verified this finding by diffing configs/nvidia-master.yaml between 7dab98e^ (pre-PR) and 7dab98e (post-PR, current HEAD) and by re-reading the current file directly.

The comment at this exact location predates the PR and was not touched by it. At the anchored location (current line 6330), the comment # ---------- 1k1k high-throughput (wide-EP decode, EAGLE MTP) ---------- sits directly above qwen3.5-fp8-b300-sglang-agentic-hicache:. Checking the pre-PR file (git show 7dab98e^:configs/nvidia-master.yaml), the identical comment already sat immediately above the identical entry at line 8196, well before this PR made any change. Running diff -u between the pre- and post-PR files confirms this line is an unchanged context line in the diff — it carries no +/- marker, meaning the PR's hunks neither added, moved, nor removed it. The mislabeling (a "1k1k"/"EAGLE MTP" header sitting over an agentic-coding scenario entry that has nothing to do with either) is real, but it is stale documentation cruft that has existed since at least the previous commit, unrelated to the glm5/glm5.1 removal this PR performs.

The "dropped entirely" claim is also factually incorrect. The synthesis description states this comment "was also dropped entirely — neither cleanly removed nor preserved in the archive." That's not accurate: grep on the current configs/nvidia-master.yaml shows the comment is still present verbatim at line 6330. There is a different comment with similar wording — # ---------- 8k1k low-latency (per-node TP=4 decode workers, EAGLE MTP) ---------- — that genuinely was part of a deleted glm5 block (glm5-fp8-gb300-dynamo-sglang-mtp), and that one was correctly carried into configs/deprecated/nvidia-glm5-glm5.1-master.yaml (verified present at line 1851 there). The synthesis conflated these two distinct "EAGLE MTP"-named comments.

Step-by-step proof:

  1. git show 7dab98e^:configs/nvidia-master.yaml | grep -n "EAGLE MTP" → line 8196 has this exact comment, immediately followed by qwen3.5-fp8-b300-sglang-agentic-hicache: (verified by reading lines 8196-8198 of that pre-PR blob).
  2. diff -u <pre> <post> → the corresponding line in the diff output has no +/- prefix, i.e. it is unmodified context — proof the PR did not act on this line at all.
  3. grep -n "EAGLE MTP" configs/nvidia-master.yaml (current tree) → the comment is present at line 6330, directly refuting "dropped entirely."
  4. grep -n "EAGLE MTP" configs/deprecated/nvidia-glm5-glm5.1-master.yaml → shows the other "8k1k low-latency ... EAGLE MTP" comment (the one that actually belonged to a deleted glm5 block) was correctly archived.

One part of the broader merged finding does hold up, for completeness even though it's a different file than this anchor: in configs/amd-master.yaml, the # === Net-new agentic recipes from chore/agentx-v0.3 === header genuinely was present pre-PR (git show 7dab98e^:configs/amd-master.yaml line 1510) and is genuinely absent post-PR — that one actually is a PR-diff artifact (the comment sat adjacent to the removed glm5.1-fp4-mi355x-atom block and was swept away with it), and it was also duplicated into the new configs/deprecated/amd-glm5-glm5.1-master.yaml where it now dangles at EOF. That is a real, minor, PR-caused documentation nit — but it lives in a different file than this bug's anchor and isn't the claim this specific inline comment should make.

Impact: none functionally either way — YAML comments are ignored by config parsing and sweep generation. For the specific location this report is anchored to, the mislabeled comment is real but pre-existing and unrelated to this PR's changes, so it should not block or be attributed to this merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

1 participant