Skip to content

[Model] Support to train glm5 - #1227

Merged
zhaoyinglia merged 1 commit into
flagos-ai:mainfrom
heavyrain-lzy:support_glm5
Jul 31, 2026
Merged

[Model] Support to train glm5#1227
zhaoyinglia merged 1 commit into
flagos-ai:mainfrom
heavyrain-lzy:support_glm5

Conversation

@heavyrain-lzy

@heavyrain-lzy heavyrain-lzy commented Jun 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

PR Category

Train

PR Types

Others

PR Description

  • Add GLM5/GLM5.1/GLM5.2 training support with new example configs under examples/glm5/, including both a small demo config and a 744B/40B Active model config.
  • Add GLM5 checkpoint conversion support in tools/checkpoint/, but not verified enoughly.
  • Update Megatron GPT builder to preserve experimental attention layer specs for MTP and print model structure/basic parameter-memory information after model construction.
  • Fix checkpoint save flow so energy monitor pause/resume is only triggered when log_energy is enabled.

chencjcj added a commit to chencjcj/FlagScale that referenced this pull request Jul 19, 2026
@chencjcj @claude
[Model] Support to train glm5 (apply PR flagos-ai#1227)
52f54d5 
Add GLM5/GLM5.1/GLM5.2 training support:
- examples/glm5/ configs (1b_a0_5b demo + 744b_a40b)
- tools/checkpoint/glm5/ HF<->mcore conversion
- gpt_builders: preserve experimental attention specs for MTP
- guard energy monitor pause/resume behind log_energy

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
chencjcj pushed a commit to chencjcj/FlagScale that referenced this pull request Jul 19, 2026
chencjcj @claude
[Model] Support to train glm5 (apply PR flagos-ai#1227)
f3b997f 
Add GLM5/GLM5.1/GLM5.2 training support:
- examples/glm5/ configs (1b_a0_5b demo + 744b_a40b)
- tools/checkpoint/glm5/ HF<->mcore conversion
- gpt_builders: preserve experimental attention specs for MTP
- guard energy monitor pause/resume behind log_energy

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
chencjcj pushed a commit to chencjcj/FlagScale that referenced this pull request Jul 19, 2026
chencjcj
[Model] Support to train glm5 (apply PR flagos-ai#1227)
528be79 
Add GLM5/GLM5.1/GLM5.2 training support:
- examples/glm5/ configs (1b_a0_5b demo + 744b_a40b)
- tools/checkpoint/glm5/ HF<->mcore conversion
- gpt_builders: preserve experimental attention specs for MTP
- guard energy monitor pause/resume behind log_energy
lxd-cumt
lxd-cumt approved these changes Jul 27, 2026
View reviewed changes

@lxd-cumt lxd-cumt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhaoyinglia
zhaoyinglia merged commit 63f7c17 into flagos-ai:main Jul 31, 2026
57 of 60 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lxd-cumt lxd-cumt lxd-cumt approved these changes
@zhaoyinglia zhaoyinglia zhaoyinglia approved these changes
@aoyulong aoyulong Awaiting requested review from aoyulong aoyulong is a code owner
@cyber-pioneer cyber-pioneer Awaiting requested review from cyber-pioneer
@ceci3 ceci3 Awaiting requested review from ceci3

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@heavyrain-lzy @lxd-cumt @zhaoyinglia