feat: support GLM-4.7-Flash (MLA + MoE) on Ascend NPU

[is-not]

Overview

Adds training/inference support for the GLM-4.7-Flash model on Ascend 910B NPU. GLM-4.7-Flash uses DeepSeek-V3-style Multi-Head Latent Attention (MLA) plus a GLM-style MoE (64 routed experts + 1 shared expert, sigmoid router with expert bias), and supports Multi-Token Prediction (MTP). This PR registers the corresponding megatron bridge, adds the model-argument and run scripts, and fixes an MTP drafter compatibility issue under NPU cudagraph capture.

Changes

New bridge plugin

• vime_plugins/megatron_bridge/glm4moe_lite.py: Adds the megatron bridge for Glm4MoeLiteForCausalLM. It subclasses GLM45Bridge to reuse its MTP plumbing (num_nextn_predict_layers mapping loop, fused-expert handling, etc.) and only overrides provider_bridge and mapping_registry for the MLA model. MindSpeed TE module types are registered via the public AutoMapping.register_module_type API through _register_mindspeed_te_module_types(), so weight loading works on NPU without manually patching the container's param_mapping.py. Non-MLA branches that are always dead code for this model (is_mla / is_mla_mapping are always true) have been removed, along with the now-unused QKVMapping / GPTModelProvider imports.
• vime_plugins/megatron_bridge/__init__.py: Imports and registers the new bridge.

MTP cudagraph compatibility fix

• docker/npu_patch/vllm.patch: Adds a static patch to vLLM's upstream Glm4MoeLiteMultiTokenPredictorLayer.forward that rewrites the bool-mask index assignment (inputs_embeds[positions == 0] = 0) into the equivalent element-wise torch.where. The original form routes to aclnnNonzeroV2 (a data-dependent-shape op), which fails with stream is captured under NPU cudagraph capture. After the rewrite, the MTP drafter runs under cudagraph on Ascend. The patch is applied at image build time (reconciled at runtime via series.conf), so no runtime hook or env gate is needed. The change is scoped to the GLM MTP class and is a no-op for non-GLM models.

Run scripts

• scripts/models/glm4.7-30B-A3B-npu.sh: Model argument definitions for GLM-4.7-Flash 30B-A3B (MoE config, MLA q/kv lora rank, qk/v head dim, etc.).
• scripts/run-glm4.7-30B-A3B-npu.sh: NPU training entry script (GRPO, 16 cards on a single node); enables MTP training and vLLM MTP speculative decoding, and sets VIME_PATCH_GLM_MTP_GRAPH=1.

Usage

bash scripts/run-glm4.7-30B-A3B-npu.sh

Weight and data paths can be overridden via the DATA_ROOT environment variable.

Testing

• GLM-4.7-Flash GRPO training runs through on a single NPU node with 16 cards; weight loading is normal and the reward keeps trending upward.

• Whether to enable MTP or not

[read-the-docs-community]

Documentation build overview

📚 vime | 🛠️ Build #34464572 | 📁 Comparing 2b25c32 against latest (8ca82e0)

  🔍 Preview build  

47 files changed · + 3 added · ± 27 modified · - 17 deleted

+ Added

• get_started/NPU.html
• _examples_synced/retool/README.html
• _examples_synced/search-r1/README.html

± Modified

• genindex.html
• index.html
• search.html
• advanced/arch-support-beyond-megatron.html
• advanced/fault-tolerance.html
• advanced/megatron-config.html
• advanced/pd-disaggregation.html
• advanced/reproducibility.html
• advanced/speculative-decoding.html
• advanced/vllm-config.html
• and 17 more...

- Deleted

• advanced/delta-weight-sync.html
• advanced/external-rollout-engines.html
• advanced/low-precision.html
• advanced/observability.html
• advanced/on-policy-distillation.html
• examples/deepseek-r1.html
• examples/glm4-9B.html
• examples/glm4.7-30B-A3B.html
• examples/glm4.7-355B-A32B.html
• examples/glm5.2-744B-A40B.html
• and 7 more...

[gemini-code-assist]

Code Review

This pull request introduces support for GLM-4.7-Flash (a model featuring Multi-Head Latent Attention and Mixture of Experts) by adding model configurations, a training run script, and a Megatron bridge implementation with Multi-Token Prediction (MTP) support on Ascend NPUs. The review feedback highlights several critical Bash scripting improvements, including quoting array expansions and wildcards to prevent globbing, replacing an overly aggressive pkill -9 python command, and addressing an undefined ${MTP_ARGS[@]} variable. Additionally, the reviewer points out opportunities to simplify the Python bridge code by removing redundant conditional checks and dead code branches, as the bridge is exclusively used for MLA-based models.

[gemini-code-assist]

--moe-layer-freq 的值包含通配符（*、[、]），如果不在定义时用双引号包裹，可能会在 Bash 中触发路径名扩展（globbing）。建议将其用双引号包裹以确保其被视为字面字符串。\n\n此外，在后续展开数组时，也必须使用双引号（例如 "${MODEL_ARGS[@]}"）以防止在展开处触发 globbing。

Suggested change

	--moe-layer-freq [0]*$N_DENSE_LAYERS+[1]*$N_MOE_LAYERS
	--moe-layer-freq "[0]*$N_DENSE_LAYERS+[1]*$N_MOE_LAYERS"

[gemini-code-assist]

在展开 Bash 数组时如果不使用双引号（例如 ${MODEL_ARGS[@]}），会导致 Shell 对展开后的元素进行单词拆分（word splitting）和路径名扩展（globbing）。这会导致像 --moe-layer-freq [0]*1+[1]*46 这样的元素在当前工作目录下存在匹配文件时被错误展开，从而导致参数解析错误。\n\n请始终使用双引号展开数组："${ARRAY[@]}"。\n\n另外，${MTP_ARGS[@]} 在此处被引用，但该变量在脚本或引入的模型配置中从未被定义。建议将其移除或进行定义。

Suggested change

	${MODEL_ARGS[@]} \
	${CKPT_ARGS[@]} \
	${ROLLOUT_ARGS[@]} \
	${OPTIMIZER_ARGS[@]} \
	${GRPO_ARGS[@]} \
	${PERF_ARGS[@]} \
	${EVAL_ARGS[@]} \
	${VLLM_ARGS[@]} \
	${MISC_ARGS[@]} \
	${MTP_ARGS[@]}
	"${MODEL_ARGS[@]}" \
	"${CKPT_ARGS[@]}" \
	"${ROLLOUT_ARGS[@]}" \
	"${OPTIMIZER_ARGS[@]}" \
	"${GRPO_ARGS[@]}" \
	"${PERF_ARGS[@]}" \
	"${EVAL_ARGS[@]}" \
	"${VLLM_ARGS[@]}" \
	"${MISC_ARGS[@]}"

[gemini-code-assist]

使用 pkill -9 python 过于激进，会强行终止系统上运行的所有 Python 进程。在多用户或共享环境中，这会中断其他用户或无关的后台服务。\n\n强烈建议仅针对特定的训练或辅助进程进行清理（例如通过匹配脚本名称 train.py）。

Suggested change

	pkill -9 python
	pkill -9 -f train.py

[gemini-code-assist]

由于 GLM47MTPBridge 仅针对 Glm4MoeLiteForCausalLM（这始终是一个基于 MLA 的模型）进行注册，因此 is_mla 将始终评估为 True。这里的 else 分支和条件判断是冗余的，可以进行简化以提高代码的可维护性。

        provider_class = self.PROVIDER_CLASS if self.PROVIDER_CLASS is not None else MLAModelProvider
        provider = provider_class(**provider_kwargs)

[gemini-code-assist]

由于此 bridge 专用于 Glm4MoeLiteForCausalLM（始终是 MLA 模型），因此当 self._hf_config 存在时，is_mla_mapping 始终为 True。我们可以通过在 layer_specific_mappings 中直接将 q_layernorm 映射到 q_a_layernorm.weight 来简化映射注册，并移除冗余的条件判断。

Suggested change

	# Detect MLA model (e.g. GLM-4.7-Flash) for conditional mapping
	is_mla_mapping = hasattr(self, "_hf_config") and hasattr(self._hf_config, "q_lora_rank")
	# For MLA models, override q_layernorm mapping from q_norm to q_a_layernorm
	if is_mla_mapping:
	layer_specific_mappings[
	"decoder.layers.*.self_attention.q_layernorm.weight"
	] = "model.layers.*.self_attn.q_a_layernorm.weight"
	# GLM-4.7-Flash is always an MLA model, so we map q_layernorm to q_a_layernorm directly
	layer_specific_mappings[
	"decoder.layers.*.self_attention.q_layernorm.weight"
	] = "model.layers.*.self_attn.q_a_layernorm.weight"

[gemini-code-assist]

由于当 self._hf_config 存在时 is_mla_mapping 始终为 True（且如果 self._hf_config 缺失，已在第 248 行提前返回），因此第 319-336 行的 else 分支是永远无法执行的死代码。\n\n建议移除冗余的 if is_mla_mapping: 判断和死代码分支，以简化代码结构。

            # MLA models (e.g. GLM-4.7-Flash): MTP transformer layer reuses the
            # last normal layer spec (MLA), so map the individual Q/KV down/up
            # projections instead of a fused QKV.
            mapping_list.extend(
                [
                    AutoMapping(
                        megatron_param=f"mtp.layers.{mtp_layer}.transformer_layer.self_attention.linear_q_down_proj.weight",
                        hf_param=f"model.layers.{mtp_layer + num_transformer_layers}.self_attn.q_a_proj.weight",
                    ),
                    AutoMapping(
                        megatron_param=f"mtp.layers.{mtp_layer}.transformer_layer.self_attention.linear_q_up_proj.weight",
                        hf_param=f"model.layers.{mtp_layer + num_transformer_layers}.self_attn.q_b_proj.weight",
                    ),
                    AutoMapping(
                        megatron_param=f"mtp.layers.{mtp_layer}.transformer_layer.self_attention.linear_kv_down_proj.weight",
                        hf_param=f"model.layers.{mtp_layer + num_transformer_layers}.self_attn.kv_a_proj_with_mqa.weight",
                    ),
                    AutoMapping(
                        megatron_param=f"mtp.layers.{mtp_layer}.transformer_layer.self_attention.linear_kv_up_proj.weight",
                        hf_param=f"model.layers.{mtp_layer + num_transformer_layers}.self_attn.kv_b_proj.weight",
                    ),
                    AutoMapping(
                        megatron_param=f"mtp.layers.{mtp_layer}.transformer_layer.self_attention.linear_q_up_proj.layer_norm_weight",
                        hf_param=f"model.layers.{mtp_layer + num_transformer_layers}.self_attn.q_a_layernorm.weight",
                    ),
                    AutoMapping(
                        megatron_param=f"mtp.layers.{mtp_layer}.transformer_layer.self_attention.linear_kv_up_proj.layer_norm_weight",
                        hf_param=f"model.layers.{mtp_layer + num_transformer_layers}.self_attn.kv_a_layernorm.weight",
                    ),
                ]
            )

[CalvinXKY]

ray start --num-gpus 16 advertises GPU capacity on an NPU job. On Ascend, vime schedules NPU bundles (see launch.py: --num-gpus 0 --resources '{"NPU": 16}'). Please align with the existing NPU launch convention (cf. run-qwen3-30B-A3B-npu.sh / launch.py).

[CalvinXKY]

DCO is failing: both commits are missing Signed-off-by: lines. Please add sign-off before merge.

[CalvinXKY]

provider.moe_router_pre_softmax = False conflicts with --moe-router-pre-softmax in the model script and with NVIDIA's GLM47FlashBridge (True). Please align provider defaults with the training CLI / official bridge to avoid router behavior mismatch during HF bridge load.

[Meihan-chen]

Please rebase onto the latest ascend branch and add a new GLM test to validate the integration and prevent regressions，the new end-to-end GLM path is not covered by NPU CI.

[Meihan-chen]

Just curious, what’s the benefit of enabling MTP here? If it’s optional, maybe we could get the model support in first and add MTP for a follow-up?

[Meihan-chen]

With the main sync in #413 changing the model-loading and weight-sync paths quite a bit, how about focusing on basic GLM-4.7-Flash support here and adding MTP after the sync lands? That should help avoid having to adapt and validate the MTP path twice.

[is-not]

From our testing, the benefits and adaptation cost of MTP are quite clear: on the inference side, with spec=1 configuration, rollout end-to-end latency is reduced by ~12%, the pos0 acceptance rate reaches ~78%, and the gains from speculative decoding are clearly realized. On the training side, MTP loss aligns perfectly with the convergence pace of the main task; the reward curve overlaps with the non-MTP baseline (both converge from 0.52 to 0.73), so MTP does not degrade training performance. In terms of adaptation cost, training and inference share the same set of checkpoints with no extra weight files required. Code changes are confined to the mapping_registry in the bridge plugin, adding only about 60 lines of MTP layer mapping logic with zero intrusion into the main framework path. MTP is fully optional: when disabled, simply remove MTP_ARGS and vllm-speculative-config related settings, and the model will run normally and complete RL convergence. All things considered, I recommend including MTP in this PR.