[FR] [Bug] vllm-plugin-FL fails to load with FlagTree on MUSA S5000: triton_v3.2.x reports 0 active drivers, main/triton 3.6 also fails

[mx-dw]

#[FR] # [Bug] vllm-plugin-FL fails to load with FlagTree on MUSA S5000: triton_v3.2.x reports 0 active drivers, main/triton 3.6 also fails

Description

When using vllm-plugin-FL for FlagTree adaptation on Moore Threads / MUSA S5000, the vLLM service fails to start after installing FlagTree/Triton.

I tested both:

1. FlagTree / Triton 3.2 path
2. FlagTree main branch, which appears to be based on the Triton 3.6 path

Both fail to load the fl plugin.

For the Triton 3.2 path, the plugin fails during flag_gems import. The root cause seems to be that Triton cannot discover any active driver in the MUSA environment:

RuntimeError: 0 active drivers ([]). There should only be one.

After that, vLLM cannot infer the device type and exits:

RuntimeError: Failed to infer device type

Steps to Reproduce

For the main branch / Triton 3.6 path, I installed FlagTree from source using the following general command sequence:

git clone https://github.com/flagos-ai/FlagTree.git
cd FlagTree
pip install . --no-build-isolation -v

export VLLM_PLUGINS=fl
export MUSA_VISIBLE_DEVICES=4,5,6,7
export VLLM_CONFIGURE_LOGGING=1
export VLLM_LOGGING_LEVEL=DEBUG

vllm serve /data/Qwen3.6-27B
--served-model-name qwen
--tensor-parallel-size 4
--host 0.0.0.0
--port 8099
--enforce-eager
--trust-remote-code

The installation path also fails when starting vLLM with vllm-plugin-FL. Therefore this does not appear to be limited to only the 3.2 branch.

Error Log

INFO 07-09 02:13:59 [init.py:44] Available plugins for group vllm.platform_plugins:
INFO 07-09 02:13:59 [init.py:46] - fl -> vllm_fl:register
INFO 07-09 02:13:59 [init.py:46] - musa -> vllm_musa:musa_platform_plugin
INFO 07-09 02:13:59 [init.py:58] Loading plugin fl
ERROR 07-09 02:14:00 [init.py:64] Failed to load plugin fl

The failure happens while importing flag_gems:

File "/workspace/vllm-plugin-FL/vllm_fl/init.py", line 20, in
from vllm_fl.utils import get_op_config as _get_op_config

File "/workspace/vllm-plugin-FL/vllm_fl/utils.py", line 7, in
import flag_gems

File "/usr/local/lib/python3.10/dist-packages/flag_gems/init.py", line 10, in
from flag_gems.fused import * # noqa: F403

File "/usr/local/lib/python3.10/dist-packages/flag_gems/fused/init.py", line 3, in
from flag_gems.fused.concat_and_cache_mla import concat_and_cache_mla

File "/usr/local/lib/python3.10/dist-packages/flag_gems/utils/shape_utils.py", line 265, in
def add_on_kernel(

File "/usr/local/lib/python3.10/dist-packages/triton/runtime/autotuner.py", line 130, in init
self.do_bench = driver.active.get_benchmarker()

File "/usr/local/lib/python3.10/dist-packages/triton/runtime/driver.py", line 8, in _create_driver
raise RuntimeError(f"{len(actives)} active drivers ({actives}). There should only be one.")

RuntimeError: 0 active drivers ([]). There should only be one.

Then vLLM exits with:

RuntimeError: Failed to infer device type, please set the environment variable VLLM_LOGGING_LEVEL=DEBUG to turn on verbose logging to help debug the issue.

Environment

Component	Version
Hardware	Moore Threads / MUSA / MTT S5000
GPU count	8
GPU memory	81920 MiB per card
Driver	3.3.5-server
OS	Ubuntu 22.04.5 LTS
Architecture	x86_64
Python	3.10
vLLM	0.20.2+empty
vllm-plugin-FL	0.2.0+gd1653d9ff
flag_gems	5.0.0
PyTorch	2.7.1
Original Triton in working container	3.2.0
Original FlagTree in working container	None
Tested FlagTree / Triton paths	triton_v3.2.x / triton==3.2.0, and main branch / Triton 3.6 path
Model	Qwen3.6-27B
Tensor Parallel	4

The original container environment uses vLLM 0.20.2+empty vllm-plugin-FL 0.2.0 FlagGems 5.0.0 Triton 3.2.0 and no FlagTree installed. After adding FlagTree or replacing Triton, the plugin fails to load.harbor.baai.ac.cn/flagrelease-public/musa_gems_tree:2607081834

Expected Behavior

The plugin should update its dependency to a version compatible with FlagTree's triton 3.6.0, or add compatibility handling for the missing function.flag_gemspow

Reference

Plugin adaptation documentation: https://qcnje6z2ldat.feishu.cn/docx/SEHtdWS3Go20Qbx4K30cI7JPnBf

[mx-dw]

hello,请问有进展吗，是否需要其他帮助@zhzhcookie
请问这个bug是否被测出，解法如何，是否需要其他帮助@JosephNew

[mx-dw]

@zhzhcookie 请问是否有进展了

[JosephNew]

可以尝试直接使用安装好的镜像 https://flagos.io/resourcedownload

[mx-dw]

可以尝试直接使用安装好的镜像 https://flagos.io/resourcedownload

实际上我们正在制作这个版本的镜像-musa+empty vllm0.20.2+vllm-plugin-fl-具体步骤见Reference

[zhzhcookie]

@zhzhcookie 请问是否有进展了

报错信息不全。看看可否让我进环境中查看一下？

[mx-dw]

@zhzhcookie请问是否有进展了

报错信息不全。看看可否让我进环境中查看一下？

我想可能是因为厂商自己的Triton 和FlagTree的冲突了，我正在制作完整日志的报告文件，后续会交给你们

[zhzhcookie]

@zhzhcookie请问是否有进展了

报错信息不全。看看可否让我进环境中查看一下？

我想可能是因为厂商自己的Triton 和FlagTree的冲突了，我正在制作完整日志的报告文件，后续会交给你们

参考 github 文档，安装 FlagTree 前需要卸载 triton，可以验证下。

[mx-dw]

@zhzhcookie请问是否有进展了

报错信息不全。看看可否让我进环境中查看一下？

我想可能是因为厂商自己的Triton 和FlagTree的冲突了，我正在制作完整日志的报告文件，后续会交给你们

参考 github 文档，安装 FlagTree 前需要卸载 triton，可以验证下。

我的意思是“厂商自己的Triton 和FlagTree的（Triton ）冲突了”
希望您能理解整体情况是，
按照https://qcnje6z2ldat.feishu.cn/docx/SEHtdWS3Go20Qbx4K30cI7JPnBf
可跑通推理，
同容器安装FlagTree后跑不通推理，
issue正文的部分报错日志可能不足支持判断，正在复现中（之前提时已经复现3次，这算第四次）