基于vLLM-Ascend的Qwen3.5-122B模型Atlas 800I A3单机混部部署实践

​作者​：昇腾实战派
​知识地图​：https://blog.csdn.net/Lumos_Lovegood/article/details/161601003

背景概述

本文档将介绍基于vLLM-Ascend的Qwen3.5-122B模型在Atlas 800I A3上的单机混部部署实践，包括支持的特性、特性配置、环境信息以及性能测试典型case。

基本信息

软件版本设备信息组网形态总卡数数据格式
0.18.0NPU: Atlas 800I A3-560T, HBM 128G
CPU: Kunpeng 920 (80核-2900MHz)
内存: 32根64G5200MHz
OS: OpenEuler 22.03 LTS-SP4		-2W4A8C16

服务化配置

低时延

exportASCEND_RT_VISIBLE_DEVICES=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15exportPYTORCH_NPU_ALLOC_CONF="expandable_segments:True"exportHCCL_IF_IP="XXXX"exportHCCL_OP_EXPANSION_MODE="AIV"exportHCCL_BUFFSIZE=1024exportOMP_NUM_THREADS=1echoperformance|tee/sys/devices/system/cpu/cpu*/cpufreq/scaling_governorsysctl-wvm.swappiness=0sysctl-wkernel.numa_balancing=0sysctlkernel.sched_migration_cost_ns=50000exportLD_PRELOAD=/usr/lib/aarch64-linux-gnu/libjemalloc.so.2:$LD_PRELOADexportTASK_QUEUE_ENABLE=1exportVLLM_ASCEND_ENABLE_FUSED_MC2=1vllm serve /mnt/share/weights/Qwen3.5-122B-A10B-w8a8-org/\--served-model-name"qwen3.5"\--host0.0.0.0\--port8000\--data-parallel-size1\--tensor-parallel-size4\--max-model-len5000\--max-num-batched-tokens8192\--max-num-seqs128\--gpu-memory-utilization0.9\--compilation-config'{"cudagraph_capture_sizes":[1,6,12,18,24,30,36,42,48,54,72,78,84,90,96,102,108,144,192], "cudagraph_mode":"FULL_DECODE_ONLY"}'\--speculative_config'{"method": "qwen3_5_mtp", "num_speculative_tokens": 5}'\--trust-remote-code\--async-scheduling\--allowed-local-media-path /\--quantizationascend\--mm-processor-cache-gb0\--additional-config'{"enable_cpu_binding": true, "enable_shared_expert_dp": true}'
典型测试用例
平均输入平均输出并行策略上下文长度Prefix Cache命中率总请求数最大并发数请求频率(req/s)
20482048MLA：DP1+TP45000096240
35001500MLA：DP1+TP48000064160
163841024MLA：DP1+TP41843203280
32768512MLA：DP1+TP43430401640
655361024MLA：DP1+TP46758401640
1310721024MLA：DP1+TP41331200410

高吞吐

exportASCEND_RT_VISIBLE_DEVICES=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15exportPYTORCH_NPU_ALLOC_CONF="expandable_segments:True"exportHCCL_IF_IP="XXX"exportHCCL_OP_EXPANSION_MODE="AIV"exportHCCL_BUFFSIZE=1024exportOMP_NUM_THREADS=1echoperformance|tee/sys/devices/system/cpu/cpu*/cpufreq/scaling_governorsysctl-wvm.swappiness=0sysctl-wkernel.numa_balancing=0sysctlkernel.sched_migration_cost_ns=50000exportLD_PRELOAD=/usr/lib/aarch64-linux-gnu/libjemalloc.so.2:$LD_PRELOADexportTASK_QUEUE_ENABLE=1exportVLLM_ASCEND_ENABLE_FUSED_MC2=1exportVLLM_ASCEND_ENABLE_FLASHCOMM1=1vllm serve /mnt/share/weights/Qwen3.5-122B-A10B-w8a8-org/\--served-model-name"qwen3.5"\--host0.0.0.0\--port8000\--data-parallel-size1\--tensor-parallel-size4\--enable-expert-parallel\--max-model-len8000\--max-num-batched-tokens4096\--max-num-seqs128\--gpu-memory-utilization0.9\--compilation-config'{"cudagraph_capture_sizes":[1,4,8,12,16,24,32,48,56,64,72,84,96,108,112,128,160,172,196,200,212,232,256,160,172,196,200,212,232,256,272,288,312,328,344,360,384,400,416,432,448,480,512], "cudagraph_mode":"FULL_DECODE_ONLY"}'\--speculative_config'{"method": "qwen3_5_mtp", "num_speculative_tokens": 3, "enforce_eager": true}'\--trust-remote-code\--async-scheduling\--allowed-local-media-path /\--quantizationascend\--mm-processor-cache-gb0\--additional-config'{"enable_cpu_binding": true, "enable_shared_expert_dp": true}'
典型测试用例
平均输入平均输出并行策略上下文长度Prefix Cache命中率总请求数最大并发数请求频率(req/s)
20482048MLA：DP1+TP2500006001500
35001500MLA：DP1+TP280000320800
163841024MLA：DP1+TP218432096240
32768512MLA：DP1+TP23430403690
655361024MLA：DP1+TP26758403280
1310721024MLA：DP1+TP213312001640

测试命令

参考aisbench官方测试指南。

aisbench测试命令

vllm-ascend社区官网

特别声明

  1. 以上配置均未开启Prefix Cache，若实际生产环境需要使用该特性，参考vLLM-Ascend社区参数指南开启–enable-prefix-caching