vLLM v0.26.0：新增 Inkling 模型族，DeepSeek-V4 优化，KV 卸载增强

vLLM v0.26.0 Release Notes

Highlights

This release features 411 commits from 212 contributors (61 new)!

• New Inkling model family with a full support stack: base modeling (#48799), piecewise CUDA graph support (#48822), Hopper FA4 relative attention (#48858), MTP=1 speculative decoding (#48869), LoRA (#48884), and standard ModelOpt NVFP4 quantization (#48990).
• DeepSeek-V4 performance push across vendors: a specialized routing kernel (2.94% E2E TPOT, #48660), fused_topk_bias (1.5–2x kernel, #47463), and redundant repeat/copy removal (1.8% E2E TPOT, #48137), plus ROCm two-stage compressor for HCA prefill (#47718), sparse decode/prefill optimizations (#48519, #48788, #46275), and DSpark speculative decoding on AMD (#47419) and XPU (#47677).
• fp32 lm_head for generation models via head_dtype (#48390), extended to the LoRA path (#48525) and given a ROCm torch.mm fast path (#48688), improving accuracy for generation heads.
• Flexible attention backends: the attention backend can now be selected per KV-cache group (#48012), and sliding-window support is now an explicit backend capability (#48011) — improving support for hybrid models.
• KV offloading & tiered secondary storage matured substantially: offloading metrics (#45958, #47666, #47679), tier-owned event handling (#46544, #47923), object-store secondary tier with workload identity (#47063, #47274, #48150), DP-replica-aware tiering (#47987), and encoder-cache (EC) connectors including CPU offloading (#42433, #47423).
• Rust frontend gained multimodal video (#47959) and audio (#48554), a Seed-OSS tool parser (#47741), and a native vllm-bench port (#48107).
• Transformers 5.13.0 (#47867) with more models migrated to the Transformers modeling backend: Olmo/Olmo2 (#48100), MistralLarge3 (#48153), and HunyuanVL (#47872).

Model Support

• New models: Inkling family (#48799, #48822, #48858, #48869, #48884, #48990), BertForMaskedLM (#48463), RobertaForTokenClassification / XLMRobertaForTokenClassification (#47991), LongCat-Flash-Lite n-gram embedding (#47857), Cosmos3 Edge Reasoner (#48291) and Cosmos3-Super registration (#48211), TranslateGemma-12b-it (#41599).
• Transformers backend migrations: Olmo/Olmo2 (#48100), MistralLarge3 to AutoWeightsLoader (#48153), HunyuanVL native transformers processor for transformers 5.13 (#47872).
• GLM5.2: migrate MoE sequence-parallel support to the non-torch-compiled path (#47881).
• LoRA: FlashInfer MoE LoRA for BF16 models (#48632), LoRA for tower/connector in LlavaNextVideo (#48594), fp32 lm_head on the LoRA path (#48525), optimized TrtLlmLoRAExperts (#48759).
• Multimodal: automatic fallback to ViT data parallelism when TP is unavailable (#49046).
• Fixes: correct pooling scores for chunked prefill under torch.compile (#48901).

Engine Core

• fp32 lm_head for generation models via head_dtype (#48390); lower memory for capturing large CUDA graph sizes (#48483); opt-in persistence and reuse of the memory-profiling result across boots (#47388); improved InstantTensor loading (#46868).
• Attention: select a different attention backend per KV-cache group (#48012); sliding-window as an explicit backend capability (#48011); KV-cache layout refactor packing K/V into the content dim across backends (#44455); MRV2 virtual-batch PCP for MLA (#46570).
• Speculative decoding: runtime draft weight update (#46725), hybrid (SWA + full attention) DFlash drafters (#47914), SWA support for qwen-eagle3 (#47568), Gemma4-12B DSpark draft model (#47216), DSv4 DSpark on AMD (#47419), separate kv_cache_dtype for speculative_config (#48787).
• KV offloading: basic offloading metrics (#45958), split CPU cache usage into read/write gauges (#47666) and tiering-lookup-delay into sync/async histograms (#47679), tier-owned event handling and BlockStored events (#46544, #47923), object-store secondary tier with workload identity (#47063, #47274, #48150), DP-replica-aware tiering (#47987), blocks_per_chunk config for heterogeneous KV groups (#48878), P2P default host/port env vars (#47636).
• Caching: partial prefix-cache hit for hybrid models (#46384), selective hybrid cache retention (#47782), report prefix-cache-reused blocks in full report mode (#45261).
• Reasoning: optimize TPOT for thinking budget when used with speculative decoding (#46662).
• RLHF: stateful trainer-send abstractions (#48042).
• Fixes: host memory leak from undrained new_block_ids (#44490), DSv3.2 + MTP + sequence-parallel accuracy (#48036).

Hardware & Performance

• DeepSeek-V4: specialized routing kernel (2.94% E2E TPOT, #48660), fused_topk_bias 1.5–2x (#47463), redundant repeat/copy removal (1.8% TPOT, #48137).
• MoE router GEMMs: BF16x3 router GEMM (#47973), FP32 router GEMV (#48335), generic CuteDSL LL BF16 router GEMM (#42562); TRTLLM BF16 MoE modular kernel (#45182); write FlashInfer combine into final output (#47156).
• Qwen: fuse more RMSNorm + all-reduce in Qwen3.5 (#46998), replace MoE all-reduce with reduce-scatter (#47006), Qwen3.5 H20 optimization (#48350), expand Triton warmup coverage (#47546).
• MLA: dense MHA path for short sparse-MLA sequences (#47327); MiniMax-M3 long-context decode indexer on sm100 (#48582).
• Kernels: CUDA kernel for ReLUSquaredActivation / relu^2 (#39058), Helion kernel lazy registration (#48264), vectorize _copy_mamba_state_block to uint64 (#48110), stop upcasting logits to fp32 in the sampler (#48641).
• ROCm: fp32 head_dtype torch.mm fast path (#48688), DSv4 two-stage compressor kernel (#47718), sparse decode/prefill optimizations (#48519, #48788, #46275), DSv3.2 sparse MLA KV-split heuristic (#46832) and MTP CUDA-graph mode (#45149), MXFP8 GEMM for MiniMax-M3 (#46117), AITER sparse paged attention + spec decode for MiniMax-M3 (#47287, #47984), MiniMax-M2 fused QK-norm + all-reduce via AITER (#44849), HybridW4A16 linear kernel (#40977), Qwen3-30B-A3B QK-Norm+RoPE+KV runtime fusion (#42749).
• XPU: batch-invariant kernels (#41934), HND KV layout support (#47975), DSpark spec decode for DSv4 (#47677), nightly/release image publishing (#47880, #48126).
• CPU: DFlash speculative decoding for GDN models on CPU (#46090), s390x NUMA topology (#40714), native macOS arm64 CPU wheel builds (#48289); POWER VSX math function optimization (#47321) and IBM Power docker builds using prebuilt wheels (#46017).
• Distributed fusion: FlashInfer MNNVL all-reduce RMS quant fusion (#48064).
• Build/autotune: arm64 Blackwell SM10x/SM110 image builds (#48041); skip CuTeDSL fp4_gemm autotuning by default (#48268).

Large Scale Serving & Distributed

• Decode Context Parallel (DCP): hybrid attention support (#40996), DCP + Eagle for Tokenspeed MLA backends (#48180).
• PD disaggregation: NIXL pipeline-parallel prefill in push mode (#45880).
• Encoder-cache connectors: EC transfer params (#42433) and CPU-offloading EC connector (#47423).

Quantization

• Humming w[2-7]a[4,8] weight-only inference with compressed-tensors (#46390); int4 quantization for the emulation MoE backend (#48451); INT2 XPU weight-only quant linear (#47521).
• NVFP4/MXFP4: nvfp4_per_token online MoE quantization (#48538), CuTe-DSL FlashInfer MXFP4 quantization (#48417); bounded peak memory when repacking FP4 MoE weights for Marlin (#47851) and for NVFP4 MoE weight loading (#46276).
• MLA: kv_cache_dtype_skip_layers support (#47309).
• ROCm: HybridW4A16 linear kernel (#40977).

API & Frontend

• Rust frontend: multimodal video (#47959) and audio (#48554), Seed-OSS tool parser (#47741), native vllm-bench port (#48107), continue_final_message handling with renderer sentinel (#47844).
• OpenAI compatibility: bad_words in /v1/completions (#46793), expose logprob_token_ids on Python OpenAI endpoints (#43463), include_reasoning param for non-Harmony models (#44301), populate num_cache_creation_tokens on Messages responses (#48535).