vLLM v0.33.0：移除Medusa与Ray执行器，CUDA升级至13.0

Highlights

This release features 927 commits from 389 contributors (122 new)!

• New models: Reflection Beam-1.0 (501B MoE, 23B active) with its DFlash2 drafter (#62074), Yandex AliceAI-Foundation-80B-A3B (#57963), Recursive Flint (40B MoE, 2.5B active) (#62938), Muse Spark Open (#61879), Step-5 Preview (#61727, #63147), NVIDIA Alpamayo 2 Super (#62957), Nemotron-Labs Dory1.0-4B (#60149), Nemotron Parse 3.5 (#61651), MiniCPM-OCR 1.0 (#63095), Domyn Edge (#62544) and the ColUltraVBERT visual document retriever (#62237).
• PyTorch 2.14: vLLM is upgraded to PyTorch 2.14.0, with torchvision 0.29.0 and Triton 3.8.0 (#52183).
• CUDA 12.9 dropped: release wheels and images are CUDA 13.0 only. The CUDA 12.9 wheels and the -cu129 / -cu129-ubuntu2404 images are no longer published (#62575); use the default CUDA 13.0 artifacts instead.
• libtorch ABI stable CUDA wheel: the CUDA wheel is now ABI stable with libtorch.
• AWS EFA image: a new vllm/vllm-openai:v0.33.0-efa image (x86_64 and aarch64) bundles the AWS EFA libraries and the EFA build of the Mooncake transfer engine, for KV transfer and multi-node serving on EFA-enabled EC2 instances (#58768).
• vllm/vllm image alias: every release image is now also published under the shorter vllm/vllm name, e.g. vllm/vllm:v0.33.0, vllm/vllm:v0.33.0-efa and vllm/vllm:v0.33.0-rocm.
• Speculative decoding: the new XPress refiner for block-diffusion drafters such as DFlash and DSpark (#54448); FLy, an opt-in lossy verification policy with 1.04-1.31x decode throughput on MI355X and B300 (#53987); DFlash/DSpark no longer reserve draft slots in the token budget, taking DeepSeek-V4.1-Flash + DSpark on GB300 from 1373 to 3390 output tok/s in the PR's benchmark (#59468); quantized draft lm_head (draft_lm_head_quantization, +18% decode at concurrency 1) (#59973); lookahead block hashes so EAGLE/MTP prefix-cache hits keep their last block, on by default (#61589); and MTP drafts in the Transformers modeling backend (#61270).
• Faster decode on NVIDIA: opt-in latency-specialized decode on GB300 for Kimi-K3 (1.5-2x at TP8/TP16, #61867) and DeepSeek-V4.1-Flash (18.3% faster decode step at batch 1, #61868); DeepSeek-V4/V4.1 MXFP4 experts run as block FP8 through Humming on Hopper (+9-21% decode on 4x H200, #61875); FA4 sparse MLA decode on Blackwell (#55866); FlashInfer DCP prefill on trtllm-gen with 39-48% lower mean TTFT at 100k input on 8xB200 (#62029); and decode-only full CUDA graphs under MRV2 prefill context parallelism (2.6x over eager, #62377).
• AMD ROCm: the default ROCm image and wheels move to ROCm 10.0, with ROCm 7.2 still published as -rocm72 (#58761); MonoKernel decode via --monokernel for DeepSeek-V4.1-Flash on MI300X/MI325X (2.32x P50 interactivity at concurrency 1, #60731) and on every MI355X layer (#61082), and for MiniMax-M3-MXFP4 on MI355X (16.6% lower TPOT, #62712, #62994); the ROCM_OCTAVE 3-4 bit KV cache (4.6x the KV tokens of fp16 on RX 7900 XTX, #59774); and gfx1250 enablement for gpt-oss-120b, DeepSeek-R1, DeepSeek-V4-Flash and MiniMax-M3 (#58800).
• API and serving: the OpenAI-compatible /v1/decisions endpoint, on by default for supported generation models (#60465, #61550); token embeddings with optional late chunking on /v1/embeddings (#61772, #63051, #60164); output_mode="derender" on /inference/v1/generate (#60082); a large Rust frontend parity push, including no more HTTP 500 on long conversations (#62374) and unsupported flags failing at startup (#62431); concurrent worker startup with TP > 1 (#60028); and a watchdog that dumps stack traces from stuck engine and worker processes (#55700).
• Breaking changes: Medusa and the MLP speculator removed (#61701); the legacy Ray compiled-graph executor removed (#61712, #62228); --mamba-cache-mode deprecated (#60043); requests sent while generation is paused now get HTTP 503 (#51488); the NIXL connector needs NIXL 1.5 or newer (#61032); LM Format Enforcer and --ir-op-priority deprecated (#63423, #63371).

Release Artifacts

Python Wheels

Docker Images

Every image is also available under the shorter vllm/vllm name: vllm/vllm:v0.33.0, vllm/vllm:v0.33.0-efa, vllm/vllm:v0.33.0-rocm (-rocm100, -rocm72), vllm/vllm:v0.33.0-cpu and vllm/vllm:v0.33.0-xpu. latest works the same way, e.g. vllm/vllm:latest-rocm.

Other Artifacts

Pre-built release artifacts are available in the Assets section at the bottom of this page, including:

• Source distribution tarball
• CUDA 13.0 Python wheels for x86_64 and arm64
• CPU Python wheels for x86_64, arm64, and macOS
• XPU Python wheel for x86_64

Model Support

• New models: Reflection Beam-1.0 (BeamForCausalLM) day-0 support with its DFlash2 drafter and fused Blackwell kernels (#62074, #62042), plus beam reasoning and tool parsers (#62052); Yandex AliceAI-Foundation-80B-A3B (Kimi Delta Attention hybrid, 512-expert MoE, 262K context) (#57963); Recursive Flint BF16 and NVFP4 GGUF checkpoints with small-batch Blackwell decode kernels (#62938); Muse Spark Open (#61879) with reasoning and tool parsers (#62207); Step-5 Preview (#61727, #62126, #62132), running on SM100 (#63147) with LoRA (#62498); NVIDIA Alpamayo 2 Super (#62957); Nemotron-Labs Dory1.0-4B (#60149); Nemotron Parse 3.5 with an MTP proposer on MRV2 (#61651); MiniCPM-OCR 1.0 (#63095); Domyn Edge (#62544); ColUltraVBERT (#62237); LFM2.5-Embedding-350M (Lfm2BidirectionalModel, #62201); Kolibri 1 through the Transformers backend (#60827); and Decision 3.0 models (#61565, #61778) and Cloudflare Clef-Flash (#61308) on /v1/systemone.
• DeepSeek-V4 and V4.1: length-aware decode indexer block selection (about 850 us saved per step at 1M context, #61039); MegaMoE experts loaded directly in the DeepGEMM layout so reload_weights works (#60096); fused_wqa_wkv split by token across TP on SM90 (#58872); MegaAttn wo_a writes MXFP8 input for wo_b (#60118); native small-head FlashMLA sparse prefill on SM90 (#61615); FlashMLA interleaved KV layout (#61371); redundant EPLB expert slots loaded (#59611); GPU Engram tables read from peer HBM, fixing DEP decode hangs on GB200 (#62091); and fixes for Engram under PP (#62208), Engram all-to-all hangs (#62269), prefill index-workspace NaNs (#60613), C128 compressed-KV writes under PIECEWISE graphs (#59709), routed-expert rows under replay (#61643, #62652) and the updated 1017 reference encoding (#62199, #62200).
• GLM: GLM-5.3-Flash defaults to an FP8 KV cache (9.8% higher end-to-end throughput in the PR's benchmark, #61228) and gets mHC TileLang decode kernels on Blackwell (#62745); GLM-5.3 per-token-group quant (2.8% ITL, #60927), in-place draft decode metadata (#61873) and skipped concat_and_cache_mla scales (1.2-3.0% TPOT on B300, #62102); FP8 indexer WK pairs kept across load_weights calls, fixing thinking loops with runai_streamer (#59223).
• Kimi-K3: KDA RecoverSSM CuTe DSL verify/commit kernels on SM10x, with verify records moved out of the KV cache so KV connectors work (#63045); online quantization of NVIDIA projections (#62980).