阿里发布Qwen-Audio-3.1全双工语音模型，API价格大幅下调
Alibaba Qwen Releases Qwen-Audio-3.1-Realtime: A Full-Duplex Voice Model Trained to Think, Act, and Decide When to Speak
全双工语音交互是Agent落地的关键瓶颈，这次不仅给出了完整的训练架构细节，API价格还砍了大半，做语音Agent的同学值得重点关注。
Alibaba’s Qwen team has released Qwen-Audio-3.1, a 5-model audio stack spanning ASR, TTS and realtime interaction. The main model is Qwen-Audio-3.1-Realtime, a full-duplex speech model built for voice agents that call tools. Qwen also cut prices: about 85% on Realtime, about 70% on TTS and up to 95% on ASR.
阿里巴巴 Qwen 团队发布了 Qwen-Audio-3.1，这是一个涵盖 ASR、TTS 和实时交互的 5 模型音频栈。主模型是 Qwen-Audio-3.1-Realtime，这是一个为调用工具的智能语音助手设计的全双工语音模型。Qwen 还降低了价格：Realtime 降价约 85%，TTS 降价约 70%，ASR 最高降价 95%。
Is it deployable? Yes, as a managed API. qwen-audio-3.1-realtime-plus is live on QwenCloud over WebSocket. No open weights were announced.
可以部署吗？可以，作为托管 API 使用。qwen-audio-3.1-realtime-plus 已在 QwenCloud 上通过 WebSocket 上线。未公布开源权重。
What Ships on QwenCloud
QwenCloud 上提供哪些服务
The model page lists text and audio as both input and output. Context is 262K tokens, with 245K max input and 16K max output. Default limits are 60 requests and 100K tokens per minute. Pricing is $6.4 per 1M audio input tokens and $0.8 per 1M text input tokens. Text and audio output costs $24 per 1M tokens, with output text not charged. Key features include function calling, web search, structured outputs, context cache and fine-tuning.
模型页面显示文本和音频均可作为输入和输出。上下文长度为 262K tokens，最大输入 245K，最大输出 16K。默认限制为每分钟 60 次请求和 100K tokens。定价为每 1M 音频输入 tokens 6.4 美元，每 1M 文本输入 tokens 0.8 美元。文本和音频输出的费用为每 1M tokens 24 美元，其中文本输出不收费。主要功能包括函数调用、网络搜索、结构化输出、上下文缓存和微调。
A companion model, Qwen-Audio-3.1-ASR-Flash-Filetrans, targets offline long-audio transcription. It supports hot words, speaker separation, punctuation and multilingual plus Chinese dialect recognition. It costs $0.15 input and $0.47 output per 1M tokens.
一个配套模型 Qwen-Audio-3.1-ASR-Flash-Filetrans 针对离线长音频转录。它支持热词、说话人分离、标点符号和多语言及中文方言识别。其价格为每 1M tokens 输入 0.15 美元，输出 0.47 美元。
Architecture: 2 Models Behind 1 Voice
架构：1 个语音背后的 2 个模型
The system runs 2 models with the same Audio Encoder and LLM design. A full-duplex decision model predicts whether to keep listening, speak, stop or resume. A speech-to-text model writes the response content as text. A context-aware voice renderer then turns that text into streaming speech. It conditions on conversation history, voice cues and acoustic context.
该系统运行 2 个具有相同 Audio Encoder 和 LLM 设计的模型。一个全双工决策模型预测是继续监听、说话、停止还是恢复。一个语音转文本模型将响应内容写为文本。随后，一个上下文感知的语音渲染器将该文本转换为流式语音。它基于对话历史、语音提示和声学上下文进行条件生成。
Training is organized into 3 layers: Think, Act, and Speak and Coordinate.
训练分为 3 层：Think（思考）、Act（行动）以及 Speak and Coordinate（说话与协调）。
Think: M²-OPD
Think：M²-OPD
Core-Cocktail SFT re-anchors the audio model to its source text LLM using million-hour-scale paired data. Multimodality OPD follows. A Text Teacher and a frozen Audio Reference score each token of the student’s own trajectory. This is on-policy distillation, not imitation of pre-written answers. Domain experts for empathy, pragmatic intent and acoustic scenes are then trained with GRPO. Multi-Teacher OPD merges them into 1 deployable model.
Core-Cocktail SFT 使用百万小时规模的配对数据，将音频模型重新锚定到其源文本 LLM 上。随后进行多模态 OPD。Text Teacher 和冻结的 Audio Reference 对学生自己的轨迹中的每个 token 进行评分。这是策略内蒸馏，而非模仿预写答案。然后利用 GRPO 对共情、语用意图和声学场景领域的专家进行训练。Multi-Teacher OPD 将它们合并为一个可部署模型。
Act: Executable Environments
Act：可执行环境
Each training domain bundles a tool pool, a stateful JSON database and a natural-language business policy. Domains are seeded from open-source tool and MCP server definitions. Every task defines 1 of 3 outcomes: a write, a justified refusal, or an unsupported request. Scoring checks terminal state, then permitted writes, then behavioral assertions. A fluent reply cannot rescue a failed state check.
每个训练域捆绑了一个工具池、一个有状态的 JSON 数据库和自然语言业务策略。域由开源工具和 MCP 服务器定义初始化。每个任务定义三种结果之一：写入、有理拒绝或不受支持请求。评分检查终端状态，然后检查允许的写入，最后检查行为断言。流畅的回复无法挽救失败的狀態检查。
GRPO receives rewards at dialogue, milestone and turn level. Search training penalizes redundant queries with rquery=qmin(1,nrefnpred)r_{\text{query}} = q \min \left( 1, \frac{n_{\text{ref}}}{n_{\text{pred}}} \right). Mean queries per search call fell from 4.37 to 1.05. Trigger F1 slipped from 60.87% to 58.61%.
GRPO 在对话、里程碑和回合级别接收奖励。搜索训练使用 r_query = q min(1, n_ref / n_pred) 对冗余查询进行惩罚。每次搜索调用的平均查询数从 4.37 降至 1.05。触发 F1 分数从 60.87% 降至 58.61%。
Speak and Coordinate
说话与协调
This layer decides whether, when and how to speak. On Full-Duplex-Bench v1.5, replies to people talking to someone else fell from 0.13 to 0.03. On v3.0, the filler rate dropped from 0.7590 to 0.2960. There are trade-offs. After interruptions, the unwanted resume rate rose from 0.035 to 0.130. Interruption stop latency is 1.116 seconds, versus 0.383 for GPT-Realtime-2.
该层决定何时、是否以及如何说话。在 Full-Duplex-Bench v1.5 上，对正在与他人交谈的人的回复率从 0.13 降至 0.03。在 v3.0 上，填充词率从 0.7590 降至 0.2960。存在权衡。中断后，非期望恢复率从 0.035 升至 0.130。中断停止延迟为 1.116 秒，而 GPT-Realtime-2 为 0.383 秒。
Interactive Explainer
交互式解释器
Explore the Think, Act, Speak loop, duplex decisions, a scored training episode and the search reward.
探索思考、行动、说话的循环，双工决策，带评分的训练片段以及搜索奖励。
Benchmarks at a Glance
基准测试概览
Against 3.0, Audio MultiChallenge rises from 47.12 to 52.21. The 14-language BBA average climbs from 81.7% to 88.1%. FLEURS WER falls from 9.01 to 3.98. The τ-Voice figures use a half-duplex speech-to-text adaptation. They are not comparable to official full-duplex results. GPT-Realtime-2 still leads the 50-session human red-team study, 96.00% versus 92.00%.
与 3.0 相比，Audio MultiChallenge 得分从 47.12 升至 52.21。14 种语言的 BBA 平均分从 81.7% 升至 88.1%。FLEURS WER 从 9.01 降至 3.98。τ-Voice 数据采用半双工语音转文本适配。它们与官方全双工结果不可比。GPT-Realtime-2 仍在 50 次会话的人类红队研究中领先，得分为 96.00%，而本模型为 92.00%。
How It Compares
对比情况
| Feature | Qwen-Audio-3.1-Realtime-Plus | OpenAI GPT-Realtime-2 | Google Gemini 3.8 Live |
|---|---|---|---|
| Input | Text, audio | Text, audio, image | Text, images, audio, video |
| Output | Text, audio | Text, audio | Text and audio |
| Context / input limit | 262K | 128K | 131,072 |
| Max output | 16K | 32K | 65,536 |
| Function calling | Yes | Yes | Yes (async by default) |
| Built-in web search | Yes | Not listed | Google Search grounding |
| Reasoning | Thinking mode, 2K max reasoning | Configurable effort | Interleaved reasoning |
| Audio input / 1M tokens | $6.4 | $32 | $3.00 |
| Audio output / 1M tokens | $24 (text and audio) | $64 | $12.00 |
| Open weights | No | No | No |
| Source | QwenCloud | OpenAI docs | Model page, pricing |
| 功能 | Qwen-Audio-3.1-Realtime-Plus | OpenAI GPT-Realtime-2 | Google Gemini 3.8 Live |
|---|---|---|---|
| 输入 | 文本、音频 | 文本、音频、图像 | 文本、图像、音频、视频 |
| 输出 | 文本、音频 | 文本、音频 | 文本和音频 |
| 上下文/输入限制 | 262K | 128K | 131,072 |
| 最大输出 | 16K | 32K | 65,536 |
| 函数调用 | 是 | 是 | 是（默认异步） |
| 内置网络搜索 | 是 | 未列出 | Google Search grounding |
| 推理 | 思维模式，最多 2K 推理步 | 可配置努力程度 | 交错式推理 |
| 音频输入 / 1M tokens | $6.4 | $32 | $3.00 |
| 音频输出 / 1M tokens | $24（文本和音频） | $64 | $12.00 |
| 开放权重 | 否 | 否 | 否 |
| 来源 | QwenCloud | OpenAI 文档 | 模型页面、定价 |
Prices are list rates checked September 28, 2026. Token rates are not directly comparable, since each provider tokenizes audio differently.
价格为截至 2026 年 9 月 28 日查证的列表价格。Token 费率不可直接比较，因为各提供商对音频的分词方式不同。
Key Takeaways
关键要点
- Task success on a τ-Voice adaptation rises from 78.4% to 82.0% over 3.0.
- Replies to background speech drop from 73.0% to 13.0% on Full-Duplex-Bench v1.5.
- 262K context, function calling and web search at $6.4 per 1M audio input tokens.
- Tool use is trained with GRPO inside self-evolving executable environments.
- Multi-turn attack success falls to 26.0% (Chinese) and 23.5% (English).
- 在 τ-Voice 适配中，任务成功率从 78.4% 提升至 82.0%（版本 3.0）。
- 在 Full-Duplex-Bench v1.5 上，对背景语音的回复率从 73.0% 降至 13.0%。
- 支持 262K 上下文、函数调用和网络搜索，音频输入 token 价格为每百万 6.4 美元。
- 工具使用训练采用 GRPO，在自演化可执行环境中进行。
- 多轮攻击成功率降至 26.0%（中文）和 23.5%（英文）。
FAQ
常见问题解答
- What is Qwen-Audio-3.1-Realtime? A full-duplex speech model from Alibaba’s Qwen team for voice agents that reason, call tools and manage turn-taking.
- Can I self-host it? No open weights were announced. Access is through the QwenCloud API as qwen-audio-3.1-realtime-plus.
- How much does it cost? $6.4 per 1M audio input tokens and $24 per 1M output tokens for text and audio.
- 什么是 Qwen-Audio-3.1-Realtime？这是阿里巴巴通义千问团队推出的一款全双工语音模型，专为具备推理能力、可调用工具并管理对话轮次的语音智能体设计。
- 我可以自行托管吗？目前未公布开源权重。可通过 QwenCloud API 以 qwen-audio-3.1-realtime-plus 的形式访问。
- 费用是多少？音频输入 token 为每百万 6.4 美元，文本和音频输出 token 为每百万 24 美元。
Check out the Paper, Qwen-Audio-3.1-ASR, and Qwen-Audio-3.1-Realtime. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
请参阅论文、Qwen-Audio-3.1-ASR 以及 Qwen-Audio-3.1-Realtime。所有功劳归于本项目的研究人员。欢迎在 Twitter 上关注我们，别忘了加入拥有 15 万+成员的 ML SubReddit，并订阅我们的新闻通讯。等等！你也在 Telegram 上吗？现在你也可以加入我们的 Telegram 群组。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力