Fish Audio发布Drama 3：35B开源MoE TTS模型
Fish Audio Releases Drama 3, a 35B Open-Weight MoE Text-to-Speech Model
Drama 3是首个提供完整表演指令控制的大规模开源TTS模型，对有声书制作和影视配音场景有直接价值，建议关注其多角色同步渲染能力。
This week, Fish Audio released Fish Audio Drama 3. Drama 3 is a text-to-speech model from Fish Audio that takes plain-language acting direction. It is a 35B mixture-of-experts model with about 3B parameters active per token. The weights are on Hugging Face, and the model is also live in the Fish Audio API. For developers, it is a self-hostable option for scripted dialogue, audiobooks and dubbing.
本周，Fish Audio 发布了 Fish Audio Drama 3。Drama 3 是 Fish Audio 推出的一款文本转语音模型，支持自然语言表演指导。它是一个拥有 35B 总参数的混合专家（MoE）模型，每个 token 约有 3B 参数处于激活状态。权重已发布在 Hugging Face 上，该模型也已上线 Fish Audio API。对于开发者而言，它是用于剧本对话、有声书和配音的自托管选项。
TL;DR
TL;DR（太长不看）
- Size: 35B total parameters, about 3B active per token (8 of 256 experts plus 1 shared). Context length: Not disclosed.
- Runs on: 1 NVIDIA GPU with at least 48 GB in FP8. No CPU, Apple MPS or AMD ROCm. Official benchmarks use 1 H200.
- Performance: Fish Audio’s highest-scoring model on Artificial Analysis, but still behind several closed APIs on quality Elo.
- Best: 95.9% on brands, places and technical terms in Artificial Analysis Pronunciation Robustness.
- Worst: 86.3% on exact sequences (codes, paths, emails), its weakest pronunciation category.
- Bottom line:
- Best: directable, multi-speaker speech with open weights.
- Worst: 48 GB GPU floor and a revenue-capped license.
- 规模：35B 总参数，每个 token 约 3B 活跃参数（256 个专家中的 8 个加上 1 个共享层）。上下文长度：未公开。
- 运行环境：至少需要 1 块配备 48 GB 显存的 NVIDIA GPU（FP8 精度）。不支持 CPU、Apple MPS 或 AMD ROCm。官方基准测试使用 1 块 H200。
- 性能：Fish Audio 在 Artificial Analysis 评分最高的模型，但在质量 Elo 评分上仍落后于几款闭源 API。
- 最佳表现：在 Artificial Analysis 发音鲁棒性测试中，品牌、地名和技术术语的正确率达 95.9%。
- 最差表现：精确序列（代码、路径、电子邮件）的正确率为 86.3%，这是其发音能力最弱的类别。
- 总结：
- 最佳：支持直接指令的多说话人语音，且权重开放。
- 最差：最低需 48 GB GPU 显存，且许可证限制收入上限。
What is Fish Audio Drama 3?
什么是 Fish Audio Drama 3？
Drama 3 turns a script along with bracketed directions into acted speech. Writers place cues like [barely holding it together] before a line. Paired tags such as <whisper> change only the enclosed words, per the Drama 3 docs. Directions can be written in any language, independent of the spoken line.
Drama 3 将剧本与方括号内的指示转化为演绎语音。作者可以在台词前放置如 [barely holding it together]（勉强撑住）之类的提示。根据 Drama 3 文档，成对的标签如 <whisper> 仅改变被包围的单词内容。指示可以用任何语言编写，与口语台词无关。
The model supports 83 languages. Speaker tags <|speaker:0|> to <|speaker:15|> allow up to 16 characters in 1 request. The API takes 1 voice ID per speaker, so a full scene renders in a single call.
该模型支持 83 种语言。说话人标签从 <|speaker:0|> 到 <|speaker:15|>，允许单次请求中包含最多 16 个角色。API 为每个说话人接受 1 个声音 ID，因此整个场景可通过单次调用渲染完成。
How does the Drama 3 architecture work?
Drama 3 的架构是如何工作的？
According to the launch post, the backbone starts from the Qwen3.5-35B-A3B language model. It has 40 layers, grouped as 3 Gated DeltaNet layers per 1 gated full-attention layer. Only 1 layer in 4 keeps a KV cache. DeltaNet layers store history in a constant-size state instead.
根据发布博文，骨干网络基于 Qwen3.5-35B-A3B 语言模型构建。它包含 40 层，分组方式为每 1 层门控全注意力层搭配 3 层门控 DeltaNet 层。4 层中仅有 1 层保留 KV 缓存，而 DeltaNet 层则以恒定大小的状态存储历史。
A new Codebook-MTP decoder replaces the per-frame Fast AR decoder used in S2. It has 4 shared layers feeding 9 layers, 1 per acoustic codebook. Only codebooks 0 and 1 feed back into the backbone; S2 fed back all 10.
新的 Codebook-MTP 解码器取代了 S2 中使用的逐帧快速自回归（Fast AR）解码器。它包含 4 个共享层，分别连接 9 个层，每个声学码本对应 1 层。只有码本 0 和 1 反馈回骨干网络；S2 则反馈所有 10 个码本。
The audio tokenizer is unchanged from S2. It is built on the Descript Audio Codec and outputs 44.1 kHz audio. It runs at about 21.5 frames per second.
音频分词器与 S2 相同。它基于 Descript Audio Codec 构建，输出 44.1 kHz 音频。运行速度约为每秒 21.5 帧。
Training ran 3 pre-training stages, then instruction tuning, then reinforcement learning. RL used GRPO without a KL penalty and a single learned human-preference reward model.
训练过程包括 3 个预训练阶段，随后是指令微调，最后是强化学习。强化学习使用 GRPO（无 KL 惩罚），并采用单个经过学习的偏好奖励模型。
How does Drama 3 perform on benchmarks?
Drama 3 在基准测试中的表现如何？
Fish Audio reports Drama 3 at 93.2% overall on Artificial Analysis Pronunciation Robustness, ranked #1. Its predecessor S2.1 Pro scored 88.1%. Human listeners judge hard English words across 4 categories:
Fish Audio 在 Artificial Analysis Pronunciation Robustness（发音鲁棒性）基准测试中，Drama 3 整体得分 93.2%，排名第一。其前身 S2.1 Pro 得分为 88.1%。人类听者对跨 4 类类别的难读英文单词进行评判：
- Brands, places, technical terms: 95.9% (S2.1 Pro: 87.3%)
- Context-dependent readings: 95.3% (86.1%)
- Shorthand such as numbers, dates and units: 93.6% (92.7%)
- Exact sequences such as codes, paths and emails: 86.3% (79.4%)
- 品牌、地名、技术术语：95.9%（S2.1 Pro：87.3%）
- 依赖上下文的读音：95.3%（86.1%）
- 缩写形式，如数字、日期和单位：93.6%（92.7%）
- 精确序列，如代码、路径和电子邮件：86.3%（79.4%）
This benchmark tests pronunciation, not emotional delivery. Fish Audio describes emotive quality only qualitatively. On the Artificial Analysis quality leaderboard, ‘Drama 3 Preview’ holds an Elo of 1242. That beats S2.1 Pro at 1190 but trails Eleven v4 Turbo at 1328.
该基准测试评估的是发音，而非情感表达。Fish Audio 仅以定性方式描述情感质量。在 Artificial Analysis 质量排行榜上，“Drama 3 Preview”的 Elo 值为 1242。这高于 S2.1 Pro 的 1190，但低于 Eleven v4 Turbo 的 1328。
What hardware does Drama 3 need?
Drama 3 需要什么样的硬件？
The Fish Speech install guide lists at least 48 GB of NVIDIA GPU memory in FP8. CPU, Apple MPS and AMD ROCm are not supported. MoE experts ship in FP8, with the remaining weights in bf16.
Fish Speech 安装指南要求至少配备 48 GB FP8 格式的 NVIDIA GPU 显存。不支持 CPU、Apple MPS 和 AMD ROCm。MoE（混合专家）专家模块以 FP8 格式提供，其余权重为 bf16。
The model card reports streaming results on 1 H200. At 1 stream, median time to first audio is 88.9 ms with a real-time factor of 0.131. At 16 streams, throughput reaches 927.6 frames per second. Serving runs through SGLang-Omni, whose fused kernels are validated only on Hopper GPUs.
模型卡片报告了在单张 H200 GPU 上的流式处理结果。在 1 个并发流下，首音延迟中位数为 88.9 毫秒，实时因子（RTF）为 0.131。在 16 个并发流下，吞吐量达到每秒 927.6 帧。服务通过 SGLang-Omni 运行，其融合内核仅在 Hopper GPU 上经过验证。
How do you access Drama 3, and what does it cost?
如何访问 Drama 3，费用是多少？
In the hosted API, set the model header to drama-3 on POST /v1/tts. The docs warn that a missing or misspelled header silently falls back to s2.1-pro. List price is $50 per 1M characters. A $10 launch price runs until November 12, 2026, per the launch post.
在托管 API 中，需在 POST /v1/tts 请求中将模型头设置为 drama-3。文档警告称，如果缺失或拼写错误，该头字段会静默回退到 s2.1-pro。标准价格为每 100 万字符 50 美元。根据发布帖，10 美元的促销价格将持续至 2026 年 11 月 12 日。
The weights use the Fish Audio Community License v1.0. Research and non-commercial use are free. Commercial use is free under USD 500,000 in annual revenue. Hosting Drama 3 as a third-party API needs a separate license. Products must display ‘Built with Fish Audio.’ Cloning a real voice without consent is prohibited.
权重采用 Fish Audio Community License v1.0 协议。研究和非商业用途免费。年收入低于 50 万美元的商业使用也免费。作为第三方 API 托管 Drama 3 需要单独获得许可。产品必须显示“Built with Fish Audio”。未经同意克隆真实声音是被禁止的。
Drama 3 vs closest competitors
Drama 3 与最接近的竞争者对比
| Feature | Fish Audio Drama 3 | Fish Audio S2 Pro | ElevenLabs Eleven v4 Turbo | Qwen-Audio-3.0-TTS-Plus |
|---|---|---|---|---|
| Total size | 35B (MoE) | 5B (4B Slow AR + 400M Fast AR) | Not disclosed | Not disclosed |
| Active parameters | ~3B | Not MoE | Not disclosed | Not disclosed |
| Weights | Open, Community License v1.0 | Open, Research License | API and apps | API (Model Studio) |
| Languages | 83 | 80+ | 90+ | 8 listed, plus dialects |
| Self-host hardware | 48 GB+ NVIDIA, FP8 | 24 GB+ NVIDIA, or CPU | Hosted only | Hosted only |
| Latency | 88.9 ms TTFA p50, H200 | ~100 ms TTFA, H200 | ~150 ms median time to first speech | Not disclosed |
| AA quality Elo (source) | 1242 (Preview) | 1116 | 1328 | 1266 |
| API price per 1M chars | $50 list, $10 to Nov 12 | $15 | $40 | $20, Singapore |
| 功能 | Fish Audio Drama 3 | Fish Audio S2 Pro | ElevenLabs Eleven v4 Turbo | Qwen-Audio-3.0-TTS-Plus |
|---|---|---|---|---|
| 总参数量 | 35B (MoE) | 5B (4B 慢速自回归 + 400M 快速自回归) | 未公开 | 未公开 |
| 活跃参数 | ~3B | 非 MoE | 未公开 | 未公开 |
| 权重许可 | 开源，社区许可 v1.0 | 开源，研究许可 | API 及应用 | API (Model Studio) |
| 支持语言 | 83 种 | 80+ 种 | 90+ 种 | 列出 9 种，外加方言 |
| 自托管硬件要求 | 48 GB+ NVIDIA 显存，FP8 精度 | 24 GB+ NVIDIA 显存，或 CPU | 仅托管服务 | 仅托管服务 |
| 延迟 | 88.9 ms TTFA p50，H200 | ~100 ms TTFA，H200 | ~150 ms 中位数首语音时间 | 未公开 |
| AA 质量 Elo (来源) | 1242 (预览版) | 1116 | 1328 | 1266 |
| API 每百万字符价格 | 标价 $50，至 11 月 12 日 $10 | $15 | $40 | $20，新加坡 |
Key Takeaways
关键要点
- Drama 3 is a 35B MoE TTS model with ~3B active parameters.
- Plain-language direction and 16 speakers per request target scripted audio.
- 93.2% on AA Pronunciation Robustness; exact sequences lag at 86.3%.
- Self-hosting needs 48 GB+ of NVIDIA memory in FP8.
- Free commercial use stops at USD 500,000 in annual revenue.
- Drama 3 是一个拥有约 3B 活跃参数的 35B MoE TTS 模型。
- 通过自然语言指令和每次请求 16 个说话人，针对剧本音频进行优化。
- 在 AA 发音鲁棒性测试中得分为 93.2%；精确序列测试得分较低，为 86.3%。
- 自托管需要至少 48 GB 的 NVIDIA 显存（FP8 精度）。
- 免费商用额度上限为年收入 50 万美元。
Check out the weights on Hugging Face, the technical report, the GitHub repo and the launch post. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
请在 Hugging Face 查看权重，阅读技术报告、GitHub 仓库以及发布博文。所有功劳归于该项目的研究者。此外，欢迎在 Twitter 上关注我们，别忘了加入我们有 15 万+成员的 ML SubReddit，并订阅我们的新闻通讯。等等！你在 Telegram 上吗？现在你也可以在 Telegram 上加入我们。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力