Cloudflare发布多模态决策模型Clef-omni
Meet Cloudflare Clef-omni: A 30B-A3B Multimodal Decision Model for Audio, Video, Image and Text
多模态决策模型落地Agent开发的关键基础设施，一文打通音视频处理链路，值得开发者关注。
Cloudflare has released Clef-omni, a new open-weight member of its Clef decision model family. Clef-omni is a Cloudflare decision model that scores text, images, audio and video in 1 call. It arrived on October 9, 2026, 8 days after the original Clef and Clef-flash launch. The same update cut Clef-flash pricing below TypeSafe’s Jev and made hosted Clef roughly 2x faster. For developers, this removes the transcription and captioning steps that multimodal agent pipelines usually need.
Cloudflare 发布了 Clef-omni，这是其 Clef 决策模型家族的新成员，采用开放权重。Clef-omni 是一款 Cloudflare 决策模型，可在一次调用中对文本、图像、音频和视频进行评分。它于 2026 年 10 月 9 日上线，比最初的 Clef 和 Clef-flash 晚了 8 天。此次更新还将 Clef-flash 的价格降至低于 TypeSafe 的 Jev，并使托管版 Clef 的速度提升约 2 倍。对于开发者而言，这消除了多模态智能体管道通常需要的转录和字幕生成步骤。
TL;DR
TL;DR（太长不看）
- Size: 30B total, 3B active parameters (MoE); 64,000-token context on Workers AI
- Runs on: Workers AI as @cf/cloudflare/clef-omni; self-hosting tested on a single H200, about 64 GB GPU memory in bfloat16
- Performance: Close to Clef on routing and intent tasks, behind it on most reasoning and workflow evals
- Best: 97.7 macro-F1 on CLINC150+OOS, top of the 4 Clef and Jev models
- Worst: 2.6 accuracy on POP909-CL, and 47.4 on GPQA Diamond versus Jev’s 78.3
- Bottom line:
- Best: one call decides across 4 modalities.
- Worst: weaker accuracy than text-focused Clef on hard reasoning.
- 规模：总计 30B，活跃参数 3B（MoE）；Workers AI 上支持 64,000 token 上下文
- 运行平台：Workers AI 上的 @cf/cloudflare/clef-omni；自托管测试在单张 H200 上进行，bfloat16 精度下约需 64 GB GPU 显存
- 性能：在路由和意图任务上接近 Clef，在大多数推理和工作流评估中略逊于 Clef
- 最佳表现：在 CLINC150+OOS 上达到 97.7 macro-F1，位居 4 款 Clef 和 Jev 模型之首
- 最差表现：在 POP909-CL 上准确率为 2.6，在 GPQA Diamond 上为 47.4，而 Jev 为 78.3
- 结论：
- 最佳：一次调用即可跨 4 种模态做出决策。
- 最差：在复杂推理任务上，准确率弱于专注于文本的 Clef。
What is a decision model?
什么是决策模型？
A decision model returns typed answers with probabilities instead of generated text. You send a state and a schema of questions. The model returns a probability for every allowed option. Clef supports 3 question types: noul (true or false), choice and score. The format follows the Jev and SystemOne API from TypeSafe AI.
决策模型返回带有概率的类型化答案，而非生成的文本。你发送一个状态和问题模式（schema）。模型会为每个允许的选项返回概率。Clef 支持 3 种问题类型：noul（真或假）、choice（选择）和 score（评分）。格式遵循 TypeSafe AI 的 Jev 和 SystemOne API。
How does Clef-omni work?
Clef-omni 如何工作？
Clef-omni is post-trained from Qwen3-Omni-30B-A3B-Instruct. Cloudflare keeps the comprehension backbone and drops the text-to-speech output path. The model runs a single prefill pass over the full payload. Video frames and audio are synced into 1 sequence for joint processing.
Clef-omni 基于 Qwen3-Omni-30B-A3B-Instruct 进行后训练。Cloudflare 保留了理解主干网络，并去除了文本转语音输出路径。模型对整个载荷执行单次预填充（prefill）传递。视频帧和音频被同步到一个序列中以进行联合处理。
A small joint schema head then scores the options. Each valid option first gathers evidence from the input. Field vectors then cross-attend across the full context to compute confidence scores. Cloudflare froze the Qwen3 backbone and trained LoRA adapters. Training combined label-smoothed cross-entropy with Brier score calibration.
随后，一个小型联合模式头对选项进行评分。每个有效选项首先从输入中收集证据。字段向量然后在全局上下文中交叉注意力计算置信度分数。Cloudflare 冻结了 Qwen3 主干网络并训练了 LoRA 适配器。训练结合了标签平滑交叉熵与 Brier 分数校准。
Supported inputs on Workers AI, per the developer docs:
根据开发者文档，Workers AI 支持的输入：
- Images: up to 4, PNG, JPEG or WebP, 64 to 1,024 tokens each
- Audio: up to 4 clips, WAV or MP3, 300 seconds each, about 780 tokens per minute
- Video: up to 2 clips, MP4 or WebM, 60 seconds each, sampled at 2 frames per second
- Questions: 1 to 64 per request
- 图像：最多 4 张，PNG、JPEG 或 WebP，每张 64 到 1,024 tokens
- 音频：最多 4 段，WAV 或 MP3，每段 300 秒，每分钟约 780 tokens
- 视频：最多 2 段，MP4 或 WebM，每段 60 秒，每秒采样 2 帧
- 问题：每次请求 1 到 64 个
Cloudflare reports text decisions in about 130 ms at the median and images in about 150 ms. A 21-second video with sound scores in about 1.5 seconds.
Cloudflare 报告称，中位数情况下文本决策耗时约 130 毫秒，图像约 150 毫秒。一段带声音的 21 秒视频评分耗时约 1.5 秒。
How does Clef-omni perform on benchmarks?
Clef-omni 在基准测试中的表现如何？
Cloudflare compared Clef-omni against Clef, Clef-flash and Jev on its Decision Index run. Clef-omni leads on intent classification: 94.8 macro-F1 on BANKING77 and 97.7 on CLINC150+OOS. It also tops Amazon ESCI at 57.8.
Cloudflare 在其 Decision Index 运行中，将 Clef-omni 与 Clef、Clef-flash 和 Jev 进行了对比。Clef-omni 在意图分类方面领先：在 BANKING77 上达到 94.8 的宏平均 F1 分数，在 CLINC150+OOS 上达到 97.7。它在 Amazon ESCI 上也以 57.8 分位居榜首。
It trails elsewhere. Home appliances case-exact accuracy is 69.3, versus 97.73 for Clef-flash. When2Call accuracy is 63.3, versus 80.97 for Jev. On TypeSafe’s workflow evals, Clef-omni scores below Clef in all 5 reported metrics.
在其他方面它则落后。家电案例精确准确率仅为 69.3，而 Clef-flash 为 97.73。When2Call 准确率为 63.3，而 Jev 为 80.97。在 TypeSafe 的工作流评估中，Clef-omni 在所有 5 项报告指标中的得分均低于 Clef。
Clef-omni vs Clef vs Clef-flash vs Jev
| Feature | Clef-omni | Clef | Clef-flash | Jev |
|---|---|---|---|---|
| Developer | Cloudflare | Cloudflare | Cloudflare | TypeSafe AI |
| Parameters | 30B total, 3B active (MoE) | 27B dense | 9B dense | Not disclosed |
| Base model | Qwen3-Omni-30B-A3B-Instruct | Qwen3.8-27B | Qwen3.5-9B | Not disclosed |
| Inputs | Text, image, audio, video | Text, image, video frames | Text, image, video frames | Text (per Cloudflare) |
| Hosted context | 64,000 tokens | 64k | 24k (256k self-hosted) | Not disclosed by TypeSafe (Cloudflare cites 32k) |
| Input price per M tokens | $0.15 | $0.24 | $0.038 | $0.042 |
| License | Apache-2.0 | Apache-2.0 | Apache-2.0 | API only, early access |
| BANKING77 (macro-F1) | 94.8 | 94.20 | 90.93 | 79.74 |
| CLINC150+OOS (macro-F1) | 97.7 | 97.43 | 66.77 | 89.27 |
| When2Call (accuracy) | 63.3 | 72.37 | 65.58 | 80.97 |
| BRIGHT (nDCG@10) | 42.0 | 45.91 | 39.26 | 47.52 |
| Invoice processing, primary action | 82.0 | 86.2 | 73.3 | 83.1 |
| 特性 | Clef-omni | Clef | Clef-flash | Jev |
|---|---|---|---|---|
| 开发者 | Cloudflare | Cloudflare | Cloudflare | TypeSafe AI |
| 参数量 | 总计 30B，活跃 3B (MoE) | 27B 稠密模型 | 9B 稠密模型 | 未披露 |
| 基础模型 | Qwen3-Omni-30B-A3B-Instruct | Qwen3.8-27B | Qwen3.5-9B | 未披露 |
| 输入类型 | 文本、图像、音频、视频 | 文本、图像、视频帧 | 文本、图像、视频帧 | 文本（据 Cloudflare） |
| 托管上下文长度 | 64,000 tokens | 64k | 24k（自托管可达 256k） | TypeSafe 未披露（Cloudflare 引用的是 32k） |
| 每百万 tokens 输入价格 | $0.15 | $0.24 | $0.038 | $0.042 |
| 许可证 | Apache-2.0 | Apache-2.0 | Apache-2.0 | 仅限 API，早期访问 |
| BANKING77 (macro-F1) | 94.8 | 94.20 | 90.93 | 79.74 |
| CLINC150+OOS (macro-F1) | 97.7 | 97.43 | 66.77 | 89.27 |
| When2Call (accuracy) | 63.3 | 72.37 | 65.58 | 80.97 |
| BRIGHT (nDCG@10) | 42.0 | 45.91 | 39.26 | 47.52 |
| 发票处理，主要动作 | 82.0 | 86.2 | 73.3 | 83.1 |
Benchmark scores are from Cloudflare’s announcement and were not independently verified.
基准测试分数来自 Cloudflare 的公告，未经独立验证。
How much faster is Clef?
Clef 的速度提升了多少？
Cloudflare moved hosted Clef to SGLang and tuned its serving stack. No new weights were released. Median latency fell from 262 ms to 152 ms at about 800 tokens, a 1.7x gain. At about 3,400 tokens it fell from 616 ms to 305 ms, a 2.0x gain. At about 16,000 tokens it dropped from 2,721 ms to 1,635 ms. Clef support lands in SGLang 0.5.22.
Cloudflare 将托管版 Clef 迁移至 SGLang 并优化了其推理服务栈。未发布新的权重。在约 800 tokens 处，中位延迟从 262 ms 降至 152 ms，提升了 1.7 倍。在约 3,400 tokens 处，从 616 ms 降至 305 ms，提升了 2.0 倍。在约 16,000 tokens 处，从 2,721 ms 降至 1,635 ms。Clef 的支持功能已落地于 SGLang 0.5.22 版本。
Key Takeaways
关键要点
- Clef-omni scores text, images, audio and video in 1 decision call.
- It is a 30B-A3B MoE on Qwen3-Omni, Apache-2.0, at $0.15 per million tokens.
- Clef-flash now costs $0.038, under Jev’s $0.042, with 24k hosted context.
- Hosted Clef is 1.7x to 2.0x faster at the median after moving to SGLang.
- Clef-omni wins intent tasks but trails Clef on reasoning and workflow evals.
- Clef-omni 在一次决策调用中即可处理文本、图像、音频和视频。
- 它是基于 Qwen3-Omni 的 30B-A3B MoE 模型，采用 Apache-2.0 许可证，价格为每百万 tokens $0.15。
- Clef-flash 现在的价格为 $0.038，低于 Jev 的 $0.042，并提供 24k 的托管上下文长度。
- 托管版 Clef 在迁移至 SGLang 后，中位速度提升了 1.7 到 2.0 倍。
- Clef-omni 在意图任务中胜出，但在推理和工作流评估中落后于 Clef。
Check out the Cloudflare announcement, model weights and Workers AI docs. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
查看 Cloudflare 的公告、模型权重和 Workers AI 文档。本项目的所有功劳归于其研究者。此外，欢迎在 Twitter 上关注我们，别忘了加入我们拥有 15 万+成员的 ML SubReddit 并订阅我们的新闻通讯。等等！你在 Telegram 上吗？现在你也可以加入我们的 Telegram 群组了。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力