Workers AI Clef-omni支持音视频输入，Clef-flash降价且上下文缩至24K
Workers AI - Clef-omni adds audio and video input, Clef-flash is now cheaper, and Clef is faster
Clef-flash 托管版上下文窗口强制缩减至 24K，依赖长上下文的读者必须立即调整配置或切换至 Clef 模型，否则将导致请求截断。
@cf/cloudflare/clef-omni is now available on Workers AI. Clef-omni is a decision model that takes audio (WAV or MP3) and video (MP4 or WebM) input alongside text and images. It joins Clef and Clef-flash in the Clef family of open-weight decision models. We also cut the price of Clef-flash, so it now costs less than Jev, and made Clef faster.
@cf/cloudflare/clef-omni 现已在 Workers AI 上提供。Clef-omni 是一个决策模型，可接受音频（WAV 或 MP3）和视频（MP4 或 WebM）输入，同时支持文本和图片。它加入了 Clef 家族的开放权重决策模型行列，与 Clef 和 Clef-flash 并列。我们还降低了 Clef-flash 的价格，使其现在比 Jev 更便宜，并提升了 Clef 的速度。
Clef-omni: one decision model for every modality
Clef-omni：适用于所有模态的单一决策模型
Previously, making a decision about a voice recording or a video meant chaining models together: transcribe the speech, split the audio and visual tracks, then pass the results to a text decision model. Clef-omni reads every modality directly in one request. A video's soundtrack is aligned with its frames, so the model can reason over what is seen and heard at the same time.
以前，对语音录音或视频做出决策意味着需要将多个模型串联使用：转录语音、分离音轨和视觉轨道，然后将结果传递给文本决策模型。Clef-omni 可以在一次请求中直接读取每种模态。视频的音轨与其帧对齐，因此模型可以同时推理所见和所听的内容。
Clef-omni is built on a 30B-parameter mixture-of-experts (MoE) backbone with 3B active parameters. Like the rest of the Clef family, it does not generate text. It scores every allowed answer in a single pass, so decisions return quickly:
Clef-omni 基于拥有 30B 参数的混合专家（MoE）主干网络，其中活跃参数为 3B。与 Clef 家族的其他成员一样，它不生成文本。它在单次传递中对每个允许的答案进行评分，因此决策返回迅速：
- Text requests: about 20 ms
- Image or audio inputs: under 100 ms
- A 21-second video clip with sound: about 300 ms
- 文本请求：约 20 毫秒
- 图片或音频输入：低于 100 毫秒
- 带有声音的 21 秒视频片段：约 300 毫秒
Pass media as base64 data URLs in the images, audio, and videos fields:
将媒体作为 base64 数据 URL 放入 images、audio 和 videos 字段中：
const response = await env.AI.run("@cf/cloudflare/clef-omni", {
model: "clef-omni",
state:
"Review the installation: a photo of the unit, an audio recording of it running, and a video of the fan.",
images: ["data:image/png;base64,<base64-png>"],
audio: ["data:audio/mpeg;base64,<base64-mp3>"],
videos: ["data:video/mp4;base64,<base64-mp4>"],
questions: {
label_visible: {
type: "noul",
instructions:
"Is the model and serial number label visible in the photo?",
},
sounds_normal: {
type: "noul",
instructions:
"Does the unit sound like it is running smoothly, without rattling or grinding?",
},
fan_running: {
type: "noul",
instructions: "Is the fan running in the video?",
},
},
});const response = await env.AI.run("@cf/cloudflare/clef-omni", {
model: "clef-omni",
state:
"Review the installation: a photo of the unit, an audio recording of it running, and a video of the fan.",
images: ["data:image/png;base64,<base64-png>"],
audio: ["data:audio/mpeg;base64,<base64-mp3>"],
videos: ["data:video/mp4;base64,<base64-mp4>"],
questions: {
label_visible: {
type: "noul",
instructions:
"Is the model and serial number label visible in the photo?",
},
sounds_normal: {
type: "noul",
instructions:
"Does the unit sound like it is running smoothly, without rattling or grinding?",
},
fan_running: {
type: "noul",
instructions: "Is the fan running in the video?",
},
},
});Clef-omni scores highest of the Clef family on BANKING77, CLINC150+OOS, and Amazon ESCI:
Clef-omni 在 BANKING77、CLINC150+OOS 和 Amazon ESCI 基准测试中得分最高：
| Benchmark | Clef-omni | Clef | Clef-flash | Jev |
|---|---|---|---|---|
| BFCL (case exact) | 98.2 | 98.47 | 98.76 | 95.75 |
| BANKING77 (macro-F1) | 94.8 | 94.20 | 90.93 | 79.74 |
| CLINC150+OOS (macro-F1) | 97.7 | 97.43 | 66.77 | 89.27 |
| Amazon ESCI (macro-F1) | 57.8 | 57.48 | 57.39 | 55.21 |
| PhishNChips (accuracy) | 73.2 | 79.60 | 75.05 | 62.55 |
| 基准测试 | Clef-omni | Clef | Clef-flash | Jev |
|---|---|---|---|---|
| BFCL (case exact) | 98.2 | 98.47 | 98.76 | 95.75 |
| BANKING77 (macro-F1) | 94.8 | 94.20 | 90.93 | 79.74 |
| CLINC150+OOS (macro-F1) | 97.7 | 97.43 | 66.77 | 89.27 |
| Amazon ESCI (macro-F1) | 57.8 | 57.48 | 57.39 | 55.21 |
| PhishNChips (accuracy) | 73.2 | 79.60 | 75.05 | 62.55 |
Clef-flash is now cheaper
Clef-flash 现在更便宜了
Clef-flash now costs $0.038 per million input tokens, down from $0.090, which makes it cheaper than Jev. To offer this price, the hosted Clef-flash context window is now 24K tokens, down from 64K. Based on usage data, only 0.24% of requests exceed 24K input tokens. If you need a larger context window, use Clef, which keeps its 64K context window.
Clef-flash 现在的价格为每百万输入令牌 0.038 美元，从之前的 0.090 美元下调，这使其比 Jev 更便宜。为了提供这一价格，托管版 Clef-flash 的上下文窗口现在为 24K 令牌，从之前的 64K 下调。根据使用数据，只有 0.24% 的请求超过 24K 输入令牌。如果您需要更大的上下文窗口，请使用 Clef，它仍保持 64K 的上下文窗口。
The Clef-flash weights on Hugging Face are unchanged and support up to a 256K context window if you self-host.
Hugging Face 上的 Clef-flash 权重未变，如果您自行托管，则支持高达 256K 的上下文窗口。
| Model | Price | Context window |
|---|---|---|
| @cf/cloudflare/clef-flash | $0.038 per M input tokens | 24K tokens |
| @cf/cloudflare/clef | $0.240 per M input tokens | 64K tokens |
| @cf/cloudflare/clef-omni | $0.150 per M input tokens | 64K tokens |
| 模型 | 价格 | 上下文窗口 |
|---|---|---|
| @cf/cloudflare/clef-flash | 每百万输入令牌 $0.038 | 24K 令牌 |
| @cf/cloudflare/clef | 每百万输入令牌 $0.240 | 64K 令牌 |
| @cf/cloudflare/clef-omni | 每百万输入令牌 $0.150 | 64K 令牌 |
All Clef models convert image inputs to input tokens, and Clef-omni does the same for audio and video. For details on how each input type is tokenized, refer to the Clef, Clef-flash, and Clef-omni model pages.
所有 Clef 模型都将图像输入转换为输入令牌，Clef-omni 也对音频和视频执行相同的操作。有关每种输入类型如何被分词的详细信息，请参阅 Clef、Clef-flash 和 Clef-omni 模型页面。
Clef is now faster
Clef 现在更快了
We optimized how Clef is served on Workers AI, so it now returns decisions up to 2x faster. The model weights are unchanged.
我们优化了 Clef 在 Workers AI 上的服务方式，使其决策返回速度提升了高达 2 倍。模型权重保持不变。
| Input size | Before: median / p95 (ms) | Now: median / p95 (ms) | Median speedup |
|---|---|---|---|
| ~800 tokens | 262 / 438 | 152 / 351 | 1.7x |
| ~3,400 tokens | 616 / 777 | 305 / 531 | 2.0x |
| ~16,000 tokens | 2,721 / 3,250 | 1,635 / 1,805 | 1.7x |
| 输入大小 | 之前：中位数 / p95 (ms) | 现在：中位数 / p95 (ms) | 中位数加速比 |
|---|---|---|---|
| ~800 tokens | 262 / 438 | 152 / 351 | 1.7x |
| ~3,400 tokens | 616 / 777 | 305 / 531 | 2.0x |
| ~16,000 tokens | 2,721 / 3,250 | 1,635 / 1,805 | 1.7x |
Part of this speedup comes from moving Clef to SGLang ↗︎. Clef support is coming to SGLang in version 0.5.22 (PR #42721 ↗︎). If you self-host Clef, launch commands are available in the Clef collection on Hugging Face ↗︎.
此次加速部分得益于将 Clef 迁移至 SGLang ↗︎。Clef 支持将在 SGLang 版本 0.5.22 中推出（PR #42721 ↗︎）。如果您自行托管 Clef，可在 Hugging Face 的 Clef 集合中找到启动命令 ↗︎。
Get started
开始使用
Clef-omni follows the same System One API as Clef and Clef-flash, and works with AI Gateway. To try it, change the model ID to @cf/cloudflare/clef-omni and set the model selector to clef-omni.
Clef-omni 遵循与 Clef 和 Clef-flash 相同的 System One API，并与 AI Gateway 兼容。要尝试使用，请将模型 ID 更改为 @cf/cloudflare/clef-omni，并将模型选择器设置为 clef-omni。
For more information, refer to the Clef-omni model page and pricing.
更多信息，请参阅 Clef-omni 模型页面和定价。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力