跳到主内容
@wquguru
精选88MarkTechPost（RSS）产品发布/更新

Google Gemini Flash模型推出Agentic视频理解

Google Launches Agentic Video Understanding for Gemini Flash Models, Cutting Video Tokens by Up to 88%

原文
发到 X
推荐理由

长视频处理是Agent落地的关键瓶颈，这次Google把视频理解从‘死算’变成了‘活查’，Token砍半还更准，做视频Agent的同学赶紧去压测你的链路。

Video has been the most expensive modality to reason over. A Gemini model handed a 90-minute lecture has, until now, ingested the whole thing at a fixed one frame per second, whether the question was ‘summarize this’ or ‘what time does the speaker switch to the pricing slide?’ That single-pass design forces a bad trade: pay for the full timeline in context, or pre-chunk the video and risk dropping the detail that mattered.

视频一直是推理成本最高的模态。迄今为止，Gemini 模型在处理一段 90 分钟的讲座时，无论问题是“总结这段内容”还是“演讲者何时切换到定价幻灯片”，都会以每秒一帧的固定速率摄入全部内容。这种单次遍历的设计迫使你做出糟糕的权衡：要么在上下文中为完整的时间线付费，要么预先将视频分块并冒着丢失关键细节的风险。

This week, Google launched agentic video understanding across its Flash models. Instead of ingesting the timeline, Gemini navigates it deciding what to watch, at what frame rate, and through which modality. Google reports up to 88% fewer tokens, up to 66% lower cost, and up to 7% higher accuracy on standard video benchmarks.

本周，Google 在其 Flash 模型中推出了代理式视频理解功能。Gemini 不再摄入整个时间线，而是对其进行导航，自主决定观看什么、以何种帧率观看以及通过哪种模态观看。Google 报告称，在标准视频基准测试中，令牌使用量最多减少 88%，成本最多降低 66%，准确率最高提升 7%。

Is it deployable? Yes, but only as a hosted API feature. There are no open weights and nothing to self-host. It ships through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform, works with both file uploads and public YouTube URLs, and bills at standard Gemini API token pricing with no additional feature fee.

它是否可部署？可以，但仅作为托管 API 功能提供。没有开源权重，也无法自行托管。它通过 Google AI Studio 中的 Gemini API 和 Gemini Enterprise Agent Platform 发布，支持文件上传和公共 YouTube URL，并按标准的 Gemini API 令牌定价计费，不收取额外的功能费用。

What actually changed

实际发生了哪些变化

Static processing, still the default on every Gemini model, extracts frames at 1 FPS in a single pass, processes audio at 1 Kbps single channel, and inserts timestamps every second. Agentic processing replaces that with a loop. The model pairs its own reasoning with native video tools to search, scan, and inspect target segments across frames, audio, and transcripts, loading only what the prompt requires. Developers could already assemble this by hand; the change is that Gemini runs the loop internally, which is where the development overhead disappears.

静态处理仍是所有 Gemini 模型的默认方式，它以单次遍历的方式按每秒 1 帧提取帧，以单声道 1 Kbps 处理音频，并每隔一秒插入时间戳。代理式处理用循环取代了这种做法。模型将其自身的推理能力与原生视频工具相结合，跨帧、音频和转录文本搜索、扫描和检查目标片段，仅加载提示词所需的内容。开发者此前已能手动组装这一流程；现在的变化在于 Gemini 在内部运行该循环，从而消除了开发开销。

Across Google’s evaluations, Gemini 3.7 Flash with agentic understanding lands on the accuracy-to-cost Pareto frontier for video analysis among the models tested. The efficiency gains concentrate on long-form content, from 10-minute how-to guides to multi-hour recordings.

在 Google 的评估中，具备代理式理解能力的 Gemini 3.7 Flash 在测试过的模型中，处于视频分析精度-成本帕累托前沿。效率提升主要集中在长格式内容上，从 10 分钟的操作指南到多小时的录制内容均受益于此。

What the API returns

API 返回什么

Agentic processing adds two step types to the response steps array: a processing_call when the model requests a segment or transcript, and a matching processing_result when that load completes. They interleave with thought steps and precede model_output, so they can drive a live progress trace in your UI. Their presence is also how you verify agentic mode actually ran.

代理式处理在响应步骤数组中增加了两种步骤类型：当模型请求某个片段或转录文本时，会出现 processing_call；当该加载完成时，会出现对应的 processing_result。它们与思考步骤交错出现，并位于 model_output 之前，因此可用于在你的 UI 中驱动实时进度追踪。它们的出现也是你验证代理模式实际运行的依据。

Token accounting splits accordingly. Navigation reasoning bills as thought tokens (total_thought_tokens); frames, audio, and transcripts loaded on demand bill as tool-use tokens (total_tool_use_tokens).

Token 计费相应拆分。导航推理按思考 token（total_thought_tokens）计费；按需加载的帧、音频和转录内容按工具使用 token（total_tool_use_tokens）计费。

Enabling it is one field on the video part:

启用它只需在视频部分设置一个字段：

代码 · 14 行
interaction = client.interactions.create(
    model="gemini-3.7-flash",
    input=[
        {
            "type": "video",
            "uri": "https://youtu.be/7Z5Vy9JBANs",
            "processing": "agentic"
        },
        {
            "type": "text",
            "text": "What are the 3 most important announcements in this keynote?",
        },
    ],
)
代码 · 14 行
interaction = client.interactions.create(
    model="gemini-3.7-flash",
    input=[
        {
            "type": "video",
            "uri": "https://youtu.be/7Z5Vy9JBANs",
            "processing": "agentic"
        },
        {
            "type": "text",
            "text": "What are the 3 most important announcements in this keynote?",
        },
    ],
)

You can also mix modes per video inside a single request, agentic on the long lecture, static on the short clip.

您可以在单个请求中为不同视频混合使用模式，例如长讲座采用智能体模式，短视频片段采用静态模式。

Key Takeaways

关键要点

  • Agentic mode lets Gemini navigate a video timeline instead of ingesting it at a fixed 1 FPS.
  • Google reports up to 88% fewer tokens, 66% lower cost, and 7% higher accuracy on video benchmarks.
  • Supported on Gemini 3.8, 3.7, 3.6 Flash and 3.5 Flash-Lite; enabled by one processing field.
  • Static remains better for clips under five minutes and for frame-by-frame precision work.
  • Standard API pricing applies, but navigation reasoning is billed as thought tokens.
  • 智能体模式使 Gemini 能够导航视频时间轴，而非以固定 1 FPS 的速度摄入视频。
  • Google 报告称，在视频基准测试中，token 数量最多减少 88%，成本降低 66%，准确率提高 7%。
  • 支持 Gemini 3.8、3.7、3.6 Flash 和 3.5 Flash-Lite；通过一个处理字段即可启用。
  • 对于五分钟以下的短片以及逐帧精确处理任务，静态模式仍然更优。
  • 适用标准 API 定价，但导航推理按思考 token 计费。

Check out the Google blog, Gemini API video understanding docs, Developer guide in AI Studio and Agentic vision announcement. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

请参阅 Google 博客、Gemini API 视频理解文档、AI Studio 中的开发者指南以及智能体视觉公告。也欢迎在 Twitter 上关注我们，别忘了加入我们拥有 150k+ 成员的 ML SubReddit 并订阅我们的新闻通讯。等等！您在 Telegram 上吗？现在您也可以加入我们的 Telegram 群组。

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us

需要与我们合作推广您的 GitHub 仓库或 Hugging Face 页面或产品发布或网络研讨会等？请联系我们要

The post Google Launches Agentic Video Understanding for Gemini Flash Models, Cutting Video Tokens by Up to 88% appeared first on MarkTechPost.

文章《Google 为 Gemini Flash 模型推出智能体视频理解功能，视频 Token 最多削减 88%》首发于 MarkTechPost。

更进一步：量化金融体系

看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力

进入量化体系 →
话题#Google#Gemini#视频理解#API更新

相似阅读

另一事件，读法相近