Google DeepMind发布Genie 4世界模型，原生支持音频
Meet Google DeepMind’s Genie 4: Real-Time World Model With Native Audio
Genie 4引入了Gemini作为内部规划器的新架构，这对构建Agent训练环境极具参考价值，建议关注其技术路线的演进。
Google DeepMind has released Genie 4 as a research preview for trusted testers. The Genie 4 world model generates explorable worlds from text and image prompts in real time. It also generates the soundscape: ambient noise, music and character dialogue. Users move with keyboard controls and steer events with plain-language prompts. The bigger change is architectural. A Gemini model now runs inside the simulation loop as a planner. That pattern is important for anyone building training environments for agents or robots.
Google DeepMind 已发布 Genie 4，作为面向受信任测试人员的研究预览版。Genie 4 世界模型能够根据文本和图像提示实时生成可探索的世界。它还能生成声音景观：环境噪音、音乐和角色对话。用户通过键盘控制移动，并使用自然语言提示引导事件。更大的变化在于架构方面。一个 Gemini 模型现在作为规划器运行在模拟循环中。这种模式对于任何为智能体或机器人构建训练环境的人来说都至关重要。
TL;DR
TL;DR（太长不看）
- Runs on: Google-hosted only (Project Genie web app, Gemini API early access).
- Performance: Photorealistic worlds with native audio and live text steering.
- Best: Scene consistency for up to 2 minutes, double Genie 3, per TechCrunch’s briefing.
- Worst: Sessions last a few minutes, and objects drift after repeated prompt edits.
- 运行平台：仅限 Google 托管（Project Genie Web 应用，Gemini API 早期访问权限）。
- 性能：具有原生音频和实时文本控制的逼真世界。
- 最佳表现：场景一致性可达 2 分钟，是 Genie 3 的两倍，据 TechCrunch 的简报所述。
- 最差表现：会话仅持续几分钟，且在多次编辑提示后物体会发生漂移。
What is Genie 4?
什么是 Genie 4？
Genie 4 is a world model from Google DeepMind that simulates interactive environments frame by frame. It is not a video clip generator. Each frame responds to user input as it arrives.
Genie 4 是 Google DeepMind 推出的世界模型，逐帧模拟交互式环境。它不是视频片段生成器。每一帧都会响应用户输入的到达。
DeepMind states that the model is natively multimodal. Audio, music and dialogue are generated together with the visuals. It is DeepMind’s first real-time world model to generate audio natively.
DeepMind 指出该模型原生支持多模态。音频、音乐和对话与视觉内容一同生成。这是 DeepMind 首个原生生成音频的实时世界模型。
It succeeds Genie 3, announced in August 2025. Genie 3 ran at 720p and 24 frames per second, with visual memory reaching back 1 minute.
它接替了于 2025 年 8 月宣布的 Genie 3。Genie 3 以 720p 分辨率和每秒 24 帧运行，视觉记忆回溯时长达 1 分钟。
How does Genie 4 use Gemini?
Genie 4 如何使用 Gemini？
Genie 4 borrows from 2 parts of the Gemini family. Gemini Omni anchors the look and behavior of each world. Other Gemini models handle the reasoning that keeps a simulation coherent.
Genie 4 借鉴了 Gemini 家族的 2 个部分。Gemini Omni 锚定每个世界的外观和行为。其他 Gemini 模型处理保持模拟连贯性的推理工作。
The runtime loop works like this:
运行时循环的工作方式如下：
- Interpret: When you type a prompt, Gemini adapts it to fit the running scene.
- Plan: Gemini 3.8 Flash breaks a goal into a sequence of steps.
- Verify: It reviews incoming frames to confirm each step succeeded.
- Adapt: If the world shifts or you intervene, it revises the plan.
- 解释：当你输入提示时，Gemini 会将其调整为适应正在运行的场景。
- 规划：Gemini 3.8 Flash 将目标分解为一系列步骤。
- 验证：它审查传入的帧以确认每个步骤是否成功。
- 适应：如果世界发生变化或你进行干预，它会修订计划。
Between user inputs, Gemini also steers the scene so the world keeps progressing. DeepMind team frames this as an early step toward hybrid world models. In that design, a language model handles structure and narrative, while Genie renders pixels and sound.
在用户输入之间，Gemini 还会引导场景，使世界不断进展。DeepMind 团队将此视为迈向混合世界模型的早期一步。在这种设计中，语言模型处理结构和叙事，而 Genie 渲染像素和声音。
What can you do inside a Genie 4 world?
你可以在 Genie 4 世界中做什么？
- Steer with text: You can change the weather, move the camera or add a new person or object mid-session.
- Simulate physics and people: DeepMind cites aerodynamics and collisions. It calls human motion the biggest leap, with characters that react to their surroundings.
- Hear the world: Every world carries ambient audio. Characters can also hold conversations that react to your inputs in real time.
- Test robot tasks: In one research demo, Gemini 4 Argon split a checkers task into steps. Genie 4 received each step as a text caption and simulated Apptronik’s Apollo humanoid playing the game.
- 使用文本引导：你可以改变天气、移动摄像机或在会话中途添加新的人物或物体。
- 模拟物理和人物：DeepMind 引用了空气动力学和碰撞。它将人类运动称为最大的飞跃，角色会对周围环境做出反应。
- 聆听世界：每个世界都包含环境音频。角色还能根据你的输入进行实时互动的对话。
- 测试机器人任务：在一个研究演示中，Gemini 4 Argon 将跳棋任务分解为多个步骤。Genie 4 接收每个步骤作为文本描述，并模拟 Apptronik 的 Apollo 人形机器人进行游戏。
How does Genie 4 compare with other world models?
Genie 4 与其他世界模型相比如何？
DeepMind released no benchmark tables for Genie 4. The comparison below uses only disclosed specs from each project’s own source.
DeepMind 未发布 Genie 4 的基准测试表。以下比较仅使用各项目来源披露的规格。
| Feature | Genie 4 | Genie 3 | Runway GWM Worlds 2 | Skywork Matrix-Game 3.0 |
|---|---|---|---|---|
| Released | Oct 27, 2026 | Aug 5, 2025 | Sep 3, 2026 | Mar 27, 2026 |
| Parameters | Not disclosed | Not disclosed | Not disclosed | 5B (plus 2x14B MoE variant) |
| Resolution / FPS | Not disclosed | 720p / 24 fps | 720p / 24 fps | 720p / up to 40 fps |
| Native audio | Yes, incl. dialogue | Not reported | Yes, 48 kHz, incl. speech | Not reported |
| Mid-session text control | Yes, Gemini-mediated | Yes, promptable world events | Yes, timestamped text actions | Keyboard and mouse actions |
| LLM planner in loop | Gemini 3.8 Flash | No | No (optional external harness) | No |
| Consistency | A few minutes; 2 min per TechCrunch | A few minutes; 1 min visual memory | No preset length; memory imperfect | Minute-long memory |
| Hardware | Google-hosted, not disclosed | Google-hosted, not disclosed | Runway-hosted, not disclosed | 8 GPUs for DiT plus 1 for VAE |
| Access / license | Trusted testers; AI Ultra and API soon | Project Genie (AI Ultra) | Research preview, contact form | MIT, open weights |
| 特性 | Genie 4 | Genie 3 | Runway GWM Worlds 2 | Skywork Matrix-Game 3.0 |
|---|---|---|---|---|
| 发布日期 | 2026年10月27日 | 2025年8月5日 | 2026年9月3日 | 2026年3月27日 |
| 参数量 | 未披露 | 未披露 | 未披露 | 5B（另有 2x14B MoE 变体） |
| 分辨率 / 帧率 | 未披露 | 720p / 24 fps | 720p / 24 fps | 720p / 最高 40 fps |
| 原生音频 | 是，包括对话 | 未报告 | 是，48 kHz，包括语音 | 未报告 |
| 会话中文本控制 | 是，由 Gemini 中介 | 是，可提示的世界事件 | 是，带时间戳的文本动作 | 键盘和鼠标动作 |
| 循环中的 LLM 规划器 | Gemini 3.8 Flash | 无 | 无（可选外部框架） | 无 |
| 一致性 | 几分钟；TechCrunch 报道为 2 分钟 | 几分钟；1 分钟视觉记忆 | 无预设长度；记忆不完美 | 分钟级记忆 |
| 硬件 | Google 托管，未披露 | Google 托管，未披露 | Runway 托管，未披露 | 8 块 GPU 用于 DiT，1 块用于 VAE |
| 访问权限 / 许可 | 受信任测试者；AI Ultra 和 API 即将推出 | Project Genie（AI Ultra） | 研究预览，联系表单 | MIT，开源权重 |
Who can use Genie 4, and when?
谁可以使用 Genie 4，何时可用？
Genie 4 is live for trusted testers today. According to DeepMind, it is expected to be available soon to Google AI Ultra subscribers in Project Genie. Early access partners of the Gemini API and Google AI Studio come next.
Genie 4 今天已向受信任的测试者开放。据 DeepMind 称，它预计很快将在 Project Genie 中对 Google AI Ultra 订阅用户开放。接下来是 Gemini API 和 Google AI Studio 的早期访问合作伙伴。
All Genie 4 video and audio will carry SynthID watermarks. DeepMind states that it ran internal safety evaluations before the preview.
所有 Genie 4 的视频和音频都将携带 SynthID 水印。DeepMind 表示它在预览前进行了内部安全评估。
Testers so far span creative industries, journalism, education, gaming and engineering.
目前的测试者涵盖创意产业、新闻业、教育、游戏和工程领域。
Reported uses include:
据报道的使用场景包括：
- Filmmakers running multiple live ‘takes’ for pre-visualization.
- Photojournalists turning a single archival photo into a grounded simulation.
- Game developers testing styles and mechanics before writing engine code.
- Enterprise partners generating zero-to-one simulation scenarios.
- 电影制作人运行多个实时“镜头”以进行预可视化。
- 摄影记者将单张档案照片转化为基于现实的模拟。
- 游戏开发人员在编写引擎代码之前测试风格和机制。
- 企业合作伙伴生成从零到一的模拟场景。
What are Genie 4’s limitations?
Genie 4 有哪些局限性？
DeepMind lists several open problems on the Genie 4 model page:
DeepMind 在 Genie 4 模型页面上列出了几个待解决的问题：
- Visual details and object permanence drift in longer sessions, after complex prompt edits, or during navigation.
- Interaction is limited to a few minutes, not extended sessions.
- Real-world locations are not simulated with perfect accuracy.
- Legible text usually appears only when it is in the input description.
- Real-time audio accuracy and multi-agent interactions remain unsolved.
- 在较长的会话中、复杂提示词编辑后或导航期间，视觉细节和物体恒常性会出现漂移。
- 交互仅限于几分钟，而非长时间会话。
- 现实世界的位置无法以完美精度进行模拟。
- 清晰可读的文字通常仅在输入描述中出现时才会显示。
- 实时音频准确性和多智能体交互问题仍未解决。
Key Takeaways
关键要点
- Genie 4 generates world, ambient audio and dialogue together, in real time.
- Gemini 3.8 Flash plans, checks frames and replans inside the loop.
- No parameters, resolution, frame rate or benchmarks were disclosed.
- Access starts with trusted testers, then AI Ultra and Gemini API partners.
- Drift and short sessions remain the main practical limits.
- Genie 4 能够实时同步生成世界环境音和对话。
- Gemini 3.8 Flash 会在循环内规划、检查帧并重新规划。
- 未披露任何参数、分辨率、帧率或基准测试数据。
- 访问权限首先向受信任的测试人员开放，随后是 AI Ultra 和 Gemini API 合作伙伴。
- 漂移问题和短时长会话仍是主要的实际限制。
Check out the Genie 4 announcement, the Genie 4 model page and the Project Genie waitlist. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
查看 Genie 4 公告、Genie 4 模型页面以及 Project Genie 候补名单。此外，欢迎在 Twitter 上关注我们，别忘了加入拥有 15 万+成员的 ML SubReddit 并订阅我们的新闻通讯。等等！你在 Telegram 上吗？现在你也可以加入我们的 Telegram 群组。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力