跳到主内容
@wquguru
精选88MarkTechPost(RSS)模型发布/更新多源精选 ×2

Reka发布Rho-1:单模型原生理解生成视频与机器人动作

Reka Releases Rho-1: A 19B Omni-Reasoning Model That Understands, Generates Video and Outputs Robot Actions in One

原文
发到 X
推荐理由

单模型原生处理视频与机器人动作的架构创新值得关注,蒸馏后的极速表现也展示了实用潜力,建议关注后续性能验证。

Reka has released a research preview of Rho-1, a 19B omni-reasoning model trained from scratch. A single neural network understands and generates text, images and video, reasons over them, and outputs robot actions. Reka frames it as a direct replacement for agentic pipelines that pass work between modality-specific models.

Reka 发布了 Rho-1 的研究预览版,这是一个从头开始训练的 19B 全模态推理模型。单个神经网络能够理解和生成文本、图像和视频,对其进行推理,并输出机器人动作。Reka 将其定位为直接替代那些在特定模态模型之间传递工作的智能体管道。

What Rho-1 Changes

Rho-1 带来的变化

Most multimodal systems today are pipelines. A central model plans, then hands jobs to specialists for images, video or detection. Each handoff adds latency, and each specialist sees only a narrow request.

当今大多数多模态系统都是管道架构。一个中央模型进行规划,然后将任务交给专门处理图像、视频或检测的专家模型。每次交接都会增加延迟,而每个专家模型只能看到狭窄的请求。

Rho-1 removes those handoffs. Text, vision and robotic actions become tokens inside one context window. According to Reka’s research, one unedited session shows the full loop. The model draws a lighthouse, boxes it, animates it, edits the video into a snowstorm, and explains the difference. That all happens in 5 turns, with no tool call and no second model.

Rho-1 消除了这些交接环节。文本、视觉和机器人动作都成为同一个上下文窗口内的 token。根据 Reka 的研究,在一个未经编辑的会话中展示了完整的循环:该模型绘制了一座灯塔,将其框选出来,为其添加动画,将视频编辑成暴风雪场景,并解释其中的差异。这一切在 5 个回合内完成,无需工具调用,也无需第二个模型。

Architecture: Two Streams, One KV Cache

架构:双流,共享 KV Cache

Every input and output uses one of two native formats:

每个输入和输出都使用两种原生格式之一:

  • Discrete tokens carry text, symbolic reasoning and high-level commands.
  • Continuous tokens carry image latents, video frames, robot actions and proprioception.
  • 离散 token 承载文本、符号推理和高级命令。
  • 连续 token 承载图像潜在变量、视频帧、机器人动作和本体感觉。

Each transformer block holds two expert weight streams. The understanding stream handles language and visual parsing. The generation stream denoises latents into images and video. Both streams share attention and operate over the same KV cache.

每个 Transformer 块包含两个专家权重流。理解流处理语言和视觉解析。生成流对潜在变量进行去噪以生成图像和视频。两个流共享注意力机制,并在相同的 KV cache 上运行。

When a reply needs pixels, the understanding stream emits a discrete handoff token. The generation stream then renders from the full accumulated state. Training combines next-token prediction for discrete sequences with flow matching for continuous outputs.

当回复需要像素时,理解流会发出离散的交接 token。随后,生成流从完整累积的状态中进行渲染。训练过程结合了针对离散序列的下个 token 预测和针对连续输出的流匹配。

This design has practical effects. Bounding boxes come out as coordinate tokens, not from a separate detector. A video’s first frame reuses the in-context image representation instead of a re-encoded copy.

这种设计具有实际影响。边界框作为坐标 token 输出,而非来自单独的检测器。视频的第一帧复用了上下文中的图像表示,而不是重新编码的副本。

Speed: Base vs Distilled

速度:基础模型与蒸馏模型

The base model generates video at 0.79x real-time (median), with a watchable stream starting in roughly 6 seconds. Reka team measured 7.0 seconds to a first clip, against an illustrative 13.8 seconds for a multi-agent pipeline.

基础模型生成视频的速度为实时速度的 0.79 倍(中位数),大约 6 秒后即可开始观看流。Reka 团队测得生成第一个片段耗时 7.0 秒,而相比之下,多智能体管道需要 13.8 秒。

A distilled variant cuts denoising from 99 steps to 8, with minimal quality loss reported. It returned a 5.3-second clip in about a second. In Reka’s internal tests, it matched the fastest dedicated image models. It was also the quickest model tested to the first text token. These are vendor-run tests, not independent benchmarks.

一个蒸馏变体将去噪步骤从 99 步减少到 8 步,据报道质量损失极小。它在约一秒钟内返回了一个 5.3 秒的片段。在 Reka 的内部测试中,它达到了最快专用图像模型的水平。它也是测试中生成第一个文本 token 最快的模型。这些是厂商运行的测试,而非独立基准测试。

World Model and Robotics

世界模型与机器人技术

Rho-1 streams continuously, clip after clip. New instructions enter through the understanding stream and update state mid-rollout. Reka demonstrates one opening forked into ‘bank left’ and ‘bank right’ continuations.

Rho-1 持续流式输出,一段接一段。新指令通过理解流进入,并在 rollout 过程中更新状态。Reka 展示了一个初始分叉为“向左转弯”和“向右转弯”延续的示例。

For robotics, actions and future frames decode from the same latent state. A LIBERO simulation episode shows Rho-1 emitting 7 action channels. To scale past scarce teleoperation logs, Reka pairs Rho-1 with its Inverse Dynamics Model, which infers control signals from raw video.

对于机器人领域,动作和未来帧从相同的潜在状态解码。LIBERO 模拟片段显示 Rho-1 输出了 7 个动作通道。为了突破稀缺遥操作日志的限制,Reka 将 Rho-1 与其逆动力学模型(Inverse Dynamics Model)配对,后者可从原始视频中推断控制信号。

How Rho-1 Compares

Rho-1 的比较情况

Data verified on October 5, 2026 from official sources.

数据于 2026 年 10 月 5 日从官方来源验证。

FeatureReka Rho-1ByteDance BAGELBAAI Emu3.5Google Genie 3
DeveloperRekaByteDance SeedBAAIGoogle DeepMind
Parameters19B14B total, 7B active (MoT)34BNot disclosed
InputsText, image, video, actions, proprioceptionText, imageInterleaved text and imageText prompt, navigation inputs
OutputsText, image, video, actions, proprioceptionText, imageInterleaved text and imageInteractive video world
Native video generationYes, capped at 672×384NoNo (image frames, not native clips)Yes, 720p at 24 fps
Real-time steeringYes, continuous rolloutsNoNoYes, promptable world events
Robot actionsNative continuous action tokensNoEmbodied manipulation demosTakes navigation actions, does not emit them
Open weightsNoYes, Apache 2.0Yes, Apache 2.0No
Access todayResearch preview via [email protected]Hugging Face, GitHubHugging Face, emu.world appProject Genie for Google AI Ultra subscribers
Source/ResourcesReka blogGitHubHugging FaceDeepMind blog
特性Reka Rho-1ByteDance BAGELBAAI Emu3.5Google Genie 3
开发者RekaByteDance SeedBAAIGoogle DeepMind
参数量19B总计 14B,激活 7B (MoT)34B未公开
输入文本、图像、视频、动作、本体感觉文本、图像交错的文本和图像文本提示、导航输入
输出文本、图像、视频、动作、本体感觉文本、图像交错的文本和图像交互式视频世界
原生视频生成是,限制在 672×384否否(图像帧,非原生片段)是,720p 24 fps
实时操控是,连续 rollout否否是,可提示的世界事件
机器人动作原生连续动作 token否具身操作演示执行导航动作,但不输出它们
开源权重否是,Apache 2.0是,Apache 2.0否
当前获取方式通过 [email protected] 联系获取研究预览Hugging Face, GitHubHugging Face, emu.world appProject Genie for Google AI Ultra 订阅者
来源/资源Reka 博客GitHubHugging FaceDeepMind 博客

Genie 3 access per Project Genie; Emu3.5 parameter count per its Hugging Face model card.

Genie 3 通过 Project Genie 获取;Emu3.5 参数量依据其 Hugging Face 模型卡片。

Key Takeaways

关键要点

  • Rho-1 is a 19B model that reads and writes text, image, video and robot actions.
  • Two expert streams share attention and one KV cache in every block.
  • Distillation cuts denoising from 99 to 8 steps, about 1 second per 5.3 s clip.
  • Trained on 320 H100s for 3 months; video is capped at 672×384.
  • Research preview only: no public weights, API or pricing yet.
  • Rho-1 是一个 19B 参数模型,能够读写文本、图像、视频和机器人动作。
  • 两个专家流在每个块中共享注意力机制和一个 KV 缓存。
  • 蒸馏将去噪步骤从 99 步减少到 8 步,每段 5.3 秒的视频约需 1 秒。
  • 使用 320 块 H100 GPU 训练了 3 个月;视频分辨率限制在 672×384。
  • 仅限研究预览:目前无公开权重、API 或定价。

Check out the Technical details and the announcement on X. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

查看技术细节以及在 X 上的公告。所有功劳归于该项目的研究人员。欢迎在 Twitter 上关注我们,别忘了加入我们有 15 万+成员的 ML SubReddit 并订阅我们的新闻通讯。等等!你在 Telegram 上吗?现在你也可以加入我们的 Telegram 群组。

更进一步:量化金融体系

看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力

进入量化体系 →