BFL联合创始人谈FLUX 3视觉智能与训练取舍
Building Visual Intelligence — Andreas Blattmann & Dustin Podell, Black Forest Labs
这是一篇极具参考价值的工程复盘访谈,详细拆解了FLUX 3在多模态融合、训练算力分配及Kontext编辑技术上的具体取舍与实现路径,适合关注底层模型架构与训练策略的研究者阅读。
This episode is sponsored by Deepgram. Try Deepgram for voice agents that truly understand your context! #contextiseverything
本期节目由 Deepgram 赞助。试试 Deepgram,打造真正理解你上下文的语音代理!#contextiseverything
Not many people know that Stable Diffusion is why I went fulltime into AI (I aptly titled Multiverse, not Metaverse to call out Zuck’s mistaken investment). It turns out that Dustin Podell also had the same aha moment - quitting his day job when the model came out. But we took very different paths… I stuck to what I knew in human content creation, and Dustin immersed deep into AI content creation with Andreas Blattmann, co-inventor of Latent Diffusion and the rest of the world-leading imagegen crew at Stability AI. Stability fell apart, but Anj Midha helped the core group spin out to BFL, which immediately built an incredible new imagegen model in the Flux series. But why are they now going into video and world models?
鲜为人知的是,Stable Diffusion 是我全职投身 AI 的原因(我恰如其分地将标题定为 Multiverse 而非 Metaverse,以指出 Zuck 的投资失误)。事实证明,Dustin Podell 也有同样的顿悟时刻——在模型发布后辞去了本职工作。但我们走了截然不同的道路……我坚持自己在人类内容创作领域的专长,而 Dustin 则与 Andreas Blattmann(Latent Diffusion 的联合发明人)以及 Stability AI 中其余世界领先的图像生成团队成员一起,深入沉浸于 AI 内容创作。Stability 分崩离析,但 Anj Midha 帮助核心团队剥离出来成立了 BFL,后者随即构建了一个令人惊叹的全新图像生成模型 Flux 系列。但他们为何现在要进军视频和世界模型领域呢?
Andreas gives an example you can try with one hand. If somebody tosses you a ball, your hand starts moving before the ball reaches you. You can do that without being able to write down the trajectory. When he talks about building visual intelligence, that’s the kind of knowledge he means. A still can show you where the ball is. A video shows you how it got there. The sound of impact can tell you when it hit something, even if the camera missed the contact. That’s BFL’s argument for training FLUX 3 on the three signals together. BFL’s FLUX 3 release
Andreas 举了一个你可以单手尝试的例子。如果有人向你扔球,你的手会在球到达之前就移动。即使无法写下轨迹,你也能做到这一点。当他谈论构建视觉智能时,指的就是这类知识。静态图片可以告诉你球在哪里,视频则展示了它如何到达那里。撞击声可以告诉你它何时击中某物,即使摄像机错过了接触瞬间。这就是 BFL 主张将 FLUX 3 在三种信号上共同训练的理由。BFL 的 FLUX 3 发布
The horse in motion is one of the FLUX 3 examples we show in the intro. The audio examples are worth playing with the sound on. FLUX 3 Video
运动中的马匹是我们片头展示的 FLUX 3 示例之一。音频示例值得开启声音来体验。FLUX 3 Video
What a moving camera teaches you
移动摄像教会你的东西
Suppose the camera starts in front of a person and pans around them. At some point the model owes you the back of their head. Andreas thinks ordinary two-dimensional video contains a great deal of evidence about what objects look like from other angles. That doesn’t settle whether a model “understands physics,” a point we go back and forth on. It does give you a much better test than asking whether one frame looks convincing.
假设摄像机开始位于一个人面前并围绕他平移。在某个时刻,模型应该为你呈现他的后脑勺。Andreas 认为普通的二维视频包含了大量关于物体在其他角度下外观的证据。这并不能确定模型是否“理解物理”,这也是我们反复争论的一点。但它确实提供了比询问单帧看起来是否令人信服更好的测试方法。
There’s a good engineering example late in the episode. Sharpen each frame of a clip independently and each frame might look terrific. Hit play and the extra detail can shimmer, because the upscaler made a slightly different decision every time. BFL derives its video upscaler from the same FLUX 3 backbone. Temporal consistency matters even when all the user asked for was a higher-resolution file.
本期节目后半部分有一个很好的工程示例。独立 sharpen 每一帧,每一帧可能看起来都很棒。但点击播放时,额外的细节可能会闪烁,因为超分辨率算法每次做出了略微不同的决定。BFL 的视频超分辨率算法源自相同的 FLUX 3 主干网络。即使用户只要求了更高分辨率的文件,时间一致性仍然至关重要。
What’s worth spending training compute on?
值得投入训练算力的地方是什么?
One of the questions I had for them was whether autoregression was taking over image generation. Andreas gives the other side its due. An autoregressive model can learn from all the tokens in a training example at once. On the diffusion side, the team can distill a long sampling process down to a handful of steps. Which advantage matters depends on whether you’re paying to train the model or waiting for an image to finish.
我曾向他们提出的一个问题是自回归是否正在接管图像生成。Andreas 对另一方的观点给予了应有的重视。自回归模型可以一次性从训练样本的所有标记中学习。在扩散模型方面,团队可以将漫长的采样过程蒸馏为寥寥几步。哪种优势更重要,取决于你是付费训练模型,还是等待一张图片生成完毕。
Dustin’s early Kontext experiment is a nice example of how they decide where to invest. He suspected an image model that could draw a dog wearing a hat already knew something about where hats go. He and a colleague collected a small set of photos labeled with relationships. A training run of a few hours got the model to put a logo on a shirt. It was rough, but it was enough to take to the next offsite. The team then spent months on the data and the model. FLUX Kontext examples
Dustin 早期的 Kontext 实验很好地展示了他们如何决定投资方向。他怀疑一个能够画出戴帽子狗的图片模型,已经知道帽子该放在哪里。他和一位同事收集了一组带有关系标注的小规模照片。经过几小时的训练,模型学会了把标志印在衬衫上。效果虽然粗糙,但足以带到下一次线下会议。随后,团队花了数月时间完善数据和模型。FLUX Kontext 示例
My favorite user story comes near the end. People were feeding FLUX old, degraded family photos, restoring them, then showing the pictures to their families. Dustin says he hadn’t predicted that use. After talking about preserving somebody’s identity across edits and across frames, it’s a lovely example of who actually notices if you get it right.
我最喜欢的用户故事出现在接近尾声时。人们将 FLUX 用于修复老旧、褪色的家庭照片,然后将这些照片展示给家人看。Dustin 表示他未曾预测到这种用法。在讨论了如何在编辑和帧之间保持某人身份的一致性之后,这是一个关于谁真正在意你是否做得好的美好例子。
Show notes
节目备注
- BFL explains what a still image, a moving camera and a soundtrack each contribute during training. FLUX 3 launch and research overview
- Watch and listen to the results of generating video and sound together. FLUX 3 Video demos
- See how the lab teaches a model to make a requested change while preserving the rest of an image. FLUX Kontext examples
- Andreas Blattmann co-created Stable Diffusion and co-invented the latent diffusion method behind it. Guest bio · Original research
- Dustin Podell led FLUX Kontext and was the lead author of SDXL. Dustin’s project credits · FLUX Kontext · SDXL
- The first Latent Space essay grew out of the Stable Diffusion release. Multiverse, not Metaverse
- BFL 解释了静态图像、移动摄像机和音轨在训练期间各自的作用。FLUX 3 发布与研究概述
- 观看并聆听同时生成视频和声音的结果。FLUX 3 视频演示
- 了解实验室如何教模型在保留图像其余部分的同时进行所需的修改。FLUX Kontext 示例
- Andreas Blattmann 共同创建了 Stable Diffusion 并共同发明了背后的潜在扩散方法。嘉宾简介 · 原始研究
- Dustin Podell 领导了 FLUX Kontext 项目,并且是 SDXL 的第一作者。Dustin 的项目贡献 · FLUX Kontext · SDXL
- 第一篇 Latent Space 文章源于 Stable Diffusion 的发布。多元宇宙,而非元宇宙
Chapters
章节
0:00 Host introduction
0:00 主持人介绍
3:39 Cold open
3:39 开场冷场
4:23 Black Forest roots
4:23 Black Forest 的渊源
9:23 Dustin’s path
9:23 Dustin 的历程
17:04 Visual intelligence
17:04 视觉智能
27:38 Inside SDXL
27:38 SDXL 内部解析
33:59 Learning motion
33:59 学习运动
39:57 Flow matching
39:57 流匹配
46:57 Starting BFL
46:57 创立 BFL
50:36 Deepgram sponsor break
50:36 Deepgram 赞助商插播
51:43 Early use cases for FLUX
51:43 FLUX 的早期用例
54:58 Editing with Kontext
54:58 使用 Kontext 进行编辑
1:03:15 Training data
1:03:15 训练数据
1:07:32 Testing an edit
1:07:32 测试编辑
1:13:12 Autoregression
1:13:12 自回归
1:20:50 World action models
1:20:50 世界动作模型
1:30:08 Robotics in 2027
1:30:08 2027年的机器人技术
1:32:16 Who uses FLUX?
1:32:16 谁在使用 FLUX?
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力