Cerebras详解万亿参数模型在多芯片上的部署架构
Cerebras Deploys Trillion-Parameter AI Models
硬核工程复盘，给出了万亿参数模型在特定硬件上的完整部署流程、内存分配逻辑与并发计算公式，对做高性能推理系统的同学极具参考价值。
Oct 22 2026
2026年10月22日
How we deploy Trillion-parameter frontier models on Cerebras
我们如何在 Cerebras 上部署万亿参数前沿模型
Sarah Chieng
One of the most frequently asked questions we get is how we deploy large models on Cerebras wafer-scale chips. A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, and trillion-parameter frontier models need terabytes for their weights alone.
我们收到的最常见的问题之一，是如何在 Cerebras 晶圆级芯片上部署大模型。单个 Cerebras WSE-3T 晶圆拥有 44 GB 片上内存，而万亿参数的前沿模型仅权重就需要太字节（TB）级的存储空间。
In this post, we’ll explain how we split a large frontier model across multiple WSE-3T wafers, to run multi-trillion parameter models at 1000-2000+ tokens per second.
在本文中，我们将解释如何将大型前沿模型拆分到多个 WSE-3T 晶圆上，从而实现每秒 1000-2000+ 个 token 的多万亿参数模型推理。
We’ll use the open source model Kimi K3, with ~3 trillion parameters, as an illustrative example.
我们将以开源模型 Kimi K3（约 3 万亿参数）作为示例进行说明。
By the end, you'll understand:
阅读完本文后，你将了解：
- How we fit the full model into on-chip memory
- How we map the model across multiple chips
- How we handle a high # of concurrent requests
- 我们如何将完整模型放入片上内存
- 我们如何将模型映射到多个芯片
- 我们如何处理高并发请求
Breaking it down with Kimi K3
以 Kimi K3 为例进行拆解
Kimi K3 is an open source, 3T parameter model with 69 linear-attention layers and 24 dense layers.
Kimi K3 是一个开源的 3T 参数模型，包含 69 个线性注意力层和 24 个密集层。
It scores an impressive 88.3% on Terminal-Bench 2.1 (vs. GPT-5.6 Sol's 88.8%) and 91.2% on BrowseComp (vs. GPT-5.6 Sol's 90.4%). These results demonstrate frontier-level coding and agentic capabilities, even outperforming leading proprietary models on some benchmarks. Read more about Kimi K3 here.
它在 Terminal-Bench 2.1 上取得了令人印象深刻的 88.3% 得分（相比之下 GPT-5.6 Sol 为 88.8%），在 BrowseComp 上得分为 91.2%（相比之下 GPT-5.6 Sol 为 90.4%）。这些结果展示了其前沿级别的编码和智能体能力，甚至在某些基准测试中超越了领先的专有模型。在此处阅读更多关于 Kimi K3 的信息。
For this illustrative example, we use:
对于这个示例，我们使用：
- A 256k assumed average request size
- 4-bit weight quantization (like the Hugging Face implementation)
- An 8-bit KV cache
- 假设平均请求大小为 256k
- 4-bit 权重量化（类似于 Hugging Face 的实现）
- 8-bit KV 缓存
#1: Size of Each layer
#1：每个层的大小
To calculate the total number of Cerebras WSE-3T wafers we need to support Kimi K3, let’s first calculate the size of each model layer.
为了计算支持 Kimi K3 所需的 Cerebras WSE-3T 晶圆总数，让我们先计算每个模型层的大小。
Kimi K3 has 3 trillion parameters. If all 3T parameters are stored at 4 bits each, the raw parameter payload is:
Kimi K3 有 3 万亿参数。如果所有 3T 参数都以每个 4 bit 存储，原始参数负载为：
Dividing that payload by 93 layers then gives:
将该负载除以 93 层，则得到：
#2: Number of Wafers
#2：晶圆数量
Now that we know each layer needs about 16 GB, we can figure out how many layers fit on each 44 GB wafer. The answer depends on the layer type:
现在我们知道每层大约需要 16 GB，我们可以计算出每个 44 GB 晶圆能容纳多少层。答案取决于层的类型：
- Dense layers need additional room for a KV cache, because each dense (full-attention) layer stores the keys and values for every token in every active request. So, we place one dense layer per wafer and leave the remaining SRAM for its KV cache.
- Linear-attention layers do not require additional room for a KV cache. Instead of storing keys and values for every token, a linear-attention layer keeps a small, fixed-size state per request, regardless of context length. That leaves room for two linear layers per wafer (2 × 16 GB = 32 GB).
- 密集层需要额外的空间用于 KV 缓存，因为每个密集（全注意力）层都会为每个活跃请求中的每个 token 存储键（keys）和值（values）。因此，我们在每个晶圆上放置一个密集层，并将剩余的 SRAM 留给其 KV 缓存。
- 线性注意力层不需要为 KV 缓存预留额外空间。与为每个 token 存储键和值不同，线性注意力层针对每个请求维护一个小型的、固定大小的状态，无论上下文长度如何。这就为每个晶圆（wafer）留出了容纳两个线性层的空间（2 × 16 GB = 32 GB）。
Therefore, with Kimi K3’s 69 linear layers and 24 dense layers, we get:
因此，对于 Kimi K3 的 69 个线性层和 24 个密集层，我们得到：
This gives us a total wafer count of 35 + 24 = 59 wafers.
这使我们总共需要 35 + 24 = 59 个晶圆。
#3: Connecting the wafers together
#3：将晶圆连接在一起
Cerebras executes models in a pipeline. After the Cerebras compiler maps the model layers across regions of wafers, the runtime maps the path for tokens to flow between wafers. For ultrafast latency, only activations need to be passed between the wafers, and this can be done via RDMA over Ethernet. This is roughly 2 microseconds per hop.
Cerebras 以流水线方式执行模型。在 Cerebras 编译器将模型层映射到晶圆的各个区域后，运行时会将 token 在晶圆之间流动的路径进行映射。为了实现超低延迟，只需在晶圆之间传递激活值（activations），这可以通过以太网 RDMA 完成。每次跳接（hop）大约耗时 2 微秒。
It’s important to emphasize that because each Cerebras wafer has 44 GB of on-chip SRAM and each layer is only 16 GB, we are typically able to fit entire layers of the model directly on the chip.
需要强调的是，由于每个 Cerebras 晶圆拥有 44 GB 片上 SRAM，而每个层仅占用 16 GB，我们通常能够将模型的整个层直接放置在芯片上。
This means that in most cases, the model weights never have to move, and layers don’t need to be split across wafers. Only activations cross from one wafer to the next.
这意味着在大多数情况下，模型权重无需移动，且层也不需要跨晶圆拆分。只有激活值会从一个晶圆传递到下一个晶圆。
#4: How we serve 1 request
#4：我们如何服务 1 个请求
Now that we have our wafers connected together and the model laid out, we can think of each wafer as one stage in an assembly line: the model’s weights for each stage stay in place, and the request’s intermediate results move through the layer stages.
现在我们已经将晶圆连接起来并布局好了模型，我们可以将每个晶圆视为流水线上的一个阶段：每个阶段的模型权重保持原位，而请求的中间结果则流经各个层阶段。
For example, Wafer 1 runs model layers 1–2 and sends its activations to wafer 2. Wafer 2 runs layers 3–4 and passes its results to Wafer 3, and so on until the last wafer produces the next token.
例如，晶圆 1 运行模型的第 1–2 层，并将其激活值发送给晶圆 2。晶圆 2 运行第 3–4 层，并将结果传递给晶圆 3，依此类推，直到最后一个晶圆生成下一个 token。
#5: How we serve many requests
#5：我们如何服务多个请求
This raises a common misconception. If a token has to travel through each wafer in order, doesn’t each wafer sit idle while it waits for its turn? In effect, wouldn't the system be running at a batch size of one?
这引出了一个常见的误解。如果一个 token 必须按顺序经过每个晶圆，那么当它等待自己的轮次时，每个晶圆是否都处于空闲状态？从本质上讲，系统是否只能以批次大小（batch size）为 1 运行？
No! If we only sent one request down the line at a time and waited for it to complete before starting the next request, this would be an inefficient use of the pipeline. Instead, we keep the entire pipeline full with different requests. Because the model is split into pipeline stages already, each stage can work on a different request at the same time.
不！如果我们一次只发送一个请求进入流水线，并等待其完成后再开始下一个请求，这将是对流水线的低效利用。相反，我们通过不同的请求使整个流水线保持满载。由于模型已经被划分为流水线阶段，每个阶段可以同时处理不同的请求。
While stage 3 processes a token for request A, stage 2 processes one for request B, and stage 1 one for request C. In our example, each stage is a layer. In production, Cerebras uses much finer-grained stages, and many more of them.
当阶段 3 处理请求 A 的一个 token 时，阶段 2 处理请求 B 的一个 token，阶段 1 处理请求 C 的一个 token。在我们的示例中，每个阶段是一个层。在实际生产中，Cerebras 使用更细粒度的阶段，并且数量要多得多。
Loading player...
加载播放器中...
Each stage can also process a micro-batch, a small group of requests, rather than a single request. So the number of requests in flight is roughly the number of stages times the micro-batch size.
每个阶段还可以处理一个微批次（micro-batch），即一小组请求，而不是单个请求。因此，飞行中的请求数量大致等于阶段数乘以微批次大小。
Loading player...
正在加载播放器...
To understand the tradeoff, smaller micro-batches make each request even faster, while larger micro-batches increase total throughput.
要理解这种权衡：较小的微批次会使每个请求更快，而较大的微批次则会提高总吞吐量。
#6: How many requests can we serve?
#6：我们可以服务多少请求？
The amount of on-chip memory allocated to KV cache determines the maximum number of concurrent requests. Every request in flight needs its own KV cache on each dense-layer wafer, so the size of on-chip memory sets the ceiling on concurrent requests. As explained above, we only need to calculate KV cache requirements for dense layers.
分配给 KV 缓存的片上内存量决定了最大并发请求数。每个飞行中的请求在每个密集层晶圆上都需要自己的 KV 缓存，因此片上内存的大小设定了并发请求的上限。如上所述，我们只需计算密集层的 KV 缓存需求。
For each 44 GB wafer that already has one dense layer, we can calculate the remaining space for the KV cache with:
对于已经包含一个密集层的每个 44 GB 晶圆，我们可以用以下公式计算用于 KV 缓存的剩余空间：
Total wafer storage - size of one dense layer. In this case:
晶圆总存储量 - 一个密集层的大小。在这种情况下：
Calculating the # of Concurrent Requests
计算并发请求数
To find the number of concurrent requests supported for any particular model mapping, we first calculate how much cache a single 256k request takes up:
为了找出任何特定模型映射所支持的并发请求数，我们首先计算单个 256k 请求占用的缓存大小：
To then calculate the total number of concurrent requests this configuration would support, we divide:
然后，为了计算该配置支持的总并发请求数，我们进行除法运算：
The total number of concurrent requests represents the number of simultaneous requests moving through the pipeline at any point in time. Unlike a GPU batch, those requests don’t wait for one another to finish, they stream through the pipeline continuously.
总并发请求数代表在任何时间点同时通过管道的请求数量。与 GPU 批次不同，这些请求不需要互相等待完成，而是连续地流经管道。
Of course, neither concurrency nor batch size tells you exactly how efficiently an inference system performs. When evaluating the throughput of an inference system, the more meaningful metric is total token throughput per unit of power (e.g. tokens per second per megawatt), when measured at the same output generation speed.
当然，无论是并发数还是批次大小，都无法精确告诉你推理系统的性能效率。在评估推理系统的吞吐量时，更有意义的指标是在相同输出生成速度下测量的每单位功率的总令牌吞吐量（例如每秒兆瓦的令牌数）。
How this scales beyond 3T parameters
如何扩展到超过 3T 参数的规模
The same approach extends to larger models: calculate the memory needed for weights and request state, distribute the model across wafers, and connect them into a pipeline. Each additional wafer contributes compute and on-chip memory, allowing us to accommodate models that exceed the capacity of any single chip.
同样的方法可扩展到更大的模型：计算权重和请求状态所需的内存，将模型分布到各个晶圆上，并将它们连接成流水线。每个额外的晶圆都提供计算能力和片上内存，使我们能够容纳超出任何单个芯片容量的模型。
Thanks for learning with us! Interested in deploying a model on Cerebras or building with our inference cloud?
感谢与我们一起学习！有兴趣在 Cerebras 上部署模型或使用我们的推理云进行构建吗？
- Get started with our free API: cloud.cerebras.ai
- Reach out to our sales team: [email protected]
- 通过我们的免费 API 开始使用：cloud.cerebras.ai
- 联系我们的销售团队：[email protected]
Designs by Halley Chang. Thanks to Atharva Talpade, Zhenwei Gao, Angela Yeung, Jessica Liu, Joyce Er, Emma Call, and Rohan Gupta for their review and feedback.
设计由 Halley Chang 完成。感谢 Atharva Talpade、Zhenwei Gao、Angela Yeung、Jessica Liu、Joyce Er、Emma Call 和 Rohan Gupta 的审阅和反馈。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力