OpenAI发布Pando Git后端，推送延迟降低80%
OpenAI Introduces Pando: A Git Backend That Cuts Median Push Latency ~80%
Agent时代单体仓库并发压力剧增，Pando的工程实践提供了极具参考价值的架构取舍，建议关注大规模代码库管理的同学细读。
OpenAI has introduced Pando, a new Git backend that now serves all development in its monorepo. The OpenAI Pando Git backend keeps the standard Git client and protocol. Engineers and Codex agents still run git push and git fetch as before. Only the server changed. OpenAI team reports median push latency fell nearly 80%. Pando now delivers nearly a petabyte of Git data per day.
OpenAI 推出了 Pando，这是一个新的 Git 后端，现在为其单体仓库中的所有开发工作提供服务。OpenAI Pando Git 后端保留了标准的 Git 客户端和协议。工程师和 Codex 代理仍然像以前一样运行 git push 和 git fetch。只有服务器端发生了变化。OpenAI 团队报告称，中位数推送延迟下降了近 80%。Pando 现在每天处理近 PB 级的 Git 数据。
TL;DR
TL;DR（太长不看）
- Scale: Nearly 1 PB of Git data per day; peak above 57 TB in 1 hour.
- Runs on: Commodity hardware; standard cloud VMs saturate 10 Gbps links on clone and fetch.
- Performance: Thousands of concurrent pushers without degradation, per OpenAI’s own testing.
- Best: Median push latency down nearly 80% (about 5x).
- Worst: Not open source; repeat branch pushes resend non-trunk history (p99 pack under 3 MiB).
- 规模：每天近 1 PB 的 Git 数据；峰值在 1 小时内超过 57 TB。
- 运行环境：商用硬件；标准云 VM 在克隆和获取时会占满 10 Gbps 链路。
- 性能：根据 OpenAI 自身的测试，支持数千个并发推送者且无性能下降。
- 最佳情况：中位数推送延迟下降近 80%（约 5 倍）。
- 最差情况：未开源；重复分支推送会重新发送非主干历史（p99 打包大小低于 3 MiB）。
What is OpenAI Pando?
什么是 OpenAI Pando？
Pando is a Git-compatible server backend from OpenAI that scales concurrent pushes and fetches for a monorepo. It was written in Rust and reached production in 10 weeks. One engineer designed the core and teams of Codex agents implemented it. It has served production traffic since April 2026.
Pando 是 OpenAI 推出的一个兼容 Git 的服务器后端，旨在扩展单体仓库的并发推送和获取能力。它由 Rust 编写，并在 10 周内投入生产环境。一名工程师设计了核心架构，Codex 代理团队负责实现。自 2026 年 4 月起，它已服务于生产流量。
Why did OpenAI need a new Git backend?
为什么 OpenAI 需要一个新的 Git 后端？
Codex multiplied parallel work. OpenAI’s commit chart shows commits landed on master up 1,378% in Q3 2026 versus Q1 2024. Its Q4 2026 estimate is +2,442%. Each push now triggers many CI fetches. By late 2025, fetches and pushes slowed, merges failed, and reliability at times fell below 90%. OpenAI concluded the bottleneck was how it served Git, not Git itself.
Codex 增加了并行工作量。OpenAI 的提交图表显示，与 2024 年第一季度相比，2026 年第三季度合并到 master 的提交数量增长了 1,378%。其对 2026 年第四季度的预估增长为 +2,442%。每次推送现在都会触发许多 CI 获取操作。到了 2025 年底，获取和推送速度变慢，合并失败，可靠性有时甚至低于 90%。OpenAI 得出结论，瓶颈在于其服务 Git 的方式，而非 Git 本身。
How does Pando work?
Pando 是如何工作的？
Pando drops 2 assumptions that most Git servers treat as requirements:
Pando 摒弃了大多数 Git 服务器视为必需条件的两项假设：
1. Repository-wide consistency: Conventional servers order every ref update repo-wide, so unrelated pushes wait. Pando uses causal consistency instead. Each branch is serialized on its own, and clients only conflict when pushing the same ref. The tradeoff: a reader viewing several branches may see them at different points in time.
1. 仓库级一致性：传统服务器会对整个仓库的所有 ref 更新进行排序，导致不相关的推送需要等待。Pando 改用因果一致性。每个分支独立序列化，客户端仅在推送相同的 ref 时才会发生冲突。权衡之处在于：查看多个分支的读者可能会看到它们处于不同的时间点。
2. A unique packfile per client: Standard servers build a fresh pack per fetch. Pando stores each pushed pack as an immutable blob. A “ref version” goes into a scalable metadata store. A push needs O(1) metadata reads and writes. A fetch needs O(log Δ) reads, where Δ is the ref-version gap to close.
2. 每个客户端唯一的 packfile：标准服务器为每次获取构建一个新的 pack。Pando 将每次推送的 pack 存储为不可变 blob。一个“ref 版本”被存入可扩展的元数据存储中。一次推送需要 O(1) 的元数据读写操作。一次获取需要 O(log Δ) 的读取操作，其中 Δ 是需要填补的 ref 版本差距。
How are pushes kept small and reusable?
如何保持推送体积小且可复用？
With millions of branches, a full ref advertisement can exceed 100 MiB. Pando advertises only the trunk branch as the base for new feature-branch pushes. Any reader with trunk history can then fetch that branch from 1 stored pack. In OpenAI’s monorepo, the median pack was under 20 KiB.
拥有数百万个分支时，完整的 ref 广告可能超过 100 MiB。Pando 仅将主干分支作为新特性分支推送的基础进行广告。任何拥有主干历史的读者都可以从 1 个存储的 pack 中获取该分支。在 OpenAI 的单仓库（monorepo）中，中位数 pack 大小低于 20 KiB。
How are fetches served at line rate?
如何以线速提供服务？
Pando returns download links via packfile-uri, an existing Git protocol feature. It sends an empty inline pack. A delivery service called Express caches packs regionally, using consistent hashing, with hit rates above 99%.
Pando 通过 packfile-uri（现有的 Git 协议功能）返回下载链接。它发送一个空的内联 pack。名为 Express 的内容分发服务使用一致性哈希在区域范围内缓存 packs，命中率超过 99%。
What replaces garbage collection?
什么取代了垃圾回收？
Each push creates an “L0” pack. Background workers group packs into geometrically larger covers, similar to LSM-tree compaction. The monorepo currently uses covers up to L4. Covers act as a skip list, so pack selection runs in logarithmic time.
每次推送都会创建一个“L0”pack。后台工作进程将 packs 组合成几何级增长的覆盖层（covers），类似于 LSM-tree 压缩。单仓库目前使用最多到 L4 的覆盖层。覆盖层充当跳表（skip list），因此 pack 选择以对数时间运行。
What are the production numbers?
生产环境中的数据是多少？
All figures below are self-reported by OpenAI.
以下所有数据均由 OpenAI 自行报告。
- Median push latency reduced by nearly 80%.
- Nearly 1 PB of Git data per day, using only a handful of CPU cores.
- Over 57 TB of packs served in a single hour, more than 99% from local disk cache.
- Clones and fetches saturate 10 Gbps links on standard cloud VMs.
- 中位数推送延迟降低了近 80%。
- 每天处理近 1 PB 的 Git 数据，仅使用少量 CPU 核心。
- 单个小时内服务超过 57 TB 的 packs，其中超过 99% 来自本地磁盘缓存。
- 克隆和拉取操作使标准云虚拟机上的 10 Gbps 链路饱和。
OpenAI also ran a 1,000-writer simulation against 3-phase-commit and write-ahead-log designs. It labels these teaching models, not benchmarks.
OpenAI 还针对三阶段提交（three-phase-commit）和预写日志（write-ahead-log）设计进行了 1,000 名作者的模拟。他们将此类模型称为教学模型，而非基准测试。
How does Pando compare with other approaches?
Pando 与其他方法相比如何？
| OpenAI Pando | GitHub Spokes | Anthropic Scion | |
|---|---|---|---|
| Git client compatible | Yes | Yes | No, own scn client |
| Write coordination | Per-branch, causal consistency | Three-phase commit with quorum | Server-side commits; refs via compare-and-swap |
| Default replicas | Not disclosed (blob + metadata stores) | 5 full copies | No fully materialized repo copy |
| Read path | Reused packs via packfile-uri, Express cache | Closest in-sync replica | Tiered tree/blob caches |
| Cache hit rate | Over 99% | Not disclosed | Over 99% per tier |
| Headline result | Median push latency down ~80% | Serves ~1 billion repositories | 200x smaller CPU/memory footprint |
| Build effort | 10 weeks, Rust | Not disclosed | 7 people, 5 months |
| Available to others | No | Yes, powers GitHub | No |
| OpenAI Pando | GitHub Spokes | Anthropic Scion | |
|---|---|---|---|
| Git 客户端兼容 | 是 | 是 | 否，拥有自己的 scn 客户端 |
| 写入协调 | 每分支，因果一致性 | 带法定人数的三阶段提交 | 服务端提交；通过比较并交换（compare-and-swap）管理 refs |
| 默认副本 | 未披露（blob + 元数据存储） | 5 份完整副本 | 无完全实例化的仓库副本 |
| 读取路径 | 通过 packfile-uri 重用 packs，Express 缓存 | 最近的同步副本 | 分层树/blob 缓存 |
| 缓存命中率 | 超过 99% | 未披露 | 每层超过 99% |
| 主要结果 | 中位数推送延迟降低约 80% | 服务约 10 亿个仓库 | CPU/内存占用减少 200 倍 |
| 构建工作量 | 10 周，Rust | 未披露 | 7 人，5 个月 |
| 是否可供他人使用 | 否 | 是，为 GitHub 提供支持 | 否 |
How was Pando built and deployed?
Pando 是如何构建和部署的？
OpenAI wrote a Git conformance suite for agents and added deterministic simulation testing. Agents using GPT-6 Astra wrote Lean and TLA+ specs to find bugs.
OpenAI 为智能体编写了一套 Git 合规性测试套件，并添加了确定性模拟测试。使用 GPT-6 Astra 的智能体编写了 Lean 和 TLA+ 规范以发现错误。
Pando first ran as a read- and write-through proxy, so traffic moved without downtime. OpenAI ships its own Git build to developer machines and the ChatGPT desktop app.
Pando 最初作为读透和写透代理运行，因此流量在不中断服务的情况下迁移。OpenAI 将其自建的 Git 版本分发给开发人员机器和 ChatGPT 桌面应用。
What goes back to upstream Git?
哪些内容会回传到上游 Git？
OpenAI fixed client handling of concurrent, interrupted and resumed pack downloads, and added packfile-uri tracing. Its build adds parallel pack indexing and “explicit haves.” Several fixes are upstream. The rest sit in a public patch tracker.
OpenAI 修复了客户端对并发、中断和恢复的包下载的处理，并添加了 packfile-uri 跟踪。其构建过程增加了并行包索引和“显式 haves”。若干修复已合并至上游。其余内容位于公共补丁追踪器中。
What are the limitations?
存在哪些限制？
- Pando is not open source; OpenAI team states that it is exploring ways to share the technology.
- Cross-branch reads have no single point-in-time snapshot.
- Repeat or stacked branch pushes resend everything outside trunk. The p99 pack was under 3 MiB.
- Every performance number is self-reported, with no independent benchmark yet.
- Pando 并非开源；OpenAI 团队表示正在探索分享该技术的途径。
- 跨分支读取没有单一的时间点快照。
- 重复或堆叠的分支推送会重新发送主干之外的所有内容。p99 包大小低于 3 MiB。
- 所有性能数据均为自我报告，尚无独立的基准测试。
Key Takeaways
关键要点
- Pando keeps Git, but rebuilds the server for parallel agent traffic.
- Per-branch causal consistency removes repository-wide write contention.
- Stored, reusable packs replace per-client pack generation.
- Median push latency dropped nearly 80%; daily traffic nears 1 PB.
- Client fixes go upstream; the backend stays internal for now.
- Pando 保留了 Git，但为并行代理流量重建了服务器。
- 基于每个分支的因果一致性消除了整个仓库的写入竞争。
- 存储的可重用包取代了按客户端生成包的方式。
- 中位推送延迟降低了近 80%；每日流量接近 1 PB。
- 客户端修复已提交至上游；后端目前仍保持内部使用。
Check out the full OpenAI engineering post, the OpenAI Git repository and the patch tracker. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
查看完整的 OpenAI 工程博客文章、OpenAI Git 仓库以及补丁追踪器。此外，欢迎在 Twitter 上关注我们，别忘了加入我们拥有 15 万+成员的 ML SubReddit，并订阅我们的新闻通讯。等等！你在 Telegram 上吗？现在你也可以加入我们的 Telegram 群组。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力