跳到主内容
@wquguru
精选90GitHub 博客（RSS）技巧与观点

GitHub Copilot 运行时重写为 Rust

Migrating the GitHub Copilot runtime to Rust, using Copilot

原文
发到 X
推荐理由

极具参考价值的工程复盘：展示了如何用 AI Agent 辅助完成百万行级的大型语言迁移与架构重构，详细剖析了从 TS 到 Rust 的性能取舍与踩坑细节，适合负责 AI 基础设施与 Agent 框架开发的工程师深入研读。

The GitHub Copilot CLI, GitHub Copilot app, and GitHub Copilot SDK are all backed by the Copilot agent runtime, an agentic harness that can be embedded into applications and services. It was originally written in TypeScript on Node.js and the V8 JavaScript engine for what is now the GitHub Copilot cloud agent (CCA), and the runtime stayed on that stack as the runtime and its capabilities grew rapidly.

GitHub Copilot CLI、GitHub Copilot 应用和 GitHub Copilot SDK 均由 Copilot 代理运行时（Copilot agent runtime）提供支持，这是一种可嵌入到应用程序和服务中的代理式框架。它最初基于 Node.js 和 V8 JavaScript 引擎使用 TypeScript 编写，用于构建如今称为 GitHub Copilot 云代理（CCA）的系统；随着运行时的快速发展和其能力的不断增强，该运行时一直沿用这一技术栈。

That has now changed. Using the GitHub Copilot app and the Copilot CLI, we completely rewrote the runtime into more than 800,000 lines of production Rust. AI agents wrote most of the code, spanning 128 pull requests that landed in main and shipped incrementally rather than waiting for a single cutover at the end. The few inevitable regressions were discovered and fixed quickly along the way, while the performance of the runtime improved by orders of magnitude. A project that would have taken a whole team of developers a year or two before agents was now completed primarily by a single developer, in only a few months, all while the rest of the team continued to greatly expand the runtime’s capabilities and reach.

情况现已改变。借助 GitHub Copilot 应用和 Copilot CLI，我们完全将运行时重写为超过 80 万行生产级 Rust 代码。AI 代理编写了大部分代码，涉及 128 个合并到 main 分支的拉取请求，这些更改是增量发布的，而非等到最后一次性切换。少数不可避免的回归问题在过程中被迅速发现并修复，同时运行时的性能提升了数个数量级。在代理出现之前，这样一个项目需要整个开发团队花费一到两年的时间才能完成；而现在，主要由一名开发者在短短几个月内就完成了，与此同时，团队其余成员继续大幅扩展运行时的能力和覆盖范围。

Why we needed to port

为何我们需要进行移植

The Copilot agent runtime isn’t just the engine behind the Copilot CLI. It backs a growing set of Microsoft, GitHub, and ecosystem solutions, for each of which AI support is, architecturally, a shell around the same runtime plus whatever customizations that solution needs. This includes not only the GitHub Copilot CLI and the GitHub Copilot app, but also the latest releases of VS Code, Visual Studio, CCA, Copilot Code Review (CCR), Copilot Cowork, Copilot Studio, and Excel and Outlook and PowerPoint and Word and… it goes on.

Copilot 代理运行时不仅仅是 Copilot CLI 背后的引擎。它还支撑着一系列不断增长的 Microsoft、GitHub 及生态系统解决方案，对于每个解决方案而言，从架构上看，AI 支持都是围绕同一运行时以及该解决方案所需的自定义部分构建的外壳。这不仅包括 GitHub Copilot CLI 和 GitHub Copilot 应用，还包括最新版本的 VS Code、Visual Studio、CCA、Copilot Code Review (CCR)、Copilot Cowork、Copilot Studio，以及 Excel、Outlook、PowerPoint、Word 等……此类产品还在不断增加。

These are very different products, and none of them wants to or should need to implement everything that goes into a production agent harness. They want all of the intelligence, security, reliability, and performance, and they want it shared so that a fix in one place fixes it in all of them. Most of the products listed in the previous paragraph initially implemented their own agent loop, but have since replaced it with the GitHub Copilot SDK, which is the entry point to the Copilot agent runtime. Doing so enables them to focus on their core business value and leave the details to the runtime. That’s all the more important given the pace of the industry and the employed agent loop needing to stay always best-of-breed in the face of intense competition.

这些都是截然不同的产品，它们都不希望、也不应该需要自行实现生产级代理框架所需的所有功能。它们希望获得所有的智能、安全性、可靠性和性能，并且希望这些能力能够共享，以便在一处进行的修复能同步应用于所有产品。上一段列出的大多数产品最初都实现了自己的代理循环（agent loop），但随后已将其替换为 GitHub Copilot SDK，后者是接入 Copilot 代理运行时的入口点。这样做使它们能够专注于自身的核心业务价值，而将细节交由运行时处理。鉴于行业发展速度之快，且所使用的代理循环必须在激烈竞争中始终保持行业最佳实践，这一点显得尤为重要。

So, shared runtime, good. The problem was the nature of the thing being shared.

所以，共享运行时是个好主意。问题在于被共享的实体的性质。

If we look at the CLI, it’s logically a terminal UI (TUI) on top of an agent loop. As it happened, the whole stack was implemented in TypeScript, using Node.js as the framework and V8 for the execution engine, with Ink and React for UI. That’s a respectable choice for a TUI application; TypeScript and Node.js are broadly accessible and enable very rapid application development. And for the needs of a console application, the performance implications in terms of startup, responsiveness, throughput, and memory consumption are also reasonable. They are, unfortunately, much less reasonable when you think about that implementation being used in other environments, with other constraints, with demands for things like fast startup and excellent server density due to low memory overhead.

如果我们审视 CLI，它在逻辑上是一个位于智能体循环（agent loop）之上的终端 UI（TUI）。实际情况是，整个技术栈都使用 TypeScript 实现，以 Node.js 作为框架，V8 作为执行引擎，并采用 Ink 和 React 构建 UI。对于 TUI 应用来说，这是一个相当不错的选择；TypeScript 和 Node.js 具有广泛的可用性，并能支持极快的应用开发速度。而对于控制台应用的需求而言，其在启动速度、响应能力、吞吐量和内存消耗方面的性能影响也是合理的。然而，不幸的是，当你考虑到这种实现方式被用于其他环境、面临其他约束，以及对快速启动和低内存开销带来的高服务器密度等需求时，这些性能表现就显得不那么合理了。

The architecture of the CLI and its runtime also contributed to challenges here. The whole industry is running extremely fast, and in that context, really bright people make decisions for delivery speed and market reach. The Copilot CLI was initially written and shipped quickly, and in doing so, the TUI and the runtime were fairly intertwined rather than separated into discrete layers. Then when an SDK was needed for programmatic access to that runtime, without clear separation of the layers, a pragmatic decision was made to layer the SDK on top of the CLI, even though logically you’d expect the inverse architecture. Rather than only being accessible via commands provided by the user at the command line, the CLI was updated with a mode where it could be run headless, reading similar commands from stdin and writing responses to stdout. A JSON-RPC protocol could then be used to marshal function calls from an external process to and from the CLI. The SDK could then be embedded in arbitrary consuming programs, which would spawn a CLI process to host the agent loop out-of-process, with the SDK calling functions in the remote process via this JSON-RPC mechanism. Neat. Fast to get out the door. Flexible. But not great for the performance (startup, memory, throughput) and reliability of those consuming applications. Creating a new CopilotClient from the SDK meant spawning another process:

CLI 及其运行时的架构也在此处带来了挑战。整个行业的迭代速度极快，在这种背景下，许多才华横溢的人为了交付速度和市场份额而做出决策。Copilot CLI 最初是快速编写并发布的，在这个过程中，TUI 和运行时紧密交织在一起，而不是分离为独立的层级。随后，当需要为对该运行时的程序化访问提供 SDK 时，由于层级之间缺乏清晰的分离，人们做出了一个务实的决定：将 SDK 叠加在 CLI 之上，尽管从逻辑上讲，你预期的是相反的架构。CLI 不再仅通过用户在命令行提供的命令进行访问，而是更新了一种模式，使其能够无头（headless）运行，从 stdin 读取类似的命令并将响应写入 stdout。然后可以使用 JSON-RPC 协议将外部进程中的函数调用编组到 CLI 中或从 CLI 中编组出来。SDK 然后可以嵌入到任意消费程序中，这些程序会生成一个 CLI 进程来在进程外托管智能体循环，并通过此 JSON-RPC 机制由 SDK 调用远程进程中的函数。设计精巧。快速推出市场。灵活。但对于这些消费应用的性能（启动、内存、吞吐量）和可靠性来说，这并不是最佳选择。从 SDK 创建新的 CopilotClient 意味着生成另一个进程：

代码 · 5 行
const client = new CopilotClient();
await client.start(); // spawns the CLI as a subprocess
const session = await client.createSession({
    /* ... */
});
代码 · 5 行
const client = new CopilotClient();
await client.start(); // spawns the CLI as a subprocess
const session = await client.createSession({
    /* ... */
});

The process would need to launch and host Node and V8. It meant parsing the significant amount of JavaScript produced from the TypeScript code in the CLI, generating bytecode for it, and potentially optimizing hot code in later JIT tiers. It meant all the memory overhead associated with V8. It meant inheriting Node’s threading model, which by default pushes us towards a model of all CPU-bound work being serialized. And it meant forced out-of-process communication just to make function calls. It meant every SDK consumer, in every language, ships Node.js or a bundled binary containing V8. It meant the C#, Python, Go, Java, and Rust SDKs all paid for a whole second language runtime per client, on the order of 100 MB of working set minimum, for a runtime their application otherwise had no use for. It meant every event, every message, and every abstracted session file system read and write was pushed across a process boundary. It meant a crash in Node took the session with it. And it meant anyone deploying this had, at a minimum, two processes to supervise, monitor, and debug.

该流程需要启动并托管 Node 和 V8。这意味着需要在 CLI 中解析从 TypeScript 代码生成的大量 JavaScript，为其生成字节码，并在后续的 JIT 层级中对热点代码进行优化。这意味着要承担与 V8 相关的所有内存开销。这意味着要继承 Node 的线程模型，该模型默认将所有 CPU 密集型工作串行化。这意味着仅为了进行函数调用就不得不采用进程间通信。这意味着每种语言的每个 SDK 消费者都必须附带 Node.js 或包含 V8 的捆绑二进制文件。这意味着 C#、Python、Go、Java 和 Rust SDK 的每个客户端都要为整个第二语言运行时付费，其工作集至少为 100 MB，而应用程序本身并不需要该运行时。这意味着每个事件、每条消息以及每个抽象的会话文件系统读写操作都必须跨越进程边界传输。这意味着 Node 中的崩溃会拖垮整个会话。这意味着任何部署此方案的人至少需要监督、监控和调试两个进程。

Instead, we wanted a runtime:

相反，我们想要一个运行时：

  • that does not include the TUI, that’s its own library the TUI and other applications and services can be properly layered on top of cleanly.
  • implemented in a language with minimal dependencies and minimal overhead.
  • implemented in a way that it can be cleanly embedded in-process rather than being forced out-of-process.
  • implemented in a language with top characteristics around performance and scalability and reliability.
  • implemented in a language that’s great for interop, such that it can be used cleanly by all six Copilot SDK language versions (C#, TypeScript, Python, Rust, Go, Java) with that stack’s foreign function interface (FFI) mechanism.
  • implemented with a tool chain that provides a more modern security posture, with less supply chain risk and greater support for correct-by-construction code.
  • 不包含 TUI，TUI 是其自身的库，其他应用和服务可以干净地在其上层构建。
  • 用依赖最少且开销最小的语言实现。
  • 以能够干净地嵌入进程内的方式实现，而不是被迫在进程外运行。
  • 用性能、可扩展性和可靠性方面表现优异的语言实现。
  • 用擅长互操作的語言实现，以便通过该堆栈的外挂接口（FFI）机制被所有六种 Copilot SDK 语言版本（C#、TypeScript、Python、Rust、Go、Java）干净地使用。
  • 使用提供更现代安全态势的工具链实现，降低供应链风险，并更好地支持按构造正确的代码。

For all those reasons, as well as softer reasons (such as team experience and industry direction), we chose Rust. This is in no way a claim that every large TypeScript program should become Rust. Our requirements emphasized embedding through a C ABI, low startup and steady-state overhead, and predictable resource use. Rust made those goals possible, at the expense of other complications, e.g. we had to represent lifetimes and shared state explicitly (the lifecycle regressions discussed later highlight the implications of that). The right target language legitimately varies from application to application.

出于所有这些原因，以及更柔和的原因（例如团队经验和行业方向），我们选择了 Rust。这绝非主张每个大型 TypeScript 程序都应转变为 Rust。我们的需求强调通过 C ABI 进行嵌入、低启动开销和稳定状态开销，以及可预测的资源使用。Rust 使得这些目标成为可能，但代价是其他复杂性，例如我们必须显式表示生命周期和共享状态（后文讨论的生命周期回归问题突出了这一点的含义）。合适的目标语言因应用而异，这是合理的。

There were then two key related tasks undertaken:

随后开展了两项关键的相关任务：

  • Separating the TUI-specific code from the runtime, so that the former is layered strictly on top of the latter, and more specifically layered strictly on top of the SDK’s public surface area. Today, the CLI still calls directly into runtime internals in several places; moving it fully onto the SDK’s surface area is ongoing work.
  • 将 TUI 特定代码与运行时分离，以便前者严格分层于后者之上，更具体地说，严格分层于 SDK 的公共接口之上。目前，CLI 仍在多个地方直接调用运行时内部实现；将其完全迁移到 SDK 的公共接口上是一项正在进行的工作。

更进一步：量化金融体系

看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力

进入量化体系 →
话题#GitHub#Copilot#Rust#AI Engineering

相似阅读

关联信息，但可能不是同一事件