跳到主内容
@wquguru
精选80MarkTechPost(RSS)产品发布/更新

微软开源代码测试生成器,任务完成率92.1%

Microsoft Open Sources code-testing-generator: a Polyglot Unit-Test Agent That Hits 92.1% Task Completion Versus 78.9% for Stock Copilot

原文
发到 X
推荐理由

做代码生成或测试自动化的同学值得关注,这个开源代理把测试生成从一次性输出变成了仓库感知的规划-验证流程,实测比 Copilot 在模糊任务上强不少,可以直接拿去你的仓库里跑跑看。

Microsoft has open sourced code-testing-generator, a polyglot agent that writes unit tests and then proves they work. It ships in the dotnet-test plugin inside the MIT-licensed dotnet/skills repository.

微软已开源 code-testing-generator,这是一个多语言代理,用于编写单元测试并证明其有效。它随 dotnet/skills 仓库(MIT 许可)中的 dotnet-test 插件一起发布。

The agent targets a gap that coding assistants usually leave open. A prompt like ‘generate unit tests’ does not say which framework, file location or assertions to use. code-testing-generator settles those decisions by reading the repository before it writes anything. It then plans, writes, runs and checks the tests it produces. On Microsoft’s internal 152-task benchmark, it completed 140 tasks against 120 for stock GitHub Copilot. Both setups used the same model and prompts.

该代理针对编码助手通常留下的空白。像“生成单元测试”这样的提示并未说明使用哪个框架、文件位置或断言。code-testing-generator 在编写任何内容之前通过阅读仓库来解决这些决策。然后,它计划、编写、运行并检查它生成的测试。在微软内部的 152 任务基准测试中,它完成了 140 个任务,而标准 GitHub Copilot 完成了 120 个。两种设置使用相同的模型和提示。

Is it deployable

它可以部署吗?

Yes. It is an agent definition along with skills, not a hosted service, so it runs inside your existing coding agent and code stays local.

可以。它是一个代理定义以及技能,而不是托管服务,因此它可以在您现有的编码代理中运行,并且代码保持本地化。

  • Company stage: viable from solo maintainers upward. Startups and mid-market teams gain most, because the agent supplies repository research a small team has no time to encode. Enterprises can fork the language guidance to match internal frameworks.
  • Industries: regulated or audit-heavy software estates — financial services, healthcare, insurance, public sector — plus platform teams paying down legacy test debt.
  • Applications: backfilling tests on untested modules, generating tests for a pull-request diff, raising coverage before a release gate, and standardising conventions across polyglot monorepos.
  • 公司阶段:从独立维护者向上均可行。初创公司和中等规模团队受益最多,因为该代理提供了小团队没有时间编码的仓库研究。企业可以分叉语言指南以匹配内部框架。
  • 行业:受监管或审计繁重的软件领域——金融服务、医疗保健、保险、公共部门——以及偿还遗留测试债务的平台团队。
  • 应用:为未测试的模块补充测试,为拉取请求差异生成测试,在发布门禁前提高覆盖率,以及统一多语言单体仓库中的约定。

What the agent actually does

该代理实际做什么

It coordinates work through a Research-Plan-Implement (RPI) pipeline. It searches the repository for code needing tests, detects the language and test framework, reads existing tests for conventions, and finds the real build and test commands. That last step targets a specific failure: a test project that builds locally but never runs in CI because nothing registered it.

它通过研究-计划-实施(RPI)管道协调工作。它搜索仓库中需要测试的代码,检测语言和测试框架,阅读现有测试以了解约定,并找到真实的构建和测试命令。最后一步针对一个特定失败:一个在本地构建但在 CI 中从未运行的测试项目,因为没有任何东西注册它。

The agent then picks one of three strategies. Direct writes and validates tests immediately. Single pass runs one cycle. Iterative repeats it for large scopes or coverage targets. It never modifies production code, and avoids tests that call external URLs, bind ports or depend on timing.

然后,代理选择三种策略之一。直接写入并立即验证测试。单次通过运行一个周期。迭代对于大范围或覆盖率目标重复进行。它从不修改生产代码,并避免调用外部 URL、绑定端口或依赖计时的测试。

The verification gate

验证门禁

Before reporting completion, the agent runs five checks. It reasons about small code changes that should make the tests fail, a lightweight form of mutation testing. It looks for weak or missing assertions. It maps every requested scenario to a test. It builds the full workspace and runs the full suite. It confirms the repository’s own test command discovers the new tests.

在报告完成之前,代理会执行五项检查。它会推理出一些小的代码改动,这些改动应该会使测试失败,这是一种轻量级的变异测试形式。它会寻找薄弱或缺失的断言。它会将每个请求的场景映射到一个测试。它会构建整个工作区并运行完整的测试套件。它会确认仓库自身的测试命令能够发现新的测试。

Benchmark results

基准测试结果

On Microsoft’s internal benchmark of 152 tasks from real repositories, the agent completed 140 (92.1%) versus 120 (78.9%) for stock GitHub Copilot on the same model and prompts (63% fewer failures).

在微软内部包含152个来自真实仓库任务的基准测试中,该代理完成了140个(92.1%),而使用相同模型和提示的stock GitHub Copilot完成了120个(78.9%)(失败次数减少了63%)。

The gain is concentrated. On 89 vague prompts, the agent resolved 79 (88.8%) against 59 (66.3%), cutting failures from 30 to 10. On 63 detailed prompts, both scored 61 (96.8%). On 15 tasks targeting a specific diff, the agent passed all 15 and stock Copilot passed none.

收益集中。在89个模糊提示中,该代理解决了79个(88.8%),而stock Copilot解决了59个(66.3%),失败次数从30次减少到10次。在63个详细提示中,两者都解决了61个(96.8%)。在15个针对特定差异的任务中,该代理全部通过,而stock Copilot一个也没有通过。

Notably, the agent generated 2.3% fewer tests (6,963 vs 7,129) at effectively identical line coverage (72.4% vs 72.2%). Average task time was 359 seconds against 380. Token use per completed task was 3.2% higher.

值得注意的是,该代理生成的测试减少了2.3%(6,963个对比7,129个),而行覆盖率几乎相同(72.4%对比72.2%)。平均任务时间为359秒,而stock Copilot为380秒。每个完成任务的令牌使用量高出3.2%。

On 45 .NET tasks, Claude Opus 4.8 reached 43/45 with the agent versus 35/45 stock; GPT-5.5 reached 41/45 versus 36/45. On the harder external SWE Atlas benchmark, completion was 16/44 versus 12/44.

在45个.NET任务中,Claude Opus 4.8配合该代理达到了43/45,而stock Copilot为35/45;GPT-5.5达到了41/45,而stock Copilot为36/45。在更难的外部SWE Atlas基准测试中,完成率为16/44,而stock Copilot为12/44。

Explainer: how the agent turns one prompt into verified tests

解释:代理如何将一个提示转化为经过验证的测试

Key Takeaways

关键要点

  • Open source, MIT-licensed, polyglot unit-test agent from Microsoft’s .NET team.
  • Research-Plan-Implement pipeline replaces one-shot generation with repository-aware planning.
  • 92.1% vs 78.9% task completion against stock Copilot on the same model.
  • Gains come almost entirely from vague prompts and diff-targeted requests.
  • Fewer tests, same coverage, 5.5% faster — reliability, not volume.
  • 来自微软.NET团队的开源、MIT许可、多语言单元测试代理。
  • 研究-计划-实施流水线取代了一次性生成,采用仓库感知的规划。
  • 在相同模型上,任务完成率达到92.1%,而stock Copilot为78.9%。
  • 收益几乎完全来自模糊提示和针对差异的请求。
  • 更少的测试,相同的覆盖率,速度提升5.5%——可靠性,而非数量。

Check out the Technical details and Repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

查看技术细节和仓库。另外,欢迎在Twitter上关注我们,别忘了加入我们的15万+ML SubReddit并订阅我们的新闻通讯。等等!你在Telegram上吗?现在你也可以在Telegram上加入我们。

更进一步:量化金融体系

看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力

进入量化体系 →

相似阅读

关联信息,但可能不是同一事件