Hugging Face发布Qwen3.8-27B pi专用LoRA
New LoRA Adapter Lifts Qwen3.8 Coding Success From 39% to 65%
本地部署Coding Agent的同学必看,这个LoRA在不牺牲准确率的前提下大幅降低了Token消耗和推理延迟,性价比极高,建议直接压测你的链路。
Hugging Face’s harness-hub team has released Qwen3.8-27B-pi-adapted, a LoRA adapter for Qwen3.8-27B. It was trained with reinforcement learning to work inside the pi coding agent. The Qwen3.8-27B-pi-adapted LoRA adapter targets one harness instead of general chat. For developers running local coding agents, that is the point. You keep the open base model and swap in a small, harness-specific adapter.
Hugging Face 的 harness-hub 团队发布了 Qwen3.8-27B-pi-adapted,这是针对 Qwen3.8-27B 的一个 LoRA 适配器。它通过强化学习训练,旨在 pi 编码智能体内部运行。Qwen3.8-27B-pi-adapted LoRA 适配器针对的是单一工具集(harness),而非通用聊天。对于运行本地编码智能体的开发者来说,这正是其价值所在。你可以保留开源的基础模型,并替换为小型、特定于工具集的适配器。
TL;DR
TL;DR(太长不看)
- Size: LoRA rank 32 (alpha 64) on Qwen3.8-27B, a 27B dense hybrid with 262,144 native context. Merged BF16 checkpoint is about 54 GB.
- Runs on: vLLM with LoRA enabled, or llama.cpp with a GGUF adapter on a Q8_0 base. Inference VRAM is not disclosed. Training used 2x NVIDIA B200.
- Performance: Large pass@1 gains inside pi with thinking off, plus far shorter sessions.
- Best: 64.7% on SWE-bench Pro (hard 51), up from 39.2%.
- Worst: 55.8% on Terminal-Bench 2.1. The base model with thinking already reaches 64.7% on SWE-bench Pro.
- Bottom line: Best: thinking-level accuracy at about 4.5x fewer tokens. Worst: small eval sets, self-reported, pi only.
- 规模:在 Qwen3.8-27B 上采用 rank 32(alpha 64)的 LoRA,这是一个拥有 262,144 原生上下文的 27B 密集混合模型。合并后的 BF16 检查点约为 54 GB。
- 运行环境:启用 LoRA 的 vLLM,或在 Q8_0 基础模型上使用 GGUF 适配器的 llama.cpp。未披露推理所需的 VRAM。训练使用了 2 块 NVIDIA B200。
- 性能:在关闭思维链的情况下,pi 内部的 Large pass@1 指标显著提升,且会话时间大幅缩短。
- 最佳表现:SWE-bench Pro(hard 51)得分 64.7%,较之前的 39.2% 有所提升。
- 最差表现:Terminal-Bench 2.1 得分 55.8%。带有思维链的基础模型在 SWE-bench Pro 上已达到 64.7% 的得分。
- 结论:最佳情况:以约 4.5 倍的 token 节省实现思维链级别的准确率。最差情况:评估数据集较小,数据由自报得出,仅限 pi 环境。
What is Qwen3.8-27B-pi-adapted?
什么是 Qwen3.8-27B-pi-adapted?
Qwen3.8-27B-pi-adapted is a PEFT LoRA adapter that turns Qwen3.8-27B into a model tuned for pi. pi is an MIT-licensed agent toolkit with a coding agent CLI. The adapter is released under Apache-2.0. It applies to the q, k, v, o, gate, up and down projections. It only works on Qwen3.8-27B.
Qwen3.8-27B-pi-adapted 是一个 PEFT LoRA 适配器,可将 Qwen3.8-27B 转变为针对 pi 调优的模型。pi 是一个采用 MIT 许可证的智能体工具包,包含一个编码智能体 CLI。该适配器采用 Apache-2.0 许可证发布。它应用于 q、k、v、o、gate、up 和 down 投影层。它仅适用于 Qwen3.8-27B。
The base model is a dense hybrid. It alternates Gated DeltaNet linear-attention blocks with Gated Attention blocks across 64 layers.
基础模型是一个密集混合模型。它在 64 层中交替使用门控 DeltaNet 线性注意力块和门控注意力块。
How does the harness adapter training work?
工具集适配器训练是如何工作的?
The method comes from Hugging Face’s harness adapters post from November 6. The idea is 1 LoRA adapter per model and harness. Training runs inside the unmodified harness on verifiable tasks.
该方法源自 Hugging Face 11 月 6 日发布的工具集适配器文章。其理念是每个模型和工具集对应 1 个 LoRA 适配器。训练在未修改的工具集内针对可验证任务进行。
For this checkpoint, the model card lists these details:
对于此检查点,模型卡片列出了以下详细信息:
- Algorithm: AsyncGRPO from TRL, 8 rollouts per task at temperature 0.8.
- Tasks: 750 scale800 tasks interleaved with 300 TermGrade tasks.
- Reward: verifier correctness plus a tool-efficiency bonus with weight 0.1.
- Budget: up to 50 agent steps and 1,800 seconds per rollout, thinking off.
- Checkpoint: step 160. The run stopped at step 191 of 1,000 planned.
- 算法:来自 TRL 的 AsyncGRPO,每个任务进行 8 次 rollout,温度设为 0.8。
- 任务:750 个 scale800 任务与 300 个 TermGrade 任务交错进行。
- 奖励:验证器正确性加上权重为 0.1 的工具效率奖励。
- 预算:每次 rollout 最多 50 个智能体步骤和 1,800 秒,关闭思维链。
- 检查点:第 160 步。运行在第 191 步停止,原计划共 1,000 步。
Every tool call passes through safeagents, a rule-based proxy. It blocks calls such as fetching benchmark answers from the Hub. Blocked rollouts score 0, so RL pushes the model away from reward hacking.
每次工具调用都经过 safeagents,这是一个基于规则的代理。它会阻止诸如从 Hub 获取基准测试答案等调用。被阻止的 rollout 得分为 0,因此强化学习会推动模型远离奖励作弊(reward hacking)。
Benchmarks: how much does the LoRA adapter help?
基准测试:LoRA 适配器带来了多少帮助?
All numbers below are self-reported. They use the pi harness, 1 attempt per task.
以下所有数据均为自报结果。它们使用 pi 工具集,每个任务仅尝试 1 次。
| Benchmark (pi, thinking off) | Base Qwen3.8-27B | pi-adapted | Change |
|---|---|---|---|
| SWE-bench Pro (hard 51) | 39.2% | 64.7% | +25.5 ± 14.2 pts (p = 0.0023) |
| Terminal-Bench 2.1 (86 tasks) | 39.5% | 55.8% | +16.3 ± 9.1 pts (p = 0.0013) |
| 基准测试(pi,思考关闭) | 基础 Qwen3.8-27B | pi 适配版 | 变更 |
|---|---|---|---|
| SWE-bench Pro(困难 51) | 39.2% | 64.7% | +25.5 ± 14.2 分 (p = 0.0023) |
| Terminal-Bench 2.1(86 个任务) | 39.5% | 55.8% | +16.3 ± 9.1 分 (p = 0.0013) |
The adapter solved 15 new SWE-bench Pro tasks and lost 2. On Terminal-Bench 2.1, it gained 16 and lost 2.
该适配器解决了 15 个新的 SWE-bench Pro 任务并输掉了 2 个。在 Terminal-Bench 2.1 上,它赢得了 16 个并输掉了 2 个。
Efficiency is the larger story. On tasks both models solved, median turns fell from 59 to 23 on SWE-bench Pro. On Terminal-Bench 2.1, they fell from 18 to 8. The adapter also generates about 4.5x fewer tokens than the base model with thinking.
效率是更大的亮点。在两个模型都解决的任务中,SWE-bench Pro 的中位数回合数从 59 降至 23。在 Terminal-Bench 2.1 上,它们从 18 降至 8。该适配器生成的 token 数量也比带思考的基础模型少约 4.5 倍。
How does it compare with thinking mode and other adapters?
它与思考模式和其他适配器相比如何?
With thinking on, the base model already scores 64.7% on SWE-bench Pro and 54.2% on Terminal-Bench 2.1. So the adapter matches thinking-level accuracy without paying for long reasoning traces. With thinking on, the adapter reaches 66.7% and 56.6%.
开启思考后,基础模型在 SWE-bench Pro 上的得分已达 64.7%,在 Terminal-Bench 2.1 上为 54.2%。因此,该适配器在不支付长推理轨迹成本的情况下达到了思考级别的准确率。开启思考后,适配器的得分分别为 66.7% 和 56.6%。
| Feature | Qwen3.8-27B-pi-adapted | Qwen3.8-27B base, thinking on | Qwen3.8-27B-opencode-adapted | bytkim/Qwen3.8-27B-pi |
|---|---|---|---|---|
| Type | LoRA r=32 | Base model | LoRA r=32 | Full fine-tune, 28B |
| Method | AsyncGRPO RL | None | AsyncGRPO RL | SFT then GRPO |
| Harness | pi | pi | OpenCode | pi |
| License | Apache-2.0 | Apache-2.0 | Apache-2.0 | Apache-2.0 |
| SWE-bench Pro (hard 51) | 64.7% | 64.7% | 51.0% (OpenCode) | Not disclosed |
| Terminal-Bench 2.1 | 55.8% | 54.2% | 53.0% (OpenCode) | Not disclosed |
| Token use | ~4.5x fewer than base + thinking | Baseline | ~6x to 8x fewer than base + thinking | ~41% fewer output tokens (Terminal-Bench) |
| Training hardware | 2x B200 | Not disclosed | 2x B200 | Not disclosed |
| 特性 | Qwen3.8-27B-pi-adapted | Qwen3.8-27B 基础模型,思考开启 | Qwen3.8-27B-opencode-adapted | bytkim/Qwen3.8-27B-pi |
|---|---|---|---|---|
| 类型 | LoRA r=32 | 基础模型 | LoRA r=32 | 全量微调,28B |
| 方法 | AsyncGRPO RL | 无 | AsyncGRPO RL | SFT 然后 GRPO |
| Harness | pi | pi | OpenCode | pi |
| 许可证 | Apache-2.0 | Apache-2.0 | Apache-2.0 | Apache-2.0 |
| SWE-bench Pro(困难 51) | 64.7% | 64.7% | 51.0% (OpenCode) | 未公开 |
| Terminal-Bench 2.1 | 55.8% | 54.2% | 53.0% (OpenCode) | 未公开 |
| Token 使用量 | 比基础模型+思考少约 4.5 倍 | 基线 | 比基础模型+思考少约 6 到 8 倍 | 输出 token 减少约 41%(Terminal-Bench) |
| 训练硬件 | 2x B200 | 未公开 | 2x B200 | 未公开 |
OpenCode adapter numbers use a different harness and are not directly comparable.
OpenCode 适配器的数据使用了不同的 Harness,无法直接比较。
How do you deploy it?
如何部署它?
The model card gives 4 routes:
模型卡片提供了 4 种路线:
- vLLM: serve Qwen/Qwen3.8-27B with --enable-lora --max-lora-rank 32, thinking disabled. Point pi at http://localhost:8000/v1.
- llama.cpp: load the GGUF adapter on a Q8_0 base GGUF.
- Llama.app on macOS: register the adapter with a 32,768-token context.
- Merge: merge with PEFT on CPU, then serve the 54 GB BF16 result without LoRA flags.
- vLLM:使用 --enable-lora --max-lora-rank 32 服务 Qwen/Qwen3.8-27B,禁用思考。将 pi 指向 http://localhost:8000/v1。
- llama.cpp:在 Q8_0 基础 GGUF 上加载 GGUF 适配器。
- macOS 上的 Llama.app:注册具有 32,768-token 上下文的适配器。
- 合并:在 CPU 上使用 PEFT 进行合并,然后在没有 LoRA 标志的情况下服务 54 GB BF16 结果。
Because adapters share 1 base model, a single server can host pi and OpenCode adapters side by side.
由于适配器共享同一个基础模型,单个服务器可以并排托管 pi 和 OpenCode 适配器。
Key Takeaways
关键要点
- Qwen3.8-27B-pi-adapted is an Apache-2.0, rank-32 RL LoRA for the pi agent.
- SWE-bench Pro hard-51 rises from 39.2% to 64.7%, thinking off.
- Median turns drop from 59 to 23 on SWE-bench Pro.
- It matches base-with-thinking accuracy at about 4.5x fewer tokens.
- Results are self-reported on small sets; the tweet’s 29.4% baseline was corrected.
- Qwen3.8-27B-pi-adapted 是一个用于 pi 代理的 Apache-2.0、rank-32 RL LoRA。
- SWE-bench Pro hard-51 从 39.2% 提升至 64.7%,思考关闭。
- SWE-bench Pro 的中位数回合数从 59 降至 23。
- 它以大约少 4.5 倍的 token 消耗匹配了带思考的基础模型准确率。
- 结果是在小规模数据集上自我报告的;推文中的 29.4% 基线已被修正。
Check out the model card, GGUF adapter, methodology post and TRL training script. You can check the deployment options and specs here through our GPULeaderboard.com chrome extension.
查看模型卡片、GGUF 适配器、方法论文章和 TRL 训练脚本。你可以通过我们的 GPULeaderboard.com Chrome 扩展程序在此处查看部署选项和规格。
Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
此外,欢迎在 Twitter 上关注我们,别忘了加入我们拥有 15 万+成员的 ML SubReddit 并订阅我们的新闻通讯。等等!你在 Telegram 上吗?现在你也可以在 Telegram 上加入我们。
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力