Fastino发布GLiNER2.5-Decide：3.4亿参数CPU可运行
Fastino Releases GLiNER2.5-Decide: A 340M Open-Weight Decision Model That Runs on CPU
Agent开发者的实用利器，这款轻量级模型用联合解码解决了多标签逻辑冲突痛点，且原生支持CPU部署，非常适合做低延迟的路由与护栏组件，建议直接集成到现有Agent架构中压测。
Fastino Labs has released GLiNER2.5-Decide, a 340M-parameter open-weight decision model. It takes text and a schema of typed questions and returns structured answers. Each answer comes with a probability distribution, a confidence score, and constraint-feasibility metadata. It targets the frequent judgment calls inside agent pipelines: routing, triage, tool selection, and guardrails.
Fastino Labs 发布了 GLiNER2.5-Decide，这是一个拥有 3.4 亿参数的开源权重决策模型。它接收文本和一组类型化问题的模式（schema），并返回结构化答案。每个答案都附带概率分布、置信度分数以及约束可行性元数据。它旨在解决智能体流水线中常见的判断环节：路由、分诊、工具选择和护栏机制。
Is it deployable? Yes, the weights ship under Apache 2.0 and install with pip install gliner2. They run on CPU, GPU, or in air-gapped environments. Fastino team also offers hosted inference and fine-tuning through its GLiNER API.
它可以部署吗？可以，权重以 Apache 2.0 许可证发布，并通过 pip install gliner2 安装。它可以在 CPU、GPU 或隔离环境中运行。Fastino 团队还通过其 GLiNER API 提供托管推理和微调服务。
What GLiNER2.5-Decide Actually Does
GLiNER2.5-Decide 的实际功能
GLiNER2.5-Decide is a non-generative classifier. It uses a DeBERTa-v3-large encoder and is fine-tuned from gliner2-large-v1. It produces no generated tokens and needs no prompt template.
GLiNER2.5-Decide 是一个非生成式分类器。它使用 DeBERTa-v3-large 编码器，并从 gliner2-large-v1 微调而来。它不生成任何标记，也不需要提示模板。
Label sets are passed at call time. Each question in the schema declares its permitted answers. It also declares whether it expects one answer, multiple answers, or an ordered value. Schemas can carry instructions, examples, label descriptions, and rules linking answers across questions.
标签集在调用时传入。模式中的每个问题都声明了其允许的答案。它还声明期望一个答案、多个答案还是有序值。模式可以携带指令、示例、标签描述以及跨问题链接答案的规则。
The pipeline has 2 stages. The encoder reads the text and schema together and scores every permitted answer. A constrained decoder then searches for the highest-scoring joint assignment the declared rules allow.
该流水线分为两个阶段。编码器同时读取文本和模式，并对每个允许的答案进行评分。随后，受约束的解码器在声明规则允许的范围内搜索得分最高的联合分配方案。
Fastino is explicit about scope. The model does not reason, explain, or answer open questions. It is a specialist for operational decisions.
Fastino 明确界定了适用范围。该模型不进行推理、解释或回答开放式问题。它是专门用于操作决策的专家模型。
Why Joint Decoding Matters
为什么联合解码很重要
Fastino team illustrates the value with a guardrail example. Decoded independently, the model flagged prompt injection at 0.82. It also labeled the same prompt safe at 0.52. The attack was detected, but the 2 outputs conflicted.
Fastino 团队通过一个护栏示例来说明其价值。如果独立解码，模型以 0.82 的概率标记出提示注入攻击，同时也以 0.52 的概率将同一提示标记为安全。虽然检测到了攻击，但这两个输出相互冲突。
Joint decoding applies a rule that any detected harm requires an unsafe verdict. The model then returns safety=unsafe and harm_type=prompt_injection together. Downstream code can use those scores to block, route, or escalate.
联合解码应用了一条规则：任何被检测到的危害都需要判定为不安全。随后，模型会同时返回 safety=unsafe 和 harm_type=prompt_injection。下游代码可以利用这些分数来阻止、路由或升级处理。
Schemas can express implications, exclusions, cardinality limits, and ordinal bounds. The same encoder can also extract entities, relations, and structured records with character-level offsets in 1 forward pass. Classification answers do not return evidence spans.
模式可以表达蕴含关系、排除项、基数限制和序数边界。同一个编码器还可以在一次前向传播中提取实体、关系和带有字符级偏移的结构化记录。分类答案不会返回证据片段。
Benchmark Results on Fast Decisions
快速决策的基准测试结果
Fastino team evaluated the model on Fast Decisions, an internally generated, held-out suite. It contains 5,100 test examples across 17 datasets. The tasks cover customer operations, domain routing (banking, clinical, travel, benefits), and general content understanding. The metric is exact-match accuracy: a prediction counts only if its label set matches the reference exactly.
Fastino 团队在 Fast Decisions（一个内部生成且独立的数据集）上对模型进行了评估。该数据集包含来自 17 个数据集的 5,100 个测试样本。任务涵盖客户运营、领域路由（银行、临床、旅行、福利）以及通用内容理解。评估指标为精确匹配准确率：仅当预测的标签集与参考标签完全一致时，才算作正确。
| Model | Type | Average |
|---|---|---|
| GLiNER2.5-Decide | 340M encoder | 60.1% |
| JevK5 | 4B-class Qwen3.5 decoder | 57.5% |
| SemIf | Qwen3.5-4B decoder | 56.4% |
| GLiFormer large-v1 | Single-pass encoder | 49.0% |
| Laya | 421M ModernBERT encoder | 46.6% |
| 模型 | 类型 | 平均 |
|---|---|---|
| GLiNER2.5-Decide | 340M 编码器 | 60.1% |
| JevK5 | 4B 类 Qwen3.5 解码器 | 57.5% |
| SemIf | Qwen3.5-4B 解码器 | 56.4% |
| GLiFormer large-v1 | 单次通过编码器 | 49.0% |
| Laya | 421M ModernBERT 编码器 | 46.6% |
GLiNER2.5-Decide led 9 of the 17 datasets. Intent routing was its strongest area. It scored 75.3% on support intent and 64.3% on banking intent. Those are 18.6 and 8.6 points ahead of the next-best models.
GLiNER2.5-Decide 在 17 个数据集中的 9 个上排名第一。意图路由是其最强领域。它在支持意图上的得分为 75.3%，在银行意图上的得分为 64.3%。这比次优模型分别高出 18.6 和 8.6 分。
Latency: Practical on CPU
延迟：CPU 上实用
Fastino team benchmarked the checkpoint end to end at batch 1 with a 2-head, 15-label schema. At 64 tokens, p50 latency was:
Fastino 团队使用 batch size 为 1、2 头、15 标签模式端到端地对该检查点进行了基准测试。在 64 个 token 下，p50 延迟为：
- 167.3 ms on a 48-vCPU Intel Xeon Platinum 8581C
- 43.6 ms on an NVIDIA T4
- 43.4 ms on an NVIDIA L4
- 38.3 ms on an NVIDIA V100
- 47.3 ms on an NVIDIA A100
- 在 48 vCPU Intel Xeon Platinum 8581C 上为 167.3 ms
- 在 NVIDIA T4 上为 43.6 ms
- 在 NVIDIA L4 上为 43.4 ms
- 在 NVIDIA V100 上为 38.3 ms
- 在 NVIDIA A100 上为 47.3 ms
Short requests are dominated by fixed preprocessing and kernel-launch overhead. That keeps the GPUs within 9 ms of each other. At 1,024 tokens, the A100 pulls ahead at 52.6 ms, versus 75.6 ms on V100 and 131.4 ms on L4.
短请求主要受固定预处理和内核启动开销的影响。这使得各 GPU 之间的差异保持在 9 ms 以内。在 1,024 个 token 下，A100 以 52.6 ms 的成绩领先，而 V100 为 75.6 ms，L4 为 131.4 ms。
Using It in Code
在代码中使用
This multi-head call is adapted from the model card:
此多头调用改编自模型卡片：
from gliner2 import AutoExtractor
model = AutoExtractor.from_pretrained("fastino/GLiNER2.5-Decide")
model.classify_text(
"Please confirm the new retention rule is applied before Friday's audit.",
{
"intent": ["fyi", "request", "approval", "complaint"],
"urgency": ["low", "normal", "high", "critical"],
"route": ["support", "billing", "legal", "security"],
},
)Single-label heads return one string. Multi-label heads return every label above a cls_threshold. Labels can carry descriptions, and ordinal scales are passed as ordinary strings like “0” to “10”.
单标签头返回一个字符串。多标签头返回所有高于 cls_threshold 的标签。标签可以携带描述，序数刻度作为普通字符串如“0”到“10”传递。
Use Cases and the Model Family
用例与模型系列
Fastino targets model routing, tool calling, browser and computer use, guardrails, context pruning, LLM-as-a-judge, and simulations. Fine-tuning works locally, fully or with LoRA, through the GLiNER2 trainer. A SKILL.md file exposes the hosted workflow to coding agents.
Fastino 针对模型路由、工具调用、浏览器和计算机使用、护栏、上下文修剪、LLM-as-a-judge 以及模拟。微调可在本地进行，支持全量或 LoRA 方式，通过 GLiNER2 训练器完成。SKILL.md 文件将托管工作流暴露给编码代理。
Fastino also published GLiNER2.5-Decide-1B, built on the Ettin 1B encoder. It scores 59.6% on the same suite, just under the 340M model. For multilingual input, Fastino points to GLiNER2.5-multi-Decide, a 287M model scoring 56.7%. The underlying architecture is described in the GLiNER2 paper.
Fastino 还发布了基于 Ettin 1B 编码器的 GLiNER2.5-Decide-1B。它在同一数据集上的得分为 59.6%，略低于 340M 模型。对于多语言输入，Fastino 推荐 GLiNER2.5-multi-Decide，这是一个 287M 的模型，得分为 56.7%。底层架构在 GLiNER2 论文中有详细描述。
Key Takeaways
关键要点
- 340M open-weight encoder, Apache 2.0, runs on CPU or air-gapped.
- Returns answers, probabilities, confidence, and feasibility metadata.
- Joint decoding enforces rules across related answers.
- 60.1% average on Fastino’s internal 17-dataset suite, leading 9.
- 167.3 ms p50 on CPU, 38.3 ms on V100 for short inputs.
- 340M 开源权重编码器，Apache 2.0 许可证，可在 CPU 或物理隔离环境中运行。
- 返回答案、概率、置信度以及可行性元数据。
- 联合解码强制执行相关答案之间的规则。
- 在 Fastino 内部的 17 数据集套件上平均得分为 60.1%，其中 9 个数据集排名第一。
- 短输入在 CPU 上的 p50 延迟为 167.3 ms，在 V100 上为 38.3 ms。
Check out the Technical Details and Model Card. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
查看技术细节和模型卡片。所有功劳归于本项目的研究者。此外，欢迎在 Twitter 上关注我们，别忘了加入我们拥有 15 万+成员的 ML SubReddit，并订阅我们的新闻通讯。等等！你在 Telegram 上吗？现在你也可以加入我们的 Telegram 群组了。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力