用Qwen3-1.7B构建单遍推理决策模型及校准实战
Build your own decision model
提供了一套完整的从约束解码到概率校准的工程落地方案，代码可复用性强，适合需要低延迟推理或高可靠性输出的开发者参考。
"System one" decision models are models that infer and respond with calibrated probabilities or every allowed answer.
"System one"决策模型是能够推断并以校准后的概率或每个允许的答案进行响应的模型。
Consider your everyday language model, to get typed output from it (JSON), you may use Structured Output to constrain the output to guaranteed valid JSON. While model prefills the input in one pass, it still has to go perform a pass for every token in order to generate a valid response.
考虑你日常使用的语言模型，要从中获取文本输出（JSON），你可以使用结构化输出来约束输出，确保其为有效的 JSON。虽然模型在一次前向传播中完成输入的预填充（prefill），但它仍需为每个 token 执行一次前向传播才能生成有效的响应。
In this example, 11 passes are required to generate the final output. (We're not accounting for speculative decoding and other inference optimization techniques.)
在此示例中，需要 11 次前向传播才能生成最终输出。（此处未考虑投机解码及其他推理优化技术。）
Step Play Reset
步骤：播放重置
prompt (prefill) generated (predicted) being predicted
提示词（预填充）已生成（预测）正在被预测
Decision models such as Jev, make the assumption that there are fixed options we can select from and we can do so quickly by making a single pass. In this example we constrain the set of possible outputs to the options A, B, C, D, E. By masking other items in the vocabulary, the model can only emit those tokens. By selecting the highest probability output, we get our answer.
像 Jev 这样的决策模型假设存在一组固定的选项供我们选择，并且我们可以通过单次前向传播快速做出选择。在此示例中，我们将可能的输出集约束为选项 A、B、C、D、E。通过屏蔽词汇表中的其他项，模型只能发出这些 token。通过选择概率最高的输出，我们便得到了答案。
Step Play Reset
步骤：播放重置
prompt (prefill) generated (predicted) being predicted
提示词（预填充）已生成（预测）正在被预测
Since the outputs are constrained to only a fixed set of options, the model can't select anything outside of those. This however does not guarantee that the output will be correct. It's also common to treat the output token probabilities a confidence scores in this context, but without additional training, those scores likely reflect its confidence in what the next token will be rather than the true probability of the response being the correct answer.
由于输出被限制在固定的一组选项内，模型无法选择范围之外的内容。然而，这并不能保证输出是正确的。在此语境下，通常也将输出的 token 概率视为置信度分数，但如果没有额外的训练，这些分数很可能反映的是模型对下一个 token 的置信度，而非该响应确实是正确答案的真实概率。
Build your own
构建你自己的模型
We can emulate this behavior by constraining output tokens using an LLM. Here I'm using Qwen/Qwen3-1.7B
我们可以通过使用大语言模型（LLM）来约束输出 token，从而模拟这种行为。这里我使用的是 Qwen/Qwen3-1.7B。
import argparse
import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Qwen/Qwen3-1.7B"
options = ["A", "B", "C", "D", "E"]
parser = argparse.ArgumentParser()
parser.add_argument("--input", default="question.json")
args = parser.parse_args()
# load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
# the token the model would emit for each option as the first assistant token
option_token_ids = [tokenizer.encode(opt, add_special_tokens=False)[0] for opt in options]
def format_prompt(item):
prompt = item["question"] + "\n"
for opt in options:
prompt += f"{opt}. {item[opt]}\n"
prompt += "Answer:"
messages = [
{"role": "user", "content": prompt}
]
return tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
enable_thinking=False
)
with open(args.input) as f:
item = json.load(f)
model_inputs = tokenizer(format_prompt(item), return_tensors="pt").to(model.device)
with torch.no_grad():
logits = model(**model_inputs).logits[0, -1]
# constrained decoding: only the option tokens are allowed
probs = torch.softmax(logits[option_token_ids].float(), dim=-1)
print(f"prediction: {options[probs.argmax().item()]}")
for opt, prob in zip(options, probs.tolist()):
print(f"{opt}: {prob:.4f} {item[opt]}")Running it with an simple question to test it yeilds the following output
使用一个简单的测试问题运行它，产生了以下输出：
// input
{
"question": "What color is the sky?",
"A": "Red",
"B": "Blue",
"C": "Green",
"D": "Purple",
"E": "I don't know"
}
// output
prediction: B
A: 0.0000 Red
B: 0.9988 Blue
C: 0.0000 Green
D: 0.0000 Purple
E: 0.0012 I don't knowThe model was able to make sense of our input and make a prediction that reasonably corresponds to the correct answer.
模型能够理解我们的输入，并做出了一个与正确答案合理对应的预测。
We can test the accuracy of the model by running it against public datasets. I ran this against a random sample holdout of CommonsenseQA
我们可以通过在公开数据集上运行模型来测试其准确性。我在 CommonsenseQA 的一个随机抽样保留集上进行了测试。
precision recall f1 support
A 0.5733 0.7197 0.6382 239
B 0.5506 0.7686 0.6416 255
C 0.5372 0.6598 0.5922 241
D 0.7206 0.3904 0.5065 251
E 0.7519 0.4255 0.5435 235
accuracy: 725/1221 = 0.5938
macro f1: 0.5844Not bad for a 1.7B model, Running a quick finetune on the dataset gives us slightly better performance
对于一个 1.7B 参数的模型来说，表现不错。在该数据集上进行快速微调后，性能略有提升。
precision recall f1 support
A 0.6475 0.6611 0.6542 239
B 0.6113 0.6784 0.6431 255
C 0.6234 0.5975 0.6102 241
D 0.6700 0.5418 0.5991 251
E 0.5808 0.6426 0.6101 235
accuracy: 762/1221 = 0.6241
macro f1: 0.6234Calibrating your model
校准你的模型
Testing the model against a very ambiguous problem demonstrates an interesting problem.
在一个非常模糊的问题上测试模型，揭示了一个有趣的问题。
// input
{
"question": "Where would you most likely find a bat?",
"A": "Cave",
"B": "Baseball game",
"C": "Attic",
"D": "Zoo",
"E": "Sporting goods store"
}
// output
prediction: A
A: 0.9978 Cave
B: 0.0004 Baseball game
C: 0.0017 Attic
D: 0.0000 Zoo
E: 0.0001 Sporting goods storeThere should be no clear answer here, but treating the output probabilities as a pseudo "confidence" score, shows that the model is extremely overconfident in this answer.
这里本不应有明确的答案，但如果将输出概率视为伪“置信度”分数，则显示模型对该答案表现出极度的过度自信。
If we bin the confidence score ranges in the eval I ran earlier, we can see that the model's confidence does not match its accuracy. This means that the model is not calibrated.
如果我们将之前评估中运行的置信度分数范围进行分箱，我们可以看到模型的置信度与其准确性不匹配。这意味着模型未校准。
bin count confidence accuracy
(0.00, 0.10] 0 0.0000 0.0000
(0.10, 0.20] 0 0.0000 0.0000
(0.20, 0.30] 3 0.2834 0.0000
(0.30, 0.40] 26 0.3761 0.2692
(0.40, 0.50] 41 0.4538 0.2683
(0.50, 0.60] 70 0.5490 0.3286
(0.60, 0.70] 74 0.6476 0.3649
(0.70, 0.80] 77 0.7495 0.4286
(0.80, 0.90] 121 0.8555 0.4711
(0.90, 1.00] 809 0.9855 0.7009We can notice that the model tends to be extremely overconfident in the 0.9 - 1.0 bin but it's only correct 70% of the time. When it makes a prediction with 0.8 - 0.9 confidence it's only accurate ~40% of the time. This means that the model is generally overconfident in its predictions.
我们可以注意到，模型在 0.9 - 1.0 的分箱中往往表现出极度的过度自信，但其正确率仅为 70%。当它以 0.8 - 0.9 的置信度进行预测时，准确率仅为 ~40%。这意味着模型对其预测通常过于自信。
Since our goal is to have the model output scores that is reflective of its accuracy, one method we can use to callibrate it is through temperature scaling. By modifying the temperature value, we can flatten its output probability distribution curve and scale it to approximate its accuracy.
由于我们的目标是让模型输出的分数反映其准确性，我们可以用来校准它的一种方法是通过温度缩放（temperature scaling）。通过修改温度值，我们可以使其输出概率分布曲线变得平缓，并将其缩放以近似其准确性。
temperature T
温度 T
Curve fitting fit the temperature parameter to the model's accuracy, I found 3.797280788421631 as a temp value.
通过曲线拟合将温度参数与模型的准确性相匹配，我找到的温度值为 3.797280788421631。
bin count confidence accuracy
(0.00, 0.10] 0 0.0000 0.0000
(0.10, 0.20] 0 0.0000 0.0000
(0.20, 0.30] 82 0.2712 0.2317
(0.30, 0.40] 217 0.3507 0.3917
(0.40, 0.50] 199 0.4472 0.5126
(0.50, 0.60] 166 0.5475 0.5482
(0.60, 0.70] 139 0.6562 0.5827
(0.70, 0.80] 140 0.7492 0.7714
(0.80, 0.90] 169 0.8507 0.7988
(0.90, 1.00] 109 0.9333 0.9541This gets us a much better calibration. If you want to play around with this, I made a GitHub repo with scripts that walk you through building a dataset, evaluating, finetuning and calibrating your own model. I encourage pulling it and trying it on other bigger models.
这使我们获得了更好的校准效果。如果你想尝试一下，我创建了一个 GitHub 仓库，其中的脚本会引导你构建数据集、进行评估、微调以及校准你自己的模型。我鼓励你拉取该仓库并在其他更大的模型上尝试。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力