提升LLM推理的8种方法
8 ways to improve LLM reasoning at inference time:
做推理优化的同学必看，这篇把8种推理时技巧的取舍和失败模式讲透了，还给了伪代码，建议直接收藏对照你的任务试一遍。
8 ways to improve LLM reasoning at inference time:
推理时提升LLM推理能力的8种方法：
(used by Google, OpenAI, and Anthropic)
（谷歌、OpenAI和Anthropic使用）
When accuracy on a task falls short, the cheapest place to intervene is inference.
当任务准确率不足时，最便宜的干预点是推理阶段。
There's no retaining, so the techniques either go in the prompt or the orchestration around the call.
无需重新训练，因此这些技术要么进入提示词，要么进入围绕调用的编排。
Inference compute scales in two ways:
推理计算以两种方式扩展：
- You sample the same prompt several times and select among the results - or you extend one trajectory further before the model answers.
- 你对同一提示词采样多次并选择结果 - 或者你在模型回答之前扩展一条轨迹。
Parallel samples batch and run concurrently, so more samples cost tokens and GPU time while it barely affects latency.
并行样本批量运行，因此更多样本消耗令牌和GPU时间，但几乎不影响延迟。
Sequential tokens each condition on the one before, so more thinking costs latency directly.
顺序令牌每个都依赖于前一个，因此更多思考直接增加延迟。
Tree methods compose both, and inherit both failure modes.
树方法组合两者，并继承两者的失败模式。
1) Chain of thought
1）思维链
The model is prompted to think step by step, so intermediate results land in the context where later tokens can condition on them.
模型被提示逐步思考，因此中间结果进入上下文，后续令牌可以依赖它们。
2) Majority voting
2）多数投票
Sample several traces and return whichever final answer appears most often. Agreement replaces a reward model.
采样多个轨迹，并返回出现频率最高的最终答案。一致性取代奖励模型。
It reduces variance and does nothing about bias.
它减少了方差，但对偏差无济于事。
When the model misreads a problem the same way every time, the traces agree, and the vote returns that answer with higher confidence.
当模型每次以相同方式误读问题时，轨迹一致，投票以更高置信度返回该答案。
3) Best-of-N
3）最佳N选一
Score N complete answers with a reward model, keep the top one.
用奖励模型对N个完整答案打分，保留最高分的一个。
True reward rises, peaks, then declines as optimization pressure on the proxy increases. Past that peak, the search starts finding answers the reward model likes instead of the answers that are actually correct.
真实奖励先上升，达到峰值，然后随着对代理的优化压力增加而下降。超过该峰值，搜索开始找到奖励模型喜欢的答案，而不是实际正确的答案。
4) Extended thinking
4）扩展思考
The model spends a token budget on internal reasoning first.
模型首先在内部推理上花费令牌预算。
Longer is not uniformly better. Anthropic built tasks where accuracy falls as traces lengthen, including counting problems where irrelevant numbers pull the model off track.
更长并不总是更好。Anthropic构建了准确率随轨迹长度增加而下降的任务，包括计数问题，其中无关数字使模型偏离轨道。
5) Self-refinement
5）自我完善
The model critiques and rewrites its own answer.
模型批评并重写自己的答案。
On GSM8K, GPT-3.5 fixed 7.6% of its wrong answers and broke 8.8% of its correct ones. This is net negative, and it works when the critique comes from a compiler or a test suite instead.
在GSM8K上，GPT-3.5修复了7.6%的错误答案，但破坏了8.8%的正确答案。这是净负面效果，当批评来自编译器或测试套件时，它才有效。
6) Tree of thought
6) 思维树
Propose several next steps, score them, keep the best few, backtrack from dead ends.
提出几个下一步，对它们进行评分，保留最好的几个，从死胡同回溯。
7) Beam search
7) 束搜索
Keep K partial solutions and score every step with a process reward model.
保留K个部分解，并用过程奖励模型对每一步进行评分。
The technique works better than best-of-N at small budgets and worse at large ones, because the search starts satisfying the scorer instead of solving the problem.
该技术在预算较小时比最佳N选一效果更好，在预算较大时效果更差，因为搜索开始满足评分器而不是解决问题。
8) MCTS
8) 蒙特卡洛树搜索
Extend a promising path, roll out to a full answer, push the score back up, repeat.
扩展一个有希望的路径，展开到完整答案，将分数推回，重复。
The action space forces a cap on node expansion, and applying that limit leaves the search in local optima.
动作空间强制对节点扩展设置上限，而应用该限制会使搜索陷入局部最优。
Every technique here generates candidates and picks among them, so the selector sets the ceiling.
这里的每种技术都会生成候选并从中挑选，因此选择器决定了上限。
As a takeaway, build the verification signal first. Scale the search second.
作为要点，先构建验证信号，再扩展搜索。
To dive deeper, I covered all 8 techniques with the research behind them, the tradeoffs, and pseudo code you can adapt.
为了更深入，我涵盖了所有8种技术及其背后的研究、权衡和你可以调整的伪代码。
Read it below.
在下面阅读。
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力