Meta新框架：LLM裁判在质疑下频繁改判
Brilliant new paper from Meta.
Brilliant new paper from Meta.
Meta 发表了一篇精彩的新论文。
LLM judges get validated on accuracy against golden data. That says nothing about whether the verdict survives when questioned.
LLM 裁判在黄金数据上的准确性得到验证，但这并不能说明其裁决在受到质疑时是否依然成立。
The Wiggle Framework stress-tests 9 frontier models across 14 judging tasks along three axes, stability under re-prompting, stability under a single challenge, and stability under sustained pressure.
Wiggle 框架对 9 个前沿模型在 14 项裁判任务中进行了压力测试，涵盖三个维度：重新提示下的稳定性、单次挑战下的稳定性以及持续压力下的稳定性。
They find that every model wiggles. Verdicts flip 25 to 71% of the time under static pushback, and 62 to 91% against an adversarial persuader.
他们发现每个模型都会“摇摆”。在静态反驳下，裁决翻转的概率为 25% 至 71%，而在对抗性说服下，翻转概率高达 62% 至 91%。
Pressure that changes a judge's verdict is almost always net-corrupting against ground truth.
改变裁判裁决的压力几乎总是相对于真实情况产生净破坏作用。
Baseline jury majority strength turns out to be the best single-shot predictor of which items will move.
基线陪审团多数优势被证明是预测哪些项目会变动的最佳单次预测指标。
Paper: https://arxiv.org/abs/2608.12645
论文链接：https://arxiv.org/abs/2608.12645
Track more trending AI papers in our academy: https://academy.dair.ai/
在我们的学院中追踪更多热门 AI 论文：https://academy.dair.ai/
更进一步：量化金融体系
看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力