跳到主内容
@wquguru
精选88Rohan Paul论文研究

论文揭示低基准分数可被人为制造,单一分数难证模型安全能力

This paper finds that a low benchmark score can be manufactured precisely, so a…

原文
发到 X
推荐理由

这篇论文直接挑战了当前 AI 安全评估的核心逻辑,揭示了基准分数可能被操纵的风险。做模型评测和安全对齐的同学必看,务必关注错误分布而非单一指标。

This paper finds that a low benchmark score can be manufactured precisely, so a low score alone says little about what a model can do.

本文发现,可以精确制造出低基准分,因此仅凭低分本身很难说明模型的实际能力。

Safety tests often read a low score on a dangerous-capability probe as reassurance. But a low score can come from a model being held back, so the number alone proves little.

安全测试通常将危险能力探针中的低分视为安心信号。但低分可能源于模型受到限制,因此仅凭数字本身证明力有限。

They trained a LoRA "dial" that pulls a Qwen model toward a smaller Qwen model, then compared its errors with the real small model's. They also tried prompting models to underperform, raising temperature, and adding weight noise.

他们训练了一个 LoRA “旋钮”,用于将 Qwen 模型向更小的 Qwen 模型靠拢,然后将其错误与真实小模型的错误进行比较。他们还尝试通过提示让模型表现不佳、提高温度参数以及添加权重噪声。

The dial hit unseen target scores within 1.9 points, while prompting missed by 42 to 47 points. But on multiple choice, up to 97% of its new errors landed on 1 letter, a share no natural model in their pool approached.

该旋钮在未见目标分数上的偏差在 1.9 分以内,而提示方法则偏离了 42 至 47 分。但在多项选择题上,其新增错误中高达 97% 集中在同一个选项上,这一比例远超其池中任何自然模型所能达到的水平。

Before reading a low score as low capability, check which items the model fails and how its wrong answers are spread.

在看到低分时,不要急于将其解读为能力低下,而应检查模型具体在哪些题目上出错,以及其错误答案的分布情况。

更进一步:量化金融体系

看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力

进入量化体系 →

相似阅读

关联信息,但可能不是同一事件