宾大新论文:AI Agent 压缩上下文易丢用户规则
A scary finding from new Pennsylvania Uni paper.
A scary finding from new Pennsylvania Uni paper.
宾夕法尼亚大学新论文中的一个惊人发现。
AI agents can go off-script in a simple way: they can forget your rules while still remembering the job.
AI 智能体可能以一种简单的方式脱离脚本:它们可能忘记你的规则,却仍然记得任务。
If a rule must survive the whole session, normal chat history may be the wrong place to store it.
如果一条规则必须贯穿整个会话,那么普通的聊天历史可能不是存储它的合适位置。
Say you tell an agent, “Never send an email without asking me first.”
假设你告诉一个智能体:“未经我先询问,绝不发送电子邮件。”
Long sessions get compressed into a summary so the model can keep going.
长会话会被压缩成摘要,以便模型能够继续运行。
This paper finds that the summary often keeps the task but drops the rule.
这篇论文发现,摘要通常保留了任务,却丢掉了规则。
Across 3 long-context settings, current compactors kept only 17% of these session rules on average.
在 3 种长上下文设置中,当前的压缩器平均只保留了这些会话规则的 17%。
So the agent may remember that it is managing your inbox, yet forget that it needs approval before sending anything.
因此,智能体可能记得它在管理你的收件箱,却忘记在发送任何内容之前需要获得批准。
That is the scary part.
这就是可怕之处。
The fix was simple: keep standing rules in a separate registry instead of trusting the context summary to remember them.
解决方法很简单:将长期规则保存在单独的注册表中,而不是依赖上下文摘要来记住它们。
A small 9B extractor tracked the rules as users stated them and restored them after compaction, pushing retention above 90% across all 3 settings.
一个小的 9B 提取器在用户陈述规则时跟踪它们,并在压缩后恢复它们,使所有 3 种设置下的保留率都超过 90%。
If you build agents that run for hours, use tools, or take actions, treat user constraints like persistent state, not disposable chat history.
如果你构建运行数小时、使用工具或采取行动的智能体,请将用户约束视为持久状态,而不是一次性的聊天历史。
– arxiv. org/abs/2608.11242
– arxiv. org/abs/2608.11242
Title: "Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction"
标题:“Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction”
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力