Anthropic研究员因AI失控风险离职,质疑安全与商业平衡
WSJ just reported: Anthropic researcher Jacob Coxon is quitting AI because he th…
WSJ just reported: Anthropic researcher Jacob Coxon is quitting AI because he thinks self-improving models could become uncontrollable by 2027.
《华尔街日报》刚刚报道:Anthropic 研究员 Jacob Coxon 因认为自我改进的模型可能在 2027 年变得不可控,正退出 AI 领域。
“We’re on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already,” he said.
他说:“我们正朝着许多最激进的场景发展,到明年年底,事情可能就已经失控了。”
His fear centers on recursive self-improvement, where AI takes over enough AI research to speed development of increasingly capable successors.
他的恐惧集中在递归自我改进上,即 AI 接管足够的 AI 研究,以加速开发能力越来越强的后继者。
He moved from OpenAI to Anthropic specifically for its safety work, yet says even sincere safeguards cannot overcome competition without coordinated restraint.
他为了 Anthropic 的安全工作从 OpenAI 跳槽而来,但表示即使真诚的保障措施也无法在缺乏协调克制的环境下克服竞争压力。
The most interesting point is, Coxon is leaving despite believing Anthropic takes safety seriously.
最有趣的一点是,尽管相信 Anthropic 认真对待安全问题,Coxon 还是选择离开。
Anthropic's $2 tn IPO makes that tension so obvious. Anthropic is simultaneously asking the world to believe 2 things: that increasingly powerful AI can create enormous economic value, potentially supporting a $2 trillion public valuation, and that development may eventually need to be slowed when capability crosses dangerous thresholds.
Anthropic 的 2 万亿美元 IPO 使这种张力显得如此明显。Anthropic 同时要求世界相信两件事:日益强大的 AI 能够创造巨大的经济价值,潜在支撑起 2 万亿美元的公开估值;以及当能力跨越危险阈值时,开发最终可能需要放缓。
Those positions create a difficult governance test: will safety commitments still bind when obeying them becomes commercially expensive.
这些立场构成了一个艰难的治理考验:当遵守安全承诺变得商业成本高昂时,安全承诺是否仍能约束行为。
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力