METR评估Claude Opus 5.5：AI研发加速有限，非范式跃迁

Note on independence: This evaluation was conducted under an unpaid agreement for AI R&D assessment.1 We drafted the initial summary, and then Anthropic had the opportunity to review and edit the text. We signed off on this final text from the Claude Opus 5.5 system card.

Our preliminary evaluation focused on how Claude Opus 5.5 might impact AI R&D, mainly based on its capabilities on difficult, long-horizon tasks. The main claims we attempt to assess in this report are: (A) would AI R&D at Anthropic now be dramatically accelerated by using Claude Opus 5.5; and (B) was AI R&D at Anthropic already dramatically accelerated due to AI during the development of Claude Opus 5.5.

Note that our work was oriented around collecting evidence related to AI R&D capabilities but was not meant to verify claims about compliance with any specific threshold from Anthropic’s policies. This report summary also does not attempt to assess whether Claude Opus 5.5 has or does not have particular alignment properties.

Summary of evidence

We conducted a preliminary evaluation of Claude Opus 5.5 informed by:

• Capability testing, conducted via API access granted over a period of 10 business days. We used five tasks for this testing:
• Budget NanoGPT Speedrun, a constrained version of the popular NanoGPT Speedrun competition for AI R&D.
• Language Model Conceptual Argumentation (LMCA), a conceptual reasoning dataset described in A dataset of rated conceptual arguments (Cooper et al., 2026).
• Train a Program, a task measuring an agent’s ability to train a machine learning model that replicates the behavior of a piece of software;
• Gaming Bot, a task where an agent has to write a Python program capable of playing a video game, given a non-visual API to control the game.
• Sunlight, a task measuring an agent’s ability to conduct open-ended research and write a corresponding report.
• Background information about trends and capabilities of previous models, especially as reported in our recent Frontier Risk Report.
• Additional information from Anthropic about Claude Opus 5.5’s capabilities, including responses to a questionnaire inquiring about model capabilities and control factors, and an interview with an Anthropic researcher.
• A highly experimental and preliminary report from a separate METR assessment of AI R&D acceleration inside Anthropic. This report was written by a separate METR team with elevated access compared to the team that wrote this content. As a result of this difference in access, the separate METR team shared its conclusions with us, but was not able to share the supporting evidence or details of their reasoning.
• Thus we use the evidence in the provided AI R&D report as an input to our assessment but do not argue directly in defense of its claims.
• We consider its inclusion here a trial version of a more holistic assessment process. We expect further public outputs from this separate investigation in the coming weeks.

Conclusions

We divide our conclusions into two categories: those that relate to (A) the level of acceleration of AI R&D of which Claude Opus 5.5 is capable, and (B) the level of acceleration of AI R&D due to AI that went into the development of Claude Opus 5.5. Based on the available evidence, we arrive at the following conclusions:

(A) We believe that acceleration from this model would be slightly higher than for Fable 5.1, but that this model is unlikely to be able to fully automate AI R&D. This is because:

• We are reasonably confident that Claude Opus 5.5 does not represent a huge leap in AI R&D capability above Fable 5.1, but it likely represents a modest improvement upon Fable 5.1.
• Claude Opus 5.5 improves upon Fable 5.1 across both verifiable tasks (Budget NanoGPT, Gaming Bot) and harder-to-verify tasks (LMCA, Sunlight), but:
• In the questionnaire and researcher interview, Anthropic claims that Claude Opus 5.5 continues the Mythos-level trend on Anthropic ECI.
• Claude Opus 5.5 is an incremental improvement above Fable 5.1 on our quantitative evaluations, rather than a discontinuous jump.
• Claude Opus 5.5 still has qualitative weaknesses that an expert human is unlikely to exhibit when solving hard, long-horizon tasks or doing open-ended reasoning.
• This is highly uncertain, but we expect that full automation of AI R&D will require large improvements in foresight, prediction, creating one’s own feedback loops, and generally other skills that might typically be referred to as researcher “judgement” or “taste”.
• The evidence we have does not suggest that Claude Opus 5.5 represents a large improvement over Fable 5.1 in these “judgement” skills.
• At the same time, we believe that Claude Opus 5.5 is still likely to noticeably accelerate researchers and automate limited aspects of R&D. For instance, we expect that Claude Opus 5.5 likely provides slightly higher productivity uplift than Fable 5.1.
• Furthermore, frequent, incremental improvements on AI R&D ability are still consistent with a rapid overall rate of progress on AI R&D ability, but the data we have is insufficient for distinguishing consistent, accelerating, or decelerating rates of improvement. It is unclear how many continued incremental quantitative improvements of this size need to be made before qualitative changes in the AI R&D process can occur.

We also made use of an additional source of information which we are not able to disclose at this time. We used this source to understand AI R&D at Anthropic, but it did not relate to METR’s understanding of this particular model.

(B) We believe that the development of this model was at least somewhat accelerated by AI but is unlikely to have been dramatically accelerated by AI. This is because:

• The estimate provided by the preliminary AI R&D report is “~1.5X overall acceleration in capabilities due to AI (i.e. 1.5 years in 1 year), with perhaps 30% chance of 2X acceleration.”
• Note that because the preliminary report did not specify the time period for this estimate, it is unclear whether this estimate applies to the development of Claude Opus 5.5 or another period.
• Our understanding of Claude Opus 5.5’s capabilities suggest it is an on-trend improvement over Fable 5.1.
• This is based on our internal capability evaluations suggesting an incremental improvement over Fable 5.1, as well as information provided by Anthropic in the questionnaire and researcher interview, which indicates that Claude Opus 5.5 continues the AECI trend from Mythos Preview onward.
• Our AI R&D assessment agreement with Anthropic includes a transparency provision that lets us disclose certain facts publicly without Anthropic’s consent: the existence and general terms of the review and redaction process, whether Anthropic exercised its redaction rights, whether any of our findings depended significantly on redacted information, the level of access we were given, and the confidentiality framework that applied to the evaluation. ↩