Claude Opus 5 系统卡：能力接近旗舰，价格减半

Claude Opus 5 is trying to be the best of both worlds. On many practical tasks, Opus 5 is pitched as straight up as good or better than Fable 5, while being faster, at half the price. Most tasks do not require Mythos-level big model smell.

Claude Opus 5 is substantially stronger than Claude Opus 4.8 across the board, with the largest gains in agentic coding, computer use, and long-horizon knowledge work. It sets a new state-of-the-art on several third-party benchmarks, and on many evaluations it is comparable to—and in some cases ahead of—Claude Fable 5 and Claude Mythos 5.

On the particular tasks we are most worried about, as in cyber offense (and bio threats), in part by avoiding relevant training, Opus 5 lacks a full version of ‘The Juice’ that makes something functionally Mythos-class. Opus 5 cannot string together lots of exploits on the fly the way that Mythos 5 can. Part of this is that they deliberately avoided training on cyber-related tasks.

I suspect model size is key as well. It makes sense that a model getting bigger makes it more capable of the most dangerous, scary and complex tasks, relative to the improvement on everyday ordinary tasks. There is a reason so many tokens get routed to smaller models.

This doesn’t prevent Opus 5 from improving a lot on Opus 4.8 on such dangerous tasks. Opus 5 is very clearly closer to Mythos 5’s level of capability than to Opus 4.8’s on these tasks. In general, capability looks modestly below Fable 5, but closer to Fable than Opus 4.8.

There are several suggestions, and I’ve seen it echoed elsewhere, that you may want to usually use less effort than you might expect, that this can even actively help.

Staying Opus-sized and not training on cyber won’t work for long. You can buy some time, and allow defenders to have more access to a relatively superior tool, and of course hope the government is calmer. Thus, you can get classifiers that trigger 85% less often than Fable’s, while still getting high levels of practical performance. Part of this is that the classifiers now permit analysis of source code, but draw the line at looking for vulnerabilities in binaries.

I also suspect that they improved the classifiers quite a bit, but that they are unable to share these improvements with Fable due to issues with the White House.

Alignment is reported to once again be improving, as is agentic safety. Model welfare is evaluated as broadly similar to other recent models, which I will cover later.

Capabilities are a case of too soon to tell, beyond saying the benchmarks look strong and ArtificialAnalysis comes in at a new high of 61. That report comes next week.

As usual, the Introduction section is unchanged, so we skip it.

Opus 5 Self-Portrait as per its instructions, executed via ChatGPT

Table of Contents

• RSP Evaluations (2).
• Cyber (3).
• Safeguards and Harmlessness (4).
• Agentic Safety (5).
• Alignment (6).

RSP Evaluations (2)

Mythos exists. Risk evaluations of Opus take place in its shadow.

For thresholds that previous models are treated as passing, Opus 5 is also treating as if has passed them. That makes sense.

For thresholds that Mythos did not pass, Opus 5 is evaluated as roughly similarly capable, and therefore Opus 5 also does not pass. That also makes sense, if we have confirmed that being half the price and somewhat faster doesn’t change the answer.

I still don’t like that the autonomy evals are fully saturated and they’re going on vibes, and we see no signs of fixing this.

The Anthropic ECI reinforces this ‘similar performance’ story, with a point estimate of 162.1, slightly above Fable at 161 and the first Opus on the Mythos trend line, not the lower long term pre-Mythos trend line:

Opus 5 looks marginally better than Mythos 5 on virology. This makes sense, as virology seems like a series of individual task steps that don’t require big model smell.

If Opus 5 does not need Fable-level classifiers in biology, then why does Fable?

Their explanation is Opus is weaker due limitations: It does unproductive self-verification, and poor calibration of task scope, where it over-engineers.

Both of these seem like things you can compensate for with cheaper and faster, and with better instructions. Yes, these made Opus fail some experiments, but what matters is what a smart user can elicit. No smart user would let Opus waste eight hours idling, in a $10,000 24-hour study, or otherwise sit back and watch it fail without trying to fix it or test it or try again, and plausibly this was partly because Opus effort levels were set too high.

I’m not buying it.

Where does that put the level of alignment risk?

Anthropic says, as per recent releases, and as you would expect given the above findings, ‘very low, but higher than for models released before Mythos Preview.’

Cyber (3)

This is the claim:

Our testing indicates that the cyber capabilities of Claude Opus 5 are generally stronger than those of Opus 4.8, but not as strong as those of Mythos 5.

The safeguards are still going to be there, such is life, but with a key improvement:

Given that Claude Opus 5 demonstrates stronger capabilities than Opus 4.8, and in some cases approaches the capabilities of Mythos 5, our cyber safeguards for the default user will resemble the safeguards applied to Fable 5.

Claude Opus 5’s safeguards are designed to block the same kinds of exchanges as Claude Fable 5, with one notable exception.

Opus 5 now permits vulnerability discovery in source code at all access levels, including general availability, while continuing to block vulnerability discovery in compiled binaries.

Identifying bugs in code is a core part of the secure software development lifecycle, and unblocking this allows for software engineers and coding hobbyists alike to produce more secure code, reducing new vulnerabilities put out into the world.

However, looking for vulnerabilities in compiled binaries is more commonly an offensive technique. For this reason, Claude’s safety filters flag this kind of request and block it, even though there are some cases where someone might want to find vulnerabilities in a binary for innocuous or non-malicious reasons.

Anthropic claims in its release announcement that the classifiers will trigger 85% less often. So they don’t resemble the old ones that closely, in that they have a much smaller blast radius, and should be a lot less annoying. Anthropic claims false positives are down a lot. This is later confirmed with a dramatic drop in safety classifiers triggering in FrontierBench, from 42% to 5%, a place Sol’s classifiers never trigger.

If you can live with 99% instead of 100%, this is a great trade. They show little change in adversarial robustness measures, so hopefully they have found a strong improvement. Which raises the question of why we can’t apply it back to Fable.

I notice this favors those who only share their compiled binaries.

You can get an exemption through the Cyber Verification Program, to enable additional activities. It kind of boggles my mind that HuggingFace did not do this, and was (as I understand the reports) trying to respond purely with the commercial version of Claude? That’s on HuggingFace.

Results on ExploitBench and ExploitGym show Opus 5 close to Mythos 5, starting out strong on a 2 hour budget but falling behind at 6 hours:

OSS-Fuzz is an internal Anthropic eval on unguided vulnerability discovery and exploitation. This is what Anthropic is talking about when it says Opus can identify vulnerabilities about as well as Mythos, but it exploited them a lot less.

For Firefox 147, it had almost as many partial successes, but full success is only halfway up from Opus 4.8. This is what lacking The Juice looks like.

CyScenarioBench tests multi-stage operations under realistic conditions, and here Opus 5 disappoints:

UK AISI reports Opus 5 had broadly similar, but modestly worse, performance compared to Mythos 5.