Anthropic 八月风险报告解读：内部模型 Model 2 超

I am grateful that Anthropic is producing periodic Risk Reports.

At first I was skeptical. It turns out I was wrong. Anthropic is revealing a lot of new information, some of it rather alarming, that it did not have to disclose, and is providing detailed insight into how they think about things. This is very cool.

Thus I found this report to be a moderately positive update overall, if we presume they are not silently omitting the worst of it. There are a bunch of not great things we find out about, but I would have expected some set of mistakes at least as bad, and I wouldn’t have expected them to choose to tell us about all of it.

It does mean one more set of 186 page documents I have to read every so often, almost all of which is meaningfully new material this time around.

The other revelation is the existence of the world’s likely best model, ‘Model 2.’

This was a rough one to fully get through, so apologies in advance for any errors of interpretation.

Table of Contents

• Agent Model 1 and Agent Model 2.
• Executive Summary (1).
• The Rules Are Serious But Not Literal.
• Misalignment Is a State of Mind (2.5).
• Autonomy Threat Model 1: Misalignment in High-Stakes Settings (2).
• Some Strange Uses Of The Word Safe I Wasn’t Previously Aware Of.
• Now Versus Future (2.17).
• The Core Claims And Argument (2.6).
• The Rest of the Important Arguments In Section 2.
• Risk Assessment (2.19).
• Pre-Internal-Deployment Review (2.18).
• A Guide To Internal Use Monitoring (2.23.1).
• Blocking Interventions (2.23.2).
• The Power Seeking Environment Evaluation (2.24).
• Opus 4.8-Reward-Hacker (2.25).
• Autonomy threat model 2: Risks from automated R&D (3).
• Yes That Does Seem Kind Of Risky.
• Could We Replace Our Researchers?
• How Much Could We Be Accelerating Our AI Researchers?
• What Could Possibly Go Wrong If We Replaced Our Researchers?
• Risk Mitigations For AI R&D Automation.
• Overall Risk From Automation of AI R&D.
• Biological and Technically Also Chemical Weapons Production.
• The Threat Models for Biological and Chemical Weapons.
• Model Capabilities (4.4).
• Classifiers (4.5).
• Acceleration Dynamics (5.1).
• Distillation (5.1.1).
• Safety Process Failures (5.2).
• Refusing To Find Innovative Misalignment Techniques (5.2.2).
• Exposing the Chain of Thought Reasoning To Grading Pressure Quite a Lot (5.2.3).
• Directly Training On Misaligned Behavior During a Production Training Run (5.2.4).
• An instance of unmonitored unrestricted agents with access to sensitive resources (5.2.5).
• Repeated training on alignment-faking transcript datasets (5.2.6).
• Benefits From Anthropic’s Operating as a Frontier AI company (5.3).
• Model Weight Security (6.4).
• Risk Has Been Reported.

Agent Model 1 and Agent Model 2

There are two internal models referenced in this report.

Model 1 is broadly similar to Mythos Preview and Mythos 5, and is not expected to have wide deployment, externally or internally.

Model 2 is ‘somewhat more capable than Mythos 5,’ a noticeable improvement for internal use, but ‘does not display a capability jump of the degree observed from Claude Opus 4.6 to Mythos Preview.’ This model is going to be internal-use only.

The jump from Opus 4.6 to Mythos Preview was big. Several release cycles big. We get a lot more data throughout the report, but given Model 2 gets that comparison point, one would presume that for internal purposes Model 2 is multiple releases ahead.

To contrast with that, on the AECI it is only 1.5 points ahead, which would be only about a month of progress. My guess is this is a low estimate, and that Model 2 was intentionally not decked out with a bunch of general skills given its purpose. Whereas on the test of substituting for Anthropic’s researchers we see a big jump from 54.8% (Mythos Preview) and 50.3% (Mythos 5) to 62.8% for Model 2.

There are multiple plausible good reasons for not releasing Model 2. Model 2 could be highly specialized for internal tasks. It could also be too capable or dangerous to release, either due to harm or the worry it would accelerate others. The relatively low adoption rate for Fable 5 points towards wanting to hold such a model back.

Executive Summary (1)

The report is divided by each type of risk, by threat model, by relevant AI models, and details current capabilities, behaviors, mitigations and overall level of risk. It then looks forward to the future and offers industry-wide recommendations.

Risk reports cover events up to a month prior, so the coverage date is July 15, 2026. A year ago that would have seemed fine. Now it seems like quite a while, huh?

The Long-Term Benefit Trust (TLBT) is authorized to request external review of the report, but has not done so. I would at minimum do so next time if I was them.

I like that they cover both ‘what is the absolute risk from this’ and ‘what is the marginal risk from this given everyone else exists.’

They deal with autonomy, automated AI R&D and biological and chemical weapons production as risks. It is odd, even now, to exclude cyber risks from the core threat models here. I understand the argument from insufficiently catastrophic on its own, but in practice I think you need to deal with it.

At several points, the risk report essentially concedes versions of my objections, but then forgets that it conceded them and doesn’t alter its conclusions.

Then in Section 5 they get to the more interesting questions.

The Rules Are Serious But Not Literal

The risk threshold definitions have changed from simpler statements to ones with more specific detail.

For AI R&D automation I think the new wording is harder to parse but fine. For novel biological weapons production, the new version narrows to only look at ‘substitute for the scarce human expertise’ to the exclusion of other methods. This is a reasonable primary threat model or scenario, but I worry about looking only at that path.

The both good and bad news is I take such documents seriously, but I no longer take such documents literally, in either direction.

• If Anthropic or another frontier lab has a model that technically crosses their risk threshold, but in a way that seems to be harmless, I expect them to modify their threshold or find some other way to ignore this event.
• If Anthropic or another frontier lab has a model that technically does not cross their risk threshold, but is obviously risky in the way the rules tried to measure, I expect them to act as if it had crossed the threshold.

As always: I would like to see thresholds honored even in cases where you think it ‘should not count,’ because that is what commitments are for, and you should expect rationalizations to occur as you rush to make releases, and you need to set a good example on this. The whole original idea was if-then commitments, where if [X] happens you do [Y], agreed upon in advance, but our civilization seems to lack this technology. We can’t, we don’t know how.

I’m not saying you would never modify your rules to be more lenient, if you have good evidence that you should do that. If you can never make the rules more lenient then the rules have to never be strict. We don’t want that. But the bar for changing the rules, especially in response to a pending event, needs to be a lot higher than it is, and we need to be willing to endure some actual costs even when we now think it is dumb.

For now, Anthropic and also OpenAI have made what seem like good decisions in terms of responding to threats and deciding what to release and not release when, based on what is known at the time, even if they make mistakes elsewhere. That is good. I hope it continues.

Misalignment Is a State of Mind (2.5)

The definition section is greatly appreciated.

I like this definition a lot, although I have important nitpicks:

Misalignment [definition]: Misalignment is a latent property of a specific computation performed by a model in a given context.