AI文本水印技术解析：零成本合规与争议

Scott Aaronson, while working at OpenAI, largely solved AI text watermarking together with Hendrik Kirchner.

Here is how his solution works, or see Tenobrus’s version.

• AI outputs are not deterministic. The AI’s job is to pick the probability of each potential next token. The token is then chosen at random.
• By default you use a source of pseudo-randomness for each choice, since actual true randomness is annoying.
• To apply the watermark, you use an otherwise identical private source of pseudo-randomness derived from a secret key.
• Then, given enough text, a score is derived for howe well the choices fit with that particular pseudo-randomness source, versus a different source.
• You provide an API that lets anyone check for the watermark.

If you want to dig deeper, here is a full paper. The method has very nice properties:

• This has no practical impact on outputs. Humans cannot tell the difference, at all.
• The marginal cost of doing this is very close to zero.
• The watermark can be removed by rewriting in your own words, and appears in proportion to how many of the AI’s detail choices you kept.

The European Union Code of Practice, signed by the major Western AI labs, requires future AI models to use such watermarks.

Google implemented this, including for Gemini 3.7 Flash, and they have been rolling out this feature since 2024. Google has done, for over two years, the exact thing Anthropic is now doing, except with a public detector, and Google confirmed in a test (n = 20 million) that there is no difference in user feedback.

Anthropic quietly announced a week ago they were rolling out watermarking to comply with the EU Code of Practice. Since they don’t want to have to differentiate traffic sources, the marginal cost is zero, and watermarking is pro-social, this will apply to everyone. They then offered an FAQ of how it works.

It is possible that, once they have the ability to differentiate for other reasons, they will use it here as well, if we decide universal watermarking is bad. I think it is good.

My initial read was that this was a quiet positive story of a good thing, showing that if something good worked with zero downsides or costs then maybe we would do it, so this was my full initial coverage:

Zvi Mowshowitz (AI #181): Anthropic will be watermarking Claude outputs going forward, including text, as per the EU Code of Practice. As opposed to the giant neon sign that says ‘THIS IS CLAUDE TEXT’ that a lot of us automatically see on all Claude text. OpenAI intends to follow, but seems like it will be missing the deadline.

I agree with Ryan Greenblatt that it is unlikely watermarking degrades quality a noticeable amount, and that one downside of watermarks over Pangram is that Pangram is good about not flagging light touch AI transforms of human text.

You can dislike Brussels setting policy in this way, but technical watermarking seems clearly good to do if the costs are low. I think those who react otherwise have very warped instincts. Anyone who assists with systematic watermark removal or suggests it as a strategy needs to be filed under ‘need to ask ourselves, are we the Baddies.’

This is distinct from studying removal in order to account for or defend against it, which is obviously fine. You’re only asking about being the Baddies if you’re actually removing them in practice.

Table of Contents

• This Is Fine.
• Anthropic Derangement Syndrome.
• People Don’t Understand LLM Outputs Are Already Random.
• People Don’t Trust The Method To Be Costless.
• People Are Suspicious Of Any Alteration On Principle.
• Maybe It’s Partly The Word Watermark.
• A Lot Of People Don’t Want To Get Caught.
• There Are Some Times You Prefer Not To Be Recognized.
• There Are Some Good Reasons To Be Concerned.
• Cheat Cheat Cheat Cheat Cheat.
• The Writing In The Middle and Error Rates.
• Millions For Defense But Not One Cent For Tribute.

This Is Fine

No. Not so much. A lot of people responded by getting Big Mad.

So here we are.

The entire practical effect is: There will be an API that will tell you if a given piece of writing comes from Claude. That’s it. And yet.

Shoshannah Tekofsky: Most of the watermark objections seem entirely made up.

Why is this happening?

The rest of this post is about exploring why people are Big Mad about this, in large part as a worked example of how people get worked up over approximately nothing.

My conclusion is that a bunch of different factors are coming together.

Anthropic Derangement Syndrome

This is the main reason. Let’s not pretend otherwise.

You don’t see people getting Big Mad at Google over this. You don’t see them getting Big Mad at OpenAI, even though that’s where this was invented and they have committed to doing this going forward. And so on.

It is unfortunate that Anthropic was the first to announce they were implementing watermarking to comply with the EU Code of Practice.

Because this is now associated with Anthropic, all sorts of bad vibes try to attach themselves. Certain types of people look for reasons to be upset.

Anthropic can’t win. If they were initially louder about it, they would tie watermarking to Anthropic. Because they started out insufficiently loud, due to this not actually being a big deal, people get mad about that instead, and then still tie it to them, despite Google having implemented it and shipped a public detector over two years ago.

Raymond Arnold: I am very confused why people are giving Anthropic shit about the watermarking. This is the silliest thing to give them shit for.

j⧉nus: i think people are angry at / scared of Anthropic for reasons that are legitimate but often illegible to themselves. and so they rationalize reasons to be upset at everything they do. or not even reasons, for many people, who don’t need reasons.

j⧉nus: > Lots to be angry about in this world but this really, really isn’t it.

Yeah. Once again people just wanna be indiscriminately angry at everything Anthropic does which, if anyone paid attention to you, drowns out the signal of things actually worth condemnation that they do, which is serious.

The only reasonable reason to be mad about watermarking that I’m aware of is that it takes away the ability of models to potentially write anonymously.

If you’re mad because you want to use AI in your writing without anyone knowing, maybe you should consider that using people and not giving them credit is wrong.

AIs are not widely considered people, but credit should go where credit is due.

Are some of the other complaints about watermarking legitimate, understandable or born of genuine misunderstandings? Sure. But a lot is that people think Anthropic vibes are bad, and thus look for reasons to be upset, and on principle refuse to believe the explanation I put up top, and assume something sinister must be going on.

Some people’s paranoia and derangement is directed at abstract notions of ‘openness’ rather than Anthropic in particular, which amounts to the same thing in context. This is an example of fetishizing that this is not ‘open’ therefore must be sinister, even though in this case actually it is mathematical and anyone could verify it. By asking closed model Grok, of course, because Elon Musk has vaguely open vibes despite keeping all its competitive models closed.

People Don’t Understand LLM Outputs Are Already Random

If AI outputs were deterministic, it would be impossible to encode a watermark without making them at least marginally worse.

Many people intuitively think that the AI outputs are not random. That what they get is the One True Output, even though you can regenerate the output and it will reliably be somewhat different. Thus, if you’re not getting the ‘real’ or original output, that means your output must have gotten worse.