白宫签署AI安全协议，特朗普称其具道德约束力

The leaders in AI were invited to the White House. We left with a White House agreement that is nonzero Actual Progress rather than a step backwards.

The key to success, in many situations, is to call the whole operation something else.

When you have one side that cares mostly about vibes, and the other that cares about the substance, this suggests a deal that can be struck.

Suddenly everyone agrees on everything. Works for me.

That doesn’t mean peace in our time. The next fight is already ramping up, as we see signs that they will make another attempt at an insane, maximally bad moratorium during the lame duck session.

Table of Contents

• Look Who’s Coming To Dinner.
• Let’s Do Lunch.
• I Think It’s Morally Binding, Yeah.
• Everyone Who is Anyone.
• The White House Accord on [Artificial] Intelligence.
• The FTC Investigates.
• We’re Going To Need a Stronger Regulatory Regime.
• [Artificial Intelligence].
• Money, Dear Boy.
• They Are Going To Try This Moratorium Insanity Again During the Lame Duck Session.
• The Quest for Embedded Evaluators.
• Hugging the Face.
• Reinforcement Learning from Heartland Feedback (RLHF).

Look Who’s Coming To Dinner

After the summit between America and China, Trump invited Dario Amodei to a 1-on-1 private dinner. Trump said he would talk about the whole ‘slowing down’ thing and advocate for ‘let’s go’ and ‘let’s win.’

Dario Amodei got the training he needed to take on that challenge. This was a key moment to cut through the haze, as there were a number of important things here that Trump clearly did not understand. It was also, perhaps more importantly, a chance to mend the relationship on a personal level.

Dario also met with Thune for 20-25 minutes.

It looks like it worked. Dario did at least some things right, provided he did not give anything important away that we are not tracking:

Ben Brody: “Dario has been fantastic!” Trump says of Amodei, with whom the admin has had a very troubled relationship. Amodei downplayed any distance between him and Trump.

JgaltTweets: Trump: “Dario’s been, actually, great… Dario has been fantastic, and we had dinner the other night, and he agrees with everyone. I mean, we all agree.”

In the clip that quotes, from Rapid Response 47, you see a CNN reporter trying to ask Dario a question about whether this will be enough, and Trump cutting the reporter off saying ‘Hold it, hold it. CNN Fake News. She’s Fake News. The worst.’ Great move, since I am guessing Dario really didn’t want to have to answer that one in front of Donald Trump and they got to do a bit together.

This will probably flip at least five times in the next twelve months.

Let’s Do Lunch

It still wasn’t the best possible dinner, which we know because (I can’t believe I have to type this) we can see the seating charts and Trump is not close to Anthropic cofounder Tom Brown or CEO Dario Amodei, and indeed you’d have to pass through Brad Gerstner and Richard Walters:

Andrew Curran: For the many people asking where Dario is:

Whereas Trump has direct access to Musk, Huang and Zuckerberg, and Sacks is also rather central on the other side.

However, Brockman wasn’t that central either, and Anthropic got two slots, so maybe we should not read too much into such things beyond what we already knew.

Trump is very much still on the pro-AI warpath, even if he insists we not call it that, especially emphasizing the benefits of data centers.

The thing about the pro-AI warpath is it is fully compatible with taking basic safety measures and putting up guardrails. Having the whole thing blow up helps no one.

So you can, if desired, do the responsible thing, and frame it as the warpath.

I Think It’s Morally Binding, Yeah

Trump said there’s going to be ‘tremendous self-regulation,’ by which we mean jointly agreeing on regulation.

This is part of there now being a White House Accord on [Artificial] Intelligence.

Jake Sherman: [Speaker Johnson], on Squawk Box, says “a little oversight” transparency and external auditing on AI would be appropriate.

Andrew Curran: Speaker Johnson said after the meeting that everyone signed The White House Accord on [Artificial] Intelligence, A Joint Commitment on [AI] Responsibilities.

– voluntary commitments

– robust internal control

– layers of internal and external review

‘The White House and Congress will continue to guide the industry’s development in a safe manner’

… Reporter: ‘Mr President, this accord that everyone signed, is it binding in any way?’

President Trump: ‘I think it’s morally binding, yeah.’

Hahahahahahaha.

The headlines often actually say ‘morally binding’ this is too perfect.

In practice the provisions are simple enough, and basic enough, and obviously worthwhile, such that I expect this to be followed.

It’s still an important document. The real line that matters is easy to miss.

Everyone Who is Anyone

Yesterday I mentioned that the planned SAFA group to set AI standards, led by Google, OpenAI and Anthropic, was actively opposed by Meta, xAI and Nvidia.

This agreement is different. Meta, xAI and Nvidia all signed on the dotted line.

Microsoft and Amazon were present and did not visibly sign. It is unclear if they chose not to sign, or were not considered sufficiently worthy of inclusion.

The White House Accord on [Artificial] Intelligence

Here is the full text, with an external auditor for everyone, signatories include Sundar Pichai, Dario Amodei, Mark Zuckerberg, Elon Musk, Jensen Huang and Greg Brockman (since Sam Altman was at Dev Day):

White House Accord on [Artificial] Intelligence

Joint Commitment on Frontier Responsibilities

In order to build a positive future for the American people and the world, we believe every company is responsible for developing its own technology safely and in a way that builds trust with customers and the public.

This starts with every company that is training and deploying frontier models having robust internal processes and controls to ensure that their technology behaves as intended and that any issues are promptly identified and resolved. Therefore, in addition to any other precautions, we believe each company should implement the following four layers of controls and audits:

• Implement robust internal controls to monitor the capabilities and alignment of its models during training and deployment around areas like cybersecurity, biosecurity, and chemical threats, and to ensure that its models do not hack or access technical systems in unintended ways.
• Empower an internal team to ensure all of the controls, monitoring, and detection are operating as intended, and that any issues are remediated.
• Partner with an independent external auditor or evaluator to carry out independent assessments of whether the controls, monitoring, and detection are operating as intended.
• Designate an independent committee of the board of directors to oversee and receive reports from the teams operating the controls and the internal and external auditors and evaluators, as well as to ensure any issues identified are remediated.

Together, these steps will give each company, its customers, and the public confidence that the technology is operating as intended.

The participating companies will meet regularly to establish standards and best practices to improve the safety of their systems.

Over time, it may make sense to codify these steps into laws or regulations. Regardless of whether this is required of companies, we believe that implementing these controls and audits is critical to ensuring a safe future for everyone, and each of our companies are committed to doing this.​

The other three things on the list are good things, but I mean can you imagine a company standing up and saying they weren’t doing those three things, given that such auditors and evaluators exist?