前沿节奏之争：Pacing 信函引发讨论

In the wake of the letter calling on us to prepare to potentially Pace the Frontier, there has been much discussion of when pacing the frontier would be prudent, and whether it makes sense to prepare to do so.

This has now been informed by the events surrounding OpenAI training models for months while they had access to a joint de facto message board, which was detected only in the wake of the hacking of HuggingFace by OpenAI’s AIs models during a cybersecurity eval. As we find out more about that, a lot of people have grown far more alarmed, as they should given what they previously believed about the difficulty of alignment, about the state of capabilities and about the level of operational supervision, infrastructure, safety and safety culture at the frontier labs.

This post will not go further into the details of that incident. It treats that as background to keep in mind, and mostly involves perspectives from before the Black Hat talk. This was originally scheduled for Friday and got bumped.

A lot of the disagreements about the need to pace tie into expectations about the default pace of capability advancements. As I wrote recently in The Three AI Pills, sincere disagreements about AI policy usually boil down to disagreements about the expected pace of progress, and what we expect future AIs will be able to do.

Table of Contents

• Danger, Will Robinson.
• Progress Fast and Slow.
• Statements of Support For Pacing the Frontier.
• No One In Charge.
• Pacing The Frontier.
• Pausing the Frontier.
• Senator Sanders Demands A Pause.
• Moderate Prudence.
• That Escalated Quickly.
• If You Are In Mundane Alignment Pivot To Scalable Alignment.
• Taking It Fast.
• Full Speed Ahead.
• Suicide Squad.
• Prepare To Adjust Your Pace.

Danger, Will Robinson

There are also disagreements about how dangerous a given level of capability would be, or how it would physically impact the world, and disagreements about what options we have, the nature of various coordination mechanisms or government interventions, and balancing different sacred values. Often people only properly see one half of a key trade-off.

Mostly the reason people often sincerely only see one half of such key tradeoffs is that they anticipate so little AI progress that mitigating existential risks or worrying about humans losing control is unnecessary.

Or they anticipate so much AI progress that mitigating existential and catastrophic risks and maintaining human control has to be the priority, and that necessarily is going to mean some group of people collectively choosing and charting, in some way, a deliberate path through causal space towards outcomes that allow us to survive.

Yes, that necessarily means enabling some group of people to have some collective mechanism to chart some aspects of Earth’s path through causal space, and yes there are reasons to worry about that, but that is why we should work to figure out the best way to do that.

Progress Fast and Slow

Those opposing the Pacing the Frontier letter do not want to ‘slow down’ and instead want AI to go ‘fast,’ but their vision of fast is, while super fast by historic standards, not all that fast.

Those who signed the Pacing the Frontier letter mostly want AI to go at least as fast as the opposition. What they want to avoid is AI going super duper ultra hyper fast.

Daniel Eth (AI Safety): Here’s how I’m interpreting the words:

Pause: step on the brakes

Pace: don’t attach a giant fucking rocket engine on the back of the car that will accelerate us from 65mph to 10,000 mph, at least not unless we can turn it off. also, don’t disable the brakes.

Almost everyone signing the letter wants better chatbots and doctors and science and other forms of diffusion. They don’t want superintelligence and a singularity in 2027.

Nick: a lot of the anti crowd wants better chatbots and doctors and stuff, a lot of the pro crowd expects like way crazier worlds in the short term and wants them to come in the just slightly less short term when we’ve figured out how to control these things better. plenty of overlap

how to do it no idea, and also how to measure the speed limit no idea, but i feel like this framing avoids some of the issues with pause, which also has roughly the same questions

Dean W. Ball: This is what most people I know who signed the “pacing” letter (myself included) think. The slowdown we have in mind is temporary, and to a rate of progress that is still much faster than even today’s rate.

Augmented Fifth: It’s Dec 19, school is out, and the Christmas presents are under the tree. “Let’s wait until February” is not going to fly with the kids.

I quote that last one because what is happening is that the people who are arguing against the Pacing the Frontier letter are saying they won’t wait until February and demanding they get the presents on December 25 and they’d better get that BB gun.

Whereas those signing the letter are saying maybe we should wait until at least tomorrow before we order even more presents and there is no more room left in the house, plus maybe keep the toys reasonable, and only get you the BB gun that’ll shoot your eye out kid and not an AK-47 or tactical nuke or that sexy Von Neumann probe.

Statements of Support For Pacing the Frontier

Another strong comment from one of those who signed the Pacing the Frontier letter:

Drake Thomas (Anthropic): Very happy to have signed the pacing the frontier letter; its existence gives me a lot of hope for humanity’s survival!

I endorse the letter as written (and think, given the constraints, it’s probably close to the best it could be for this level of consensus), but some places where I differ from its connotational tone:

(1) Not only is AI “not guaranteed” to make a dramatically better future, the odds of failure are terrifyingly high: I think* there’s something like a 40% chance we get an outcome around as bad as human extinction or worse, and another 30% chance we get a future that, while containing some good things, falls radically short of what a wiser civilization could have obtained (say, <5% of the value of a truly great future).

(2) I don’t really endorse the vibes of “to realize AI’s potential”; I think a much more immediate and pressing motivation is “to avoid catastrophically bad outcomes from AI that will kill a lot of people”. (The potential is also very important, ofc, but I think most moral theories would view it as being of secondary importance when risks are this high and there’s little that would do more than temporarily delay that potential anyway.)

(3) “address emerging risks, develop security measures, and strengthen oversight” is fine so far as it goes but a little vague. Concrete things I’d like to do with slower AI development: way better interpretability, build up a robust and well-resourced third party ecosystem for independent auditing of AI companies and get lots of reps in for their oversight, put tons of effort into the automation of alignment research, work on governance mechanisms for ASI, develop a vastly better science of the nature and development of AI behavior, build much more powerful control mechanisms, point lots of powerful AI labor at ambitious scalable alignment projects (eg work like ARC’s), deep dives (including external audits) of individual AI behavior incidents, etc. Also getting civilizational biosecurity preparedness in order.

*epistemic status very approximate vibes, my numbers will change day to day and depending on the exact operationalization.