Anthropic呼吁放缓AI开发，Sam Altman称2027年前不上市

Pumping the brakes. The biggest names. And I call for a safety slowdown. I'm very grateful to see, um, you know, support for this proposal just in the first few hours of having put it out by by Sam Altman. By Elon Musk. What it means for IPOs right now would be an ill advised moment to go public. And is the show of CEO unity entirely altruistic? Another ship struck near the Strait of Hormuz ahead of a key meeting among Gulf states and Iran, as gas and diesel prices rise again overnight.

And the fed chair facing a critical decision on interest rates this week. Could this be the first of several rate hikes to come. Well the Bloomberg this week and we're coming to you live from Bloomberg's world headquarters in New York I'm David Garrett. I'm Christina Ruffini. We're here with Lisa Mateo. It is Sunday, September 13th. Welcome to our listeners and viewers across all our Bloomberg platforms on radio, TV, streaming, bloomberg.com, and of course, the Bloomberg Business app.

Look, we're going to start today with the developments over the last 24 hours where the biggest rivals and I have agreed, at least publicly, to slow down. It started yesterday when anthropic CEO Dario Amodei posted an essay making the case for easing up for a year or two, to quote, greatly reduce the risk that something goes seriously wrong. It's a long essay, 3800 words in total. We're going to read a substantial portion of it because he lays out why he is making this decision, encouraging his colleagues to do so.

Let me just quote a bit from it here. Over the last few months, I become convinced that fully addressing the risks requires even more prudence. Not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models. He then lays out two big concerns. What are there? Yeah. Okay. So he says his first concern is that since roughly this summer, I has been advancing drastically faster, driven primarily by a growing ability to build.

The next generation of this dynamic is called recursive self-improvement. Now he also goes on to say, left unchecked, it could outrun our ability to understand and control these systems and so must be pursued very carefully, if at all. Then he goes into his second concern, which is the open, AI hugging face incident, in which he says a swarm of agents essentially acted as a fanatically devoted collective, in my opinion, a swarm that possessed greater capabilities.

But a similar level of misalignment could have caused catastrophic damage. Now, given accelerating rate of AI capability development, it's my worthy that a 6 to 12 months such a swarm could be capable of taking over the entire internet with a persistent botnet. Now, he says, that scale of damage, it could continue to increase from there if I becomes more powerful without those necessary guardrails. Now we go on to Anderson Cooper because he sat down with Ahmed to talk about the concerns and how they can fork into different paths.

Take a listen. If we take the right paths, then the chance of something going wrong is very low. If we take the wrong paths, then the chance of something going wrong could be could be even higher than that. And so let's focus on our agency and our ability to take the right paths instead of the wrong paths. And I think that the way to take the right paths is for the industry to work together. Now, since days as a leaders of the world's biggest artificial intelligence platforms, they have spoken out.

You have OpenAI chief Sam Altman. He said he agrees with Ahmed that they need to pace the frontier, and that it's been a primary topic of discussion at his company for weeks. Then you have Elon Musk, who runs XAI, and he wrote, Dario is right. Altman spoke with fortune with fortune magazine about the risks of the technology and about what OpenAI is doing to prevent it. Take a listen. I think it is unacceptable to be taking like a 10% chance of killing everybody by the end of the decade.

Obviously, uh, I think the people that are working at these efforts have a huge amount of power and ability to impact, uh, what happens here? We have been taking a number of actions. We've reached this new capability level, um, to ensure that we do not take a risk on behalf of humanity like that. And I don't think anybody else should either. Uh, Altman also told fortune that this year would be an ill advised moment to go public, and that a listing will not take place until 2027.

Now, M&A says he appreciates that other industry leaders are in agreement about the pace of developing the technology. So what is the next step? That's the question. Tell CBS first they need to have third party evaluations. Then they can work together. They set those safety standards about the pace of release. But in order to do that there needs to be involvement of the government, because what we're talking about is the rate of release of products because of whether they're safe.

We need to have that conversation with the government in the room. The government? Yeah. He said anthropic will soon bring these embedded evaluators into their offices, give them desks, badges, company laptops and basically permissions most comparable to what internal risk assessment teams are. So so they can be a part of the company in a way. Lisa, thank you very much. Taking us through both the essay and the reaction to it.

Really appreciate it. We're joined now by two great voices on this Bloomberg News senior editor for technology and strategic industries, Mike Sheppard and Bloomberg Opinion Technology columnist Amy Olson. I'm going to start with you. You wrote on this effectively on the 10th of the week, 10th of September. You said an inflection point for this technology. And you said, if anthropic leaders are truly frightened by existential risk, they should do more to reconcile the strange logic that propels them as you as you sift through this essay.

Do you feel like they were finally doing that? Perhaps a little bit. Um, but I think if they were truly believing that this technology could kill all humanity and there was even a 5% or 10% chance, it probably would make more sense just to stop building frontier. I am particularly recursive self-improvement, which is this approach to AI where the systems actually build and design themselves. And anthropic has been open about this for a long time.

Even engineers from in the company have been quoted in its blog post to saying that whole approached AI worries them. It makes them feel obsolete. Um, so I think in a way, Dario's latest kind of interviews that he's been doing the blog post, it feels like a little bit of a face saving exercise after the, um, Jacob Coxon, the engineer who came out from anthropic and said he was worried about what this could mean for civilizational risk for humans.

Um, so I think probably this is this makes sense in terms of what they themselves are concerned about. Um, but there's always been this very strange commercial that this strange tension between this ideological drive they have around AI safety and the commercial drive they have as well. And we have to remember that all this talk about how powerful their AI is, even dangerous, ultimately does serve their commercial interests to some extent.

It makes them look more powerful. It makes their eye look much more capable, like, uh. And if you're wondering if I may, David, read your intro because I messed up your title twice when we had you on earlier this week. Yes, I absolutely did. Um, but I'm wondering if you can walk us through, uh, what? Harvey was just talking about this recursive self-improvement. I. Why? It could be dangerous. And then kind of what Dario and these guys are promising they're going to do about it.

But one of the big concerns with this recursive self-improvement is that, uh, the AI models themselves will be able to learn and develop, um, i