Anthropic CEO呼吁放缓AI开发，OpenAI与马斯克表态支持

Now some of the biggest names in I are now calling for a slower pace of development. Anthropic chief executive Dario Amodei has warned safeguards are failing to keep pace, with increasingly powerful models pledging to implement new safety steps, including third party evaluators. Nowadays, the suggestion of a slow down was backed publicly by Elon Musk and OpenAI chief executive Sam Altman. He said the ChatGPT maker will not go public this year as it focuses on addressing safety concerns.

In the U.S., House Democrats plan a caucus tomorrow to discuss AI policy and action. But President Trump has downplayed the growing alarm. We're leading China and AI will most sophisticated country in the world. And frankly, I want to keep it that way because whoever wins, I wins. And we can put guardrails, we can do this and that. But I think you have a lot of negative voices in bringing it up that, uh, we shouldn't be bringing it up.

Well, we're joined by Olson Technology, a columnist at Bloomberg Opinion. She's also the author of supremacy AI ChatGPT and the Race That Will Change the World. Actually one of my favorite books, so I highly recommend everyone go and read it for me. So look, you know, we sat on set and I said, I'm worried because I'm worried about the future. People are anxious. You don't really know what to make of it. What are your impressions?

I mean, there's no consensus. Us. This needs to be slow down. Yeah. And let's remember why. Potentially what really sparked this was in the last few days, even before Dario Armada's essay about slowing down. We had a former anthropic engineer saying, this technology could kill us all. I mean, really dramatic language. And that was agreed with by a current safety director at anthropic. Um, well, first of all, just remember, Dario Ahmed has a tendency towards hyperbole.

A couple of years ago, he talked about, you know, 50% of all white collar jobs being wiped out in the next few years. Um, and then he pulled that back a little bit. And there is a very strong undercurrent, ideological undercurrent, believing in eye safety among people who work at that company. But I think there is a genuine belief among leaders that the most cutting edge technology, particularly AI, that can improve itself and create its own successors, known as recursive self-improvement, which is what the engineer who left was most worried about, that perhaps the risks of that outweigh the potential benefits.

Personally, I think it's good that he has called for a slowdown. And also it's very encouraging that Sam Altman and Elon Musk agree with him. So we've got some industry consensus, perhaps some collaboration on a potential slowdown and pacing, as he calls it. Personally, I think a pause would be even better because pacing or slowing down does also just essentially kick the can down the road. Dario's essay was quite vague in terms of how he thinks that should happen with government coordination.

The evaluators, third party evaluators being inside the company, that's the most concrete thing he suggested, but he didn't really go beyond that. But I mean, that actually happened if the U.S. is not behind it. I think it's difficult. He'd need to have. I mean, that's the other wrinkle in all this is Donald Trump, although, you know, maybe that he's coming to the end of his run. We don't know how much longer that administration in that belief is going to be in place at the white House.

You've got on the other side, you've got Bernie Sanders, who thinks that whoever builds superintelligence should go to jail. Like so there's a real split right now, um, among lawmakers. It's not it's not all Trump's view on things. I think there's a real divergence of opinion. But for me, I mean, in the past, I used to, you know, come to you and say like, oh, I see a good guy, see a bad guy. And you have always said, you know, there's no good or bad guy is because it's all driven by by commercial want.

And this is also beautifully explained in your book. Does that change? I mean, you can say I want to slow it down, but if you're looking at IPO, if you're looking to make money out of it, it's will someone else do it? I think and a lot of actually it's perhaps a little mixed here. There is the commercial pull of the IPO, a potential $2 trillion valuation through an IPO for anthropic, which is incredible. Um, but also, I think these people split away from OpenAI to start their company because they were genuinely concerned about safety as well as opportunists in the moment.

They wanted to capitalize on their expertise and make money. Sure. Um, but I think Dario is so radically transparent. He has this almost confessional approach which would obviously disrupt any potential market. We see markets already are declining because of these. The general worry about that. Um, so at the end of the day, though, it doesn't really matter what the motivation is. You could say what really matters is the outcome.

And the evidence is that these systems can hack third party systems. They can manipulate humans. Um, and there are unintended consequences to giving them direction, particularly when they can do tasks online. Um, so I think that is the thing we need to kind of be concerned about. And that's what regulators need to focus on. Whereas China and this because again, if you know some of the big companies in the US self-regulate even without the US administration on board, but China goes ahead, then it's pointless.

The obvious thing to say here is it makes no sense that China would want to pause, you know, on this, particularly because the Chinese government and companies in China aren't as worried about the existential risk of AI as a Silicon Valley is. That's a very kind of distinct ideology for the tech sector in the US. In China, they're much more worried about social disruption. And what I could already do with deepfakes and manipulating people.

So I think there's a little bit of a kind of a consensus that we need to take, have more of a handle on things. And I don't think it's completely hopeless, this idea of perhaps some kind of US-China pact. We had, you know, there was a little bit of a historic precedent on cyber, um, issues between Obama and a number of years ago, there was like a 12 month pause. Um, so, you know, something could emerge. I don't think it's completely hopeless.