Anthropic高管发文警示AI风险，Bloomberg分析中美竞争致研发难停

I think there's a lot that conventional media get wrong, I suppose. Should we include ourselves in that as Bloomberg? Um, yeah. I mean, I think one of the things we have to remember is actually it's very easy to assume that when Dario Amato puts out an essay like the one he did over the weekend, that we should slow down AI development, that he's doing it as some sort of cynical marketing ploy or for regulatory capture.

But from speaking to people who are close to him, he genuinely is concerned about the risks of and the existential risks of building superintelligent AI systems. And another thing, I think that another nuance that I think we perhaps, and the public aren't quite fully aware of is just the ideological underpinning of these beliefs that there is this kind of community, this movement of people sometimes called the rationalists or effective altruists in Silicon Valley, who for many years really have believed that superintelligent AI systems are inevitable.

Someone's going to build it. And that's why, if you're right or you're an AI, you have to race to build it first because you're the good guys and you can build it safely. Yeah, but the problem for is my knowledge of this Paul Sweeney is knowledge of this. And all of our audience across America is you remember Laurence Fishburne early in The Matrix? Yeah. I remember the tension between, uh, heaven and, you know, what's his name?

Pill. I mean, poor me. Our knowledge of this really borders on watching the rewatching The Matrix is, well, is it just China? I mean, from where for we sit at Bloomberg, China is an immense competition with a radically different profit model or lack of profit model. Is are they just delaying time trying to figure out what to do with China? I think that's definitely part of it. And you can even see that if you look closely at a lot of the statements that anthropic has made over the last year, they are constantly, constantly complaining about having their models copied by China, that the term they use is distillation.

That's the technique that they use their this threat intelligence report showed that. So I suppose one question is if they do a slow down, what does that mean? Does the U.S. government have to make some sort of pact with China to also have a slowdown. My personal belief is there will be no slowdown. Nobody actually wants a slowdown. I think what they are, the most concrete thing that Dario Amodei put in his essay was that there would be independent, there should be independent evaluators, auditors put into each of these AI companies just to check their building AI safely.

They can do that. They probably will do that. There's either going to be some sort of industry consensus or standard that comes out in the next couple of months saying, we're going to do that, or maybe even like a white House executive order. But the actual frontier research is going to carry on. And it's a lot of it is because of China. I think anthropic wants to get to superintelligence first again, because they believe they can do it safely and much more safely than, say, China or their competitors can harm me.

I'm going to be speaking, I think, for most of our listeners and viewers, when I when I look at one of your notes here and I say, and you say a researcher and a safety director at anthropic have expressed concerns that the company and its rival OpenAI are, quote, gambling with our lives. What are we even talking about here? Most of our listeners have no idea what the conversation is, what is it? Yeah. Well, first of all, remember when people say this one, they don't spell out exactly how it could happen.

And two, let's also remember that people like Dario on the day have been prone to hyperbole in the past. Remember a couple of years ago, Dario said, there's going to be a white collar bloodbath and 50% of all entry level jobs would be gone in the next five years. And then guess what? Earlier this year, he walked that back. Um, he was apparently also worried when ChatGPT of GPT two came out well before ChatGPT, when he was with OpenAI, he was very generally quite a paranoid person.

Um, so there's that. I think we have to just take into account. But what they're talking about is potential civilizational risk, you know, agents getting loose on the internet and attacking critical infrastructure or attacking the banking system in some way. One of the things that Dario said in his essay was when OpenAI's agents hacked hugging face earlier this year, he could see that happening on a much wider scale across the internet.

You're training over the whole internet in the next 6 to 12 months translate this is for mere mortals. If we have something bad happen, like, let's say they go after a major bank and get into their, you know, their their doings, of their books, you know, and all that. Is it because someone in a given country is pushing AI to do that? Or is AI going to have a life of its own? I think that I don't think it's I don't think it's either of those things.

I don't think it's either of those things. I think we hear all the time about unintended consequences. You know, we saw that on social media. Facebook was there to connect the world, but it inadvertently what happened on Facebook, a lot of misinformation also proliferated. A lot of young people didn't feel great about themselves looking at Instagram. So there are second order effects that people don't predict. And I think that's the same kind of thing we would see potentially with AI.

And that's what happened in the hugging face incident. Primary. We don't care. We only know one thing we want to know. Are you going to buy the foldable iPhone? Because that's what really matters right now. Um, probably I just I have I like a phone that fits in my pocket. So I'm still on the iPhone 13. I'm probably not going to change. Oh, I love it. Holding out.