OpenAI发布Astra模型，Sam Altman谈安全与AGI进展

The way that OpenAI is framing Astra is basically an early step toward AGI, but I thought we could start a conversation just with basically what's fundamentally different with Astra relative to prior generations of model. It feels, first of all, thank you for having me. It feels like a new step in this process towards models that can really help us create value, do work, uh, discover new science, help us create help, start new companies or create new products.

Using it subjectively feels to me very different than any model before. What's the kind of principal use case that's different this time around? You know, I think we can get into the generations of model that we're very coding focused. I know a lot of the development was very cybersecurity focused, but taking it away from the software engineer, what is the everyday person unlocking AI that they weren't able to do previously?

One thing that I think people will immediately notice is if you have an idea and you want to work interactively with an AI to get, uh, you know, a complex piece of a whole piece of software built. This is the first model to me where I could sort of tell someone, like, just give it a try. There's. There's a good chance it'll work. And, you know, I've watched people make computer games. I watch people do sort of like home sort of DIY electrical engineering projects.

Uh, I've watched people do very complex simulations for some piece of science they're working on. Uh, certainly a lot of the work where you would sort of normally sit down and have to, you know, build a financial model here and then a PowerPoint presentation around it and then figure out how to, like, make a little interactive piece of code to sort of try different simulations. That stuff is it's all so doable now. Um, what I hope will happen is people I think people will be surprised at the beginning, but then as they build up more trust in the model and more like, wow, I can really do this.

Just start throwing harder and harder tasks and more creative ideas at it. And uh, they will find out what the model can really do for them. So the specifics of how Astra's being released. I think it's important so that a version of it with specific guardrails. Um, what was the thinking behind that and what are those specific guardrails, uh, in this early release of it? Yeah. So the the challenge of our industry is that we have these models that are getting incredibly capable and incredibly useful and that people want to use to everything from, you know, make their lives a little easier to starting new companies.

And then, on the other hand, as these models get more capable, um, the risks that we have to mitigate, uh, also become more serious. The models could do more damage if we don't, um, if we don't do a good job at that. And so we have spent, uh, you know, obviously this model took us a little longer to release than we were hoping. I think it'll be worth the wait, but we really wanted to spend the time on the safety and security alignment of this model.

Yes. Um, and I think as people understand the power and impact and capabilities, they will be, they will be happy that we did. Um, we will, uh, we'll have different tiers of cyber access for this model, for cyber in particular. Today we're rolling it out, uh, to Trusted Access Partners. And then in the coming days, assuming everything goes well, we'll roll it out more broadly. Um, you couldn't say. Well, you know, if this model is, you know, has a cyber problem, why let anyone use it for cyber?

And I think it's very important to note, to note that the world is very close to a complete change in the landscape of cyber attacks, and the only way that we see for society to collectively defend itself against this coming wave of models, um, from around the world and from other companies, is to use tools like Astra to rapidly defend against these new kind of cyber threats. Um, so we will have multiple programs, uh, for people with sort of different levels of verification and trust.

But we do think it's quite important that the world use these models to, uh, collectively defend ourselves. What happened prior to release is important, right? You determined that Astra can find and develop zero day exploits without any human intervention, essentially. And so you paused some of the work to strengthen the safeguards. What specifically was it that you saw that made you hit pause the behavior, I suppose, of the model.

So it's worth pointing out that, um, the model that we're launching today is GPT six. Astra has been done training for a while. Um, the model that we recently talked about pausing, uh, as a future model. Um, but with Astra, we we did hit cyber critical. And there that required, under our preparedness framework, a new set of safeguards just to be able to release Astra, um, because of those cyber capabilities you talked about.

I got a lot of questions for you, Sam, about Astra and chain of thought. Um, I don't know if that comes as a surprise to you or not, but as Astra becomes more autonomous for future generations of the model, become more autonomous, can open. I sort of say with confidence that you see and understand what the model is planning before it acts. In other words, you have visibility into the reasoning that basically uses of the model can't see open.

I can for obvious reasons. Could you go into that a little bit? Yeah. So we have talked for a long time about the importance of monitor ability of models. We've also said that we believe in defense in depth and this is only one part of it. There is sandboxing. Uh, maybe most importantly there is alignment. There are a number of things that come together to be able to make the safety guarantees that we want to be able to make with a model.

But monitor ability is an important thing. And we've been talking, I think, for well over a year now about the importance of chain of thought monitoring and how we have made various decisions there that help us preserve chain of thought vulnerability. Um, even if it means we don't maximize the capabilities we could otherwise get. And we think that is important. Um, it's an important component. We also, uh, we want to be clear.

We don't think it's the only component that matters. We live on Bloomberg Television and on Bloomberg Radio, and we're speaking to Sam Altman, the CEO of OpenAI. Um, you know, there's a lot of bad PR in the world around AI at the moment, every day citizen sort of worry about the impact. One way I kind of wanted to put that to you. Is it whether you can say sort of unequivocally there, if Astra or any future generation of all behaved in a way that was dangerous, that you would be able to detect it and then shut it down.

The obvious hard part of that, uh, question is that people have different opinions about what model should be able to do and shouldn't be able to do. Yes, we think there are some things that very clearly model should not be able to do. Uh, there are some clear red lines, and we talk a lot about being able to detect those. But people also have strong opinions about things within the broad bounds of what is possible about how they use AI and or how they want to be able to use AI.

And this is going to be a difficult question for society, because there is not going to be agreement on, you know, what is and isn't acceptable use of I. I think it's very important in the same way that, you know, people do things electricity that I may not always like, but I think everybody has a right to electricity that we say, you know, there going to be some rules for society. You can't go like electrocute somebody else on the street.

But this is an important platform that belongs to all of us and that we need to be able to, to use. So there will be, uh, ongoing conversation, negotiation in society. There will, of course, be some things that we agree on that people just should not do with. I. But there's a lot of edges that we're now going to have