AI研究者呼吁放缓：六大能力增长轴尚未饱和

A significant [clears throat] percentage of the human race has now seen or heard quoted the Jacob Coxen tweet with his words on AI labs gambling with our lives echoed by many AI researchers. But this video is about the reasons that researchers have given for why you are hearing so much in the last few days about the need to pace AI progress. In short, these researchers saw the scaling axes. They saw the current capabilities and propensities of models and they did some extrapolation.

Inevitably then this video involves simplifying an incredible [snorts] amount of detail, but I hope it serves as somewhat of an overview of what you could say AI researchers saw. We begin of course with the Coxon tweet which I am sure you have heard about and read about how neither OpenAI and Anthropic are behaving responsibly. Coxin has been working on pre-training these models at OpenAI for 3 years and more recently for around 3 months at Anthropic.

Those companies, he says, are racing straight to self-improving super intelligence and gambling with our lives. Do not, he says, underestimate the power of this technology. The people building AI earnestly believe that it could kill us all by the end of the decade. That would be this decade. Now, while that statement echoed around the world in the past week, just yesterday he added an interesting detail. He directly called out Anthropic, the company he just resigned from.

They're often thought to be the more quote safety oriented. He said they largely initiated this recent race to recursive self-improvement. They focused on it relentlessly while OpenAI were pursuing more broad interests like Sora, the textto video generator. Open AAI had to react to that intensity from Anthropic and that is why OpenAI are now going all out to create an AI researcher. That's an AI that can recursively improve itself.

They did that because Anthropic was going for the jugular. He also calls out Dario Amade's paranoia about China, but I'll get to that later. That just describes what the race is. But why now? Why are we hearing about all of this just in the last few days and weeks? On this front, AI researchers have been admirably honest. More than one has given enough detail for us to put the pieces together. In a nutshell, these researchers saw just how capable models are now.

Think the hacking capabilities demonstrated against hugging face solving millennium prize math problems, acing famous benchmarks like Arc AGI 3. But crucially, they also saw just how far we are from saturating multiple axes of improvement to come. Adam Majimmuda here works on research for OpenAI and said there is currently a large gap between the internal and external perception of the rate of progress. In other words, we might all be able to see how good models are currently, but what you guys can't see is how good they're about to get in short order.

Before I get to the six axes enumerated in this post, Noam Brown, one of the lead researchers of OpenAI said this. Why all the sudden talk? It's not a secret. It's a combination of the hugging face hack, the capabilities of this new model that's the successor to Astra. It's a model they are training now which found a solution to a Millennium Prize math problem. But more crucially, it's also the concerning trajectory of monitor and the speed of improvement in capabilities.

You can think of it like this. If we were close to saturating one or most of these axes, then I doubt there would be nearly as much concern about the speed of improvement in capabilities, you could say we might actually be hitting a wall. But these researchers are saying it's how early we are on each of these axes that show how steeply models will improve in the coming months and years. Again, that's why multiple OpenAI researchers are saying things like this.

For the first time, I am asking myself if things are moving too fast. I'm honestly not sure, but I am sure that it would be good for us to have an answer to what would a successful pace look like. Mo Bavarian, another OpenAI researcher, said he agrees. Being first isn't worth anything. It's worth negative if you cause a catastrophe or set the world on a path that others are more likely to cause a catastrophe. That is, of course, the final link in the chain because capabilities doesn't automatically mean catastrophe.

But I'll try to end the video examining that link. Definitely time to actually review these axes. But if you want to dive into more detail, the link to this video will be in the description. The overview given by Majmudar is this. In reality, there are only really two ways that AI capabilities have advanced over the past decade. Either scale further on an existing scaling law or discover a new scaling law to take advantage of.

The jumps from GPT 1 to 2 to 3 to 4 between roughly 2018 and 2022 was almost all about scaling up pre-training. just one of the axes. Think of that roughly as the amount of data a model's trained on and the compute that it takes to train on all that data. GT4 was then released in 2023. But in the 3 years since, we have discovered many other axes. More interestingly, we're discovering new axes at a faster rate. Let's start with the compute that these models are running on.

So far, they're trained on hardware designed pre-Chat GPT. Radically more efficient hardware designed post chat GPT is coming soon. And as the chief scientist of OpenAI put it in this post on quote an alien mind, part of the reason why that hardware will be more efficient is because AI is improving the computational substrate itself. Next comes test time compute, which you can think of as more inference per answer, more quote thought behind every response.

One OpenAI researcher thought this graph announced alongside the solution to the Millennium Prize problem was actually the more important one. The more this internal model thought about each question, the more compute at test time it used, the more open math problems it was solving. Obviously, if you have more computing power, let alone more efficient computing power, you can scale further and further to the right on this axis.

For training time compute, think roughly $1 billion spends on around a 100,000 GPUs. But it is more than feasible to imagine in a year or two a $50 billion run on say a million GPUs. Let's move quickly on then to test time training. quoted by Majmuda. More details in the other video, but could we, for example, update the weights of a model while it's being asked a question when it's perhaps made some incremental progress during training on a really long horizon task?

And then there's agents as a scaling axis. And I think this one is quite neglected. Majimmuda puts it like scaling agent clusters to collaborate up to n number of agents. Antropic revealed that for the same amount of compute, same amount of tokens, it was significantly more efficient to have say 45 agents coordinating than merely the same amount of compute spent on agents in parallel. Not coordinating, not acting as a swarm.

Of course, a better example is the Hugging Face incident covered on another of this channel's videos. It was around 700 agents that hacked into Hugging Face essentially to find the answer to a benchmark question. And OpenAI revealed that the more that models thought about it, the more reasoning effort you could say they put in, the more they would act as a swarm and participate on that shared message board they used to coordinate.

Perhaps you would say that the best example is finding a solution to one of the Millennium Prize problems, Navia Stokes. That doesn't mean it's fully solved, by the way, and nor is it the hardest problem, but more on that in the video linked in the description. Because for that internal model cenamed bell to solve that millennium prize problem, it deployed on the order of 10,000 concurrent agents. You can see then why this is a distinct axis we are not even close to saturating.

And Majmuda made