呼吁建立LLM数学理论：从数据到架构全面建模

It's high time to develop a proper mathematical theory of LLMs.

Model everything from data to architectures, to get heuristics plausibly explaining what is actually happening when you train and get answers to questions like what data distribution you need to make models better at a particular skill, architecture/data relation, training time/output and so on.

I'm pretty sure labs have developed internally rudimentary ad hoc methods, but we're nowhere near a proper theory that would give heuristics why certain things make sense.

We are so early.