无标准答案也能训练LLM编程：RiVER方法提升代码能力

LLMs can learn better coding behavior from problems with no known answers.

Many real problems do not have a gold solution waiting in a database, especially in optimization, where the best answer may be unknown, expensive, or impossible to certify.

Normal reinforcement learning works well when it can check a clear right answer, but that breaks down when the best answer is unknown.

The paper’s method, called RiVER, lets the model write several programs, runs them on the same hidden tests, and rewards the programs that perform better than the others.

The key trick is that RiVER does not trust raw scores directly, because some test cases naturally produce much bigger numbers and can distort training.

Instead, it ranks programs within each test case, gives extra weight to the best one, and still gives smaller graded feedback to other valid programs.

The authors trained models on 12 AtCoder Heuristic Contest tasks, and RiVER improved both score-based contest performance and normal pass-or-fail coding benchmarks.

Link – arxiv. org/abs/2606.27369

Title: "Reinforcement Learning without Ground-Truth Solutions can Improve LLMs"