Google 发布 JAXBench：TPU 内核优化基准

Great technical paper from Google.

Great read on why context beats scale for agents working against unfamiliar APIs.

(bookmark it)

GPU kernel optimization has KernelBench to hillclimb on. TPUs had nothing, and the Pallas DSL is documented thinly enough that models mostly guess. New research from Google, Harvard, and UC Berkeley introduces JAXBench, 50 JAX workloads built from real MaxText architectures like Llama-3.1, DeepSeek-V3, Mixtral, Mamba-2, and AlphaFold2.

Eight operators ship with hand-tuned Pallas kernels from Tokamax, so agent output gets measured against expert work instead of a naive baseline.

With Gemini 3 Flash, conditioning on curated TPU documentation lifts per-sample correctness from 5.8% to 37.3%, solving 48 of 50 benchmarks at a 1.28x geomean speedup. Beam search then pushes it to 1.36x.

Correctness turned out to be a documentation problem and speed turned out to be a search problem.

Paper: https://arxiv.org/abs/2607.20466

Learn to build effective AI agents in our academy: https://academy.dair.ai/