PyTorch Conference 2026：编译器、内核与优化技术详解

Open Research, Tooling & Optimization at PyTorch Conference North America 2026

推荐理由

PyTorch 底层优化干货密集，涵盖编译加速、异构内核开发与自动调优等硬核工程实践，适合关注框架性能与系统架构的开发者深入研读。

TL;DR

TL;DR（太长不看）

Taking place October 20 to 21 in San Jose, California, PyTorch Conference North America 2026 highlights open research, tooling, and performance optimization across compiler architecture, cross-hardware kernel domain-specific languages, exascale distributed training, and low-precision quantization. Read this blog to learn more.

2026年10月20日至21日，加州圣何塞将举办PyTorch北美大会。本次大会聚焦编译器架构、跨硬件内核领域特定语言（DSL）、艾克斯级（exascale）分布式训练以及低精度量化等方面的开放研究、工具链与性能优化。阅读本博客了解更多详情。

Introduction

引言

Every year, the beating heart of PyTorch Conference is the work that happens underneath the flashy headlines- the compilers, kernels, autotuners, distributed runtimes, profilers, and open-source libraries that make the rest of the ecosystem possible. At PyTorch Conference North America 2026, taking place October 20–21 in San Jose, California, this theme – Open Research, Tooling & Optimization – runs through dozens of sessions across the Core PyTorch, Kernel Engineering, Training, and Inference tracks.

每年，PyTorch大会的核心都在于那些隐藏在耀眼头条背后的工作——正是编译器、内核、自动调优器、分布式运行时、分析器以及开源库构成了整个生态系统的基础。在2026年10月20日至21日于加州圣何塞举行的PyTorch北美大会上，“开放研究、工具链与优化”这一主题贯穿了核心PyTorch、内核工程、训练和推理等多个轨道的数十场会议。

This is where you’ll find the engineers rewriting torch.compile’s internals for speed, the teams building DSLs (Helion, CuteDSL, FlyDSL) that let a single kernel target NVIDIA, AMD, Intel, and TPU silicon, the researchers pushing quantization down to FP4 and NVFP4 without losing accuracy, and the maintainers open-sourcing tools like TinyTorch, Pyrefly, LMCache, and OpenEnv so the whole community can build on their work. It’s also where PyTorch’s own maintainers are candidly discussing how AI agents are reshaping how the framework itself gets built, reviewed, and released.

在这里，你将看到工程师们为提升速度而重写torch.compile的内部机制；看到构建DSL（Helion、CuteDSL、FlyDSL）的团队，让单个内核能够适配NVIDIA、AMD、Intel和TPU等硅片；看到研究人员在不损失精度的前提下将量化推进至FP4和NVFP4；看到TinyTorch、Pyrefly、LMCache和OpenEnv等工具的维护者将其开源，以便整个社区能在此基础上构建。同时，PyTorch自身的维护者也在坦诚地探讨AI代理如何重塑框架本身的构建、审查和发布流程。

In this blog we have picked out the sessions that fall into this theme – open source tooling, systems research, and hard-won performance optimization.

在本篇博客中，我们精选了符合这一主题——开源工具、系统研究与来之不易的性能优化——的会议环节。

View the full conference schedule

查看完整会议日程

注册参加2026 PyTorch北美大会

Compilers, Graph Capture & Dynamic Shapes

编译器、图捕获与动态形状

Everything to do with torch.compile, Dynamo, and the machinery that turns eager PyTorch code into a graph format that can be optimized at runtime, or used to generate artifacts for later execution

所有与torch.compile、Dynamo相关的内容，以及将急切模式（eager）下的PyTorch代码转换为可在运行时优化的图格式，或用于生成后续执行工件的底层机制。

Beyond Size and Stride: Unleash Performance with Device-Aware Tensor Layouts

超越大小与步长：利用设备感知张量布局释放性能

Olivier Tardieu; Matthew Arnold (IBM)

Tue 20 Oct, 16:20–16:30, LL20AB

10月20日周二，16:20–16:30，LL20AB

A lightweight tensor layout extension that lets torch.compile (Inductor) automatically adapt tiling and NUMA-aware placement to the target device, without changing existing PyTorch model code.

一种轻量级的张量布局扩展，使torch.compile（Inductor）能够自动适应目标设备的分块策略和NUMA感知放置，而无需更改现有的PyTorch模型代码。

Nested Graph Breaks: Reducing the Cost of Graph Breaks in torch.compile

嵌套图断裂：降低torch.compile中图断裂的成本

William Wen (Meta)

Tue 20 Oct, 16:35–16:45, LL20AB

10月20日周二，16:35–16:45，LL20AB

New Dynamo support reduces the cost of a graph break nested O(N) layers deep from O(N) duplicate breaks and O(N²) frame traces down to O(1) duplicate breaks and O(N) frame traces – yielding larger captured graphs, fewer breaks, and faster, more debuggable compilation.

新的 Dynamo 支持将嵌套深度为 O(N) 层的图断裂成本，从 O(N) 次重复断裂和 O(N²) 帧追踪降低至 O(1) 次重复断裂和 O(N) 帧追踪——从而生成更大的捕获图、更少的断裂点，以及更快、更易调试的编译过程。

Static Tensor Shape Checking for PyTorch with Pyrefly

使用 Pyrefly 对 PyTorch 进行静态张量形状检查

Steven Troxler; Avik Chaudhuri (Meta)

Tue 20 Oct, 17:30–17:55, LL20AB

10月20日周二，17:30–17:55，LL20AB

Shape mismatches cause roughly 45% of deep learning program failures and drive slow torch.compile recompilations. Pyrefly brings static, near-instant tensor shape checking to the type checker itself, evaluated across 28 real LLM, vision, recommender, and RL models.

形状不匹配导致了约 45% 的深度学习程序失败，并导致缓慢的 torch.compile 重新编译。Pyrefly 将静态、近乎即时的张量形状检查引入类型检查器本身，并在 28 个真实的 LLM、视觉、推荐系统和强化学习模型中进行了评估。

Parametrized Dynamic Shape CUDA Graphs

参数化动态形状 CUDA 图

Elias Ellison (Meta); Daniel Galvez (NVIDIA)

Wed 21 Oct, 11:45–12:10, LL21ABC

10月21日周三，11:45–12:10，LL21ABC

Building on parametrized CUDA Graphs and torch.compile’s symbolic tracing and guard infrastructure, this work captures and re-parametrizes a single CUDA Graph across dynamic shapes, eliminating the need for whole-model rewrites, padding, or re-recording across every possible shape.

基于参数化 CUDA 图和 torch.compile 的符号追踪及守卫基础设施，这项工作跨动态形状捕获并重新参数化单个 CUDA 图，消除了对整个模型进行重写、填充或针对每种可能形状重新录制的需求。

From Backed to Unbacked: Sound, Predictable, and Controllable Dynamic Shapes in PyTorch

从有支撑到无支撑：PyTorch 中健全、可预测且可控的动态形状

Laith Sakka (Meta)

Wed 21 Oct, 16:20–16:45, LL21ABC

10月21日周三，16:20–16:45，LL21ABC

This session explains why explicit graph-capture workflows (vLLM, export, pre-compilation) need unbacked dynamic shapes, which disallow implicit guards and force the compiler to prove general validity.

本环节解释了为什么显式的图捕获工作流（vLLM、导出、预编译）需要无支撑的动态形状，这些形状不允许隐式守卫，并迫使编译器证明其普遍有效性。

Speeding Up torch.compile: A New FakeTensor

加速 torch.compile：一种新的 FakeTensor

Angel Li (Meta)

Wed 21 Oct, 16:55–17:05, LL21ABC

10月21日周三，16:55–17:05，LL21ABC

FakeTensor propagation eats roughly 20% of Dynamo’s tracing time. This talk introduces a new C++ FakeTensor implementation that delivers a 30x speedup on operations like aten.mm, meaningfully cutting torch.compile’s notorious cold-start compilation time.

FakeTensor 传播消耗了 Dynamo 追踪时间的大约 20%。本次演讲介绍了一种新的 C++ FakeTensor 实现，在 aten.mm 等操作上实现了 30 倍的速度提升，显著缩短了 torch.compile 臭名昭著的冷启动编译时间。

Lightweight FX Tracing in PyTorch

PyTorch 中的轻量级 FX 追踪

Richard Zou; Yidi Wu (Meta)

Wed 21 Oct, 17:10–17:20, LL21ABC

10月21日周三，17:10–17:20，LL21ABC

A new FX tracer based on make_fx, targeting functionally pure PyTorch code instead of trying to capture full Python semantics like Dynamo. This means trading flexibility for a much simpler, more learnable, JAX-style tracing programming model for users who need a full graph.

一种基于 make_fx 的新 FX 追踪器，旨在针对函数式纯 PyTorch 代码，而不是像 Dynamo 那样试图捕获完整的 Python 语义。这意味着为了换取更简单、更易学、类似 JAX 风格的追踪编程模型，牺牲了一定的灵活性，适用于需要完整图的开发者。

Unlocking the Full Potential of TorchDynamo: Accelerating, Comparing, and Debugging ML Systems

释放 TorchDynamo 的全部潜力：加速、比较与调试 ML 系统

Yi Pan (UC Berkeley); Megan Frisella (University of Washington); Stephanie Wang (Paul Allen School, University of Washington)

Wed 21 Oct, 17:30–17:55, LL21ABC

10月21日周三，17:30–17:55，LL21ABC

Research using TorchDynamo’s graph-interception capability well beyond torch.compile itself. DynaFlow and Piper for transparent parallelism acceleration, Magneton for automatically comparing equivalent operations across competing systems, and numerical debugging via matched-subgraph tensor comparison.

利用 TorchDynamo 的图拦截能力进行远超 torch.compile 本身的研究。包括用于透明并行加速的 DynaFlow 和 Piper，用于自动比较竞争系统中等效操作的 Magneton，以及通过匹配子图张量比较进行的数值调试。

Kernel Engineering & Domain-Specific Languages

内核工程与领域特定语言

The DSLs, autotuners, and hand- and agent-written kernels that squeeze more performance out of every GPU cycle, spanning Triton, Helion, CUTLASS, FlyDSL, and the agentic pipelines now writing and optimizing kernels themselves.

DSL、自动调优器以及由人工和智能体编写的内核，旨在榨取每个 GPU 周期的更多性能，涵盖 Triton、Helion、CUTLASS、FlyDSL，以及现在正在自行编写和优化内核的智能体流水线。

Extending TorchInductor with FlyDSL: A New MLIR-Native Backend for High-Performance GEMMs

将 FlyDSL 扩展至 TorchInductor：一种面向高性能 GEMM 的新 MLIR 原生后端

Liz Li (AMD)

Tue 20 Oct, 11:10–11:35, 210BF

10月20日周二，11:10–11:35，210BF

AMD’s FlyDSL, a Python-native, MLIR-based GPU kernel DSL, gets integrated into TorchInductor’s GEMM compilation pipeline. The talk covers how FlyDSL plugs into autotuning, coexists with existing backends, and delivers measured speedups over Triton on AMD Instinct GPUs.

AMD 的 FlyDSL 是一种基于 Python 原生、MLIR 的 GPU 内核 DSL，现已集成到 TorchInductor 的 GEMM 编译流水线中。本次演讲介绍 FlyDSL 如何接入自动调优、如何与现有后端共存，并在 AMD Instinct GPU 上实现相对于 Triton 的可测量加速。

Helion: CuteDSL and TPU Backends for Heterogeneous Hardware, and Why It Suits Agents

Helion：面向异构硬件的 CuteDSL 和 TPU 后端，及其为何适合智能体

Oguz Ulgen, Dunfan Lu, Jason Ansel (Meta)

Tue 20 Oct, 11:45–12:10, 210BF

10月20日周二，11:45–12:10，210BF

Helion, PyTorch’s high-level kernel-authoring DSL, gains two new compiler backends – CuteDSL for recent NVIDIA GPUs and Pallas for TPUs – letting one kernel source target very different hardware. The second half explores why Helion’s abstraction level is particularly well suited to LLM-agent-generated kernel code.

PyTorch 的高级内核编写 DSL Helion 新增了两个编译器后端——针对近期 NVIDIA GPU 的 CuteDSL 和针对 TPU 的 Pallas——使得单一内核源码能够 targeting 截然不同的硬件。后半部分探讨为何 Helion 的抽象层级特别适合由 LLM 智能体生成的内核代码。

Practical GPU Programming with Triton for PyTorch Developers

面向 PyTorch 开发者的 Triton 实用 GPU 编程

Suman Debnath, JanakiRam Goteti (Crusoe AI)

Tue 20 Oct, 11:45–12:10, LL20CD

10月20日周二，11:45–12:10，LL20CD

A friendly, from-scratch introduction to writing GPU kernels in Triton – no CUDA or C++ required. Building from a simple vector-add up to a small matrix multiplication, with an emphasis on reading and understanding what PyTorch already generates for you.

从零开始使用 Triton 编写 GPU 内核的友好入门指南——无需 CUDA 或 C++。从简单的向量加法构建至小型矩阵乘法，重点在于阅读和理解 PyTorch 已为你生成的内容。

High-Velocity GPU Kernel Authoring with CUTLASS Python

使用 CUTLASS Python 实现高速 GPU 内核编写

Michael Goldfarb, Guray Ozen (NVIDIA)

Tue 20 Oct, 12:20–12:45, 210BF

10月20日周二，12:20–12:45，210BF

New Python-first capabilities in CUTLASS’s CuTe DSL – high-level building blocks, low-level hardware-instruction primitives, and a zero-cost Resource and Task Scheduler – that reduce boilerplate for kernel authors while keeping the DSL’s zero-cost abstractions intact.

CUTLASS 的 CuTe DSL 中新推出的 Python 优先功能——高级构建块、低级硬件指令原语，以及零开销的资源与任务调度器——在保持 DSL 零开销抽象完整的同时，减少内核编写者的样板代码。

Smarter Autotuning for Kernels: From Bayesian Optimization to LLM-Guided Search in Helion DSL

更智能的内核自动调优：从贝叶斯优化到 Helion DSL 中的 LLM 引导搜索

Jongsok Choi; Ethan Che (Meta)

Tue 20 Oct, 14:15–14:40, 210BF

10月20日周二，14:15–14:40，210BF

Two leaps in Helion’s autotuning: Likelihood-Free Bayesian Optimization (LFBO), which cuts tuning time by 36.5% via an on-the-fly Random Forest filter, and LLM-guided autotuning, which proposes strong kernel configurations in seconds – up to 10x faster tuning on NVIDIA B200.

Helion 自动调优的两大突破：无似然贝叶斯优化（LFBO），通过实时随机森林过滤器将调优时间缩短 36.5%；以及 LLM 引导的自动调优，可在数秒内提出强内核配置——在 NVIDIA B200 上实现高达 10 倍的更快调优。

KernelAgent: Hardware-Guided GPU Kernel Optimization via Multi-Agent Orchestration

KernelAgent：基于多智能体编排的硬件引导 GPU 内核优化

Kaiming Cheng; Laura Wang (Meta)

Tue 20 Oct, 14:15–14:40, LL21DEF

10 月 20 日周二，14:15–14:40，LL21DEF

Building on KernelAgent’s 100% correctness across all KernelBench L1/L2/L3 tasks, this talk adds a hardware-guided optimization layer that feeds GPU performance signals into a closed-loop multi-agent workflow, achieving 1.56x speedup over default torch.compile and 89% of H100 roofline efficiency.

在 KernelAgent 在所有 KernelBench L1/L2/L3 任务中实现 100% 正确性的基础上，本次演讲增加了一个硬件引导的优化层，将 GPU 性能信号反馈到闭环多智能体工作流中，相比默认的 torch.compile 实现了 1.56 倍的速度提升，并达到了 H100 理论峰值效率的 89%。

更进一步：量化金融体系

看懂新闻只是起点——沿量化金融路径，把它变成能交付的工程能力

进入量化体系 →

话题#PyTorch #Conference #torch.compile #Kernel Engineering #Optimization