用AI代理实现新模型在IBM Spyre加速器上的首日适配

TL;DR

The AI model landscape never stops moving, and the software stack that runs those models is always a step behind: even on a mature compilation stack, a new model family often arrives with some novel module that doesn’t lower well, and on new hardware – where a young stack meets a whole ecosystem at once – the gap is far wider. Extending stack enablement to each new family has traditionally been slow, specialist work that delays deployment. In this blog we show how AI agents can facilitate day-one enablement for a new model or for entire ecosystems, by bridging between the model workflows and the compilation stack. We demonstrate this at the harder end of that spectrum: through a small number of AI-written adapters, we run stock HuggingFace Transformers models on IBM’s Spyre AI accelerator, achieving full enablement for thousands of models.

The problem: catching up to an evolving model landscape

The model landscape never stops moving. New architectures and checkpoints appear constantly — each a particular composition of operations, shapes, and numerical ranges — and the software that has to run them is always a step behind. Even on the most mature, widely-deployed stack, a brand-new model family lands with some novel module, some fused attention variant, some numeric range that the existing software doesn’t yet handle cleanly.

New model families added to the HuggingFace Transformers library over time. Each step marks the inclusion of a prominent model family, illustrating the steady, accelerating pace at which new architectures arrive.

Running a model on any device requires a software stack – a compiler, a set of operator lowerings, and a runtime – that maps it onto the hardware’s cores, memory hierarchy, and number formats. That stack is a large, evolving piece of software, and a new architecture can exercise a corner of it that isn’t yet mature. Until that corner is filled in, the model cannot run, or doesn’t run well – and closing such a gap has traditionally taken weeks or months of specialist work.

New hardware is where the gap is at its widest. A new accelerator is not just a chip: it ships with a young stack still growing into the needs of different models. Here it is not one new model meeting a mature stack, but a whole ecosystem of models meeting a stack that is still being built – and that must simultaneously keep pace with new architectures landing on top of it. Enablement can’t wait for the stack to be complete; it should progress in parallel, letting real models run while the platform underneath them matures.

AI accelerator releases. Each dot marks the announcement date for a new chip. New hardware arrives almost as steadily as new models do – and every chip architecture needs a continually evolving software stack.

New hardware is the extreme case, but the challenge is a general one: keeping the software in step with the model landscape, whether the stack is mature or brand new. Below we describe a strategy for bridging that gap, and demonstrate it through a concrete instance – running stock HuggingFace Transformers models on IBM’s Spyre accelerator.

The strategy: adapters as a bridge between models and stack

The strategy we describe is a thin layer of runtime patches – adapters – that allow a stock model to run on a given platform today, without waiting for every underlying gap in the stack to be closed first. We assume that the platform already provides a PyTorch compiler that lowers

core tensor operations in ordinary torch code – including matrix multiplications, elementwise operations, and reductions – onto the target hardware, so most of the model logic runs through it unchanged. When some operation in a model doesn’t yet have a clean path through the stack, the adapter reaches in at runtime and swaps it for an equivalent one that does. The patches change how the model is expressed for the device, but not what it computes: the underlying math is preserved. An adapter isn’t a hand-optimized kernel – it hands the stack a form it can lower well, but performance remains the compiler’s responsibility.

Rather than blocking on the state of the stack, adapters provide a practical bridge across it as it stands today. A useful analogy is a large construction project. Around a building with active construction – whether it is still going up or already standing and being renovated — there is almost always scaffolding somewhere: temporary access ramps and support beams that keep work moving on whatever part isn’t finished yet. No single piece of scaffolding is permanent; each comes down once the structure behind it can stand on its own.

Adapters play the same role. An individual adapter is usually transitional: its job is to carry a model across one specific gap in the stack as it stands today, and it is designed to be removed once that gap is closed — as the platform matures, new optimizations are integrated, more architectures are enabled, and the permanent path opens up underneath it. Not every adapter comes down, though: some bridge a temporary gap the stack will eventually close, while others accommodate a genuine, lasting difference in the target hardware; and where an accelerator harware has a unique architecture, those bridges may stay in place for good.

The adapter layer, however, is permanent in a way no individual adapter is. Because the model landscape never stops moving, there is always some new gap the stack hasn’t caught up to yet — so even as old scaffolding comes down, new scaffolding goes up elsewhere. That layer connects two things evolving in parallel. On one side is a model ecosystem – in our case HuggingFace Transformers, thousands of models with a stable API and an enormous community. On the other is the platform’s hardware-lowering stack – in our case Spyre, a compiler and runtime stack that maps those models onto the accelerator device and is itself under continual development. The adapters sit in between, letting models run on the hardware while the two ends continue to evolve.

What makes this strategy practical at the scale of thousands of models is AI. Historically, enabling each new model family was a slow, specialist effort, requiring a great deal of work for every hardware platform and chip architecture. Coding agents change the economics, turning what used to be a bespoke effort into something that can keep pace with the ecosystem. The rest of this post shows how that plays out – but first we will introduce the concrete platform in which we implement this.

The platform: Spyre and torch-spyre

The hardware in our example is Spyre, IBM’s AI accelerator, built on the AIU (Artificial Intelligence Unit). It is made of small cores connected by a high-bandwidth ring, each with its own local scratchpad memory and arrays of processing elements that carry out the matrix multiplications at the heart of a model. A few things make it distinctive. First, it is dataflow-driven: computation is triggered by data arriving at a compute engine, which reduces the bottleneck of sequential control flow, and keeps the hardware busy on the structured, repetitive math that models are made of. This is in contrast to a conventional GPU, which executes kernels as explicit instruction streams across scheduled groups of threads. Second, Spyre uses reduced-precision number formats designed for inference, which lets it deliver high throughput at low power.