三星 LPDDR5X-PIM 存内计算亮相 Hot Chips 2026

Hot Chips 2026: Samsung’s Processing-in-Memory (PIM)

In-memory compute with LPDDR5X

Chester Lam

Aug 29, 2026

25

2

1

Share

In-memory compute has been an attractive proposition for many years because compute within a memory chip can exploit its higher internal bandwidth. Additionally, in-memory compute avoids the long latency path between DRAM and traditional compute cores. At Hot Chips 2026, Samsung discusses their continued pursuit of in-memory compute with their PIM (Processing-in-Memory) push. They’re implementing MAC units within LPDDR5X chips, while preserving the chip’s ability to interface with a standard memory controller.

DRAM chips are internally divided into banks, each with their own read and write logic. During a normal DRAM access, the memory controller selects a bank, activates a row within it, and then accesses data via column access strobe (CAS) commands. Bandwidth is limited by the chip’s external DRAM interface. Even if the memory controller could activate all of the banks simultaneously, it wouldn’t be able to get its hands the full bandwidth available across all the banks.

Samsung’s LPDDR5X-PIM is like a normal LPDDR5X-9600 chip with 16 banks, but places a PIM (Processing-in-Memory) block at each bank. These PIM blocks access their attached DRAM bank without being constrained by the chip’s external bus. Together, they can utilize the chip’s internal bandwidth across all 16 banks, which comes out to 614 GB/s. For comparison, regular DRAM accesses can hit two banks in parallel and max out at 76.8 GB/s.

PIM blocks internally consist of a MAC tree with surrounding register files and control logic. A 1024-bit instruction register file holds up to 64 16-bit instructions. A 4 kbit source register file is meant for activation vectors, and supplies one source operand for the MAC array. Samsung expects software to load model weights into DRAM, so the attached DRAM block supplies the second operand. Model weights can be scaled before the MAC computation, with scale factors coming from a 2 kbit scale register.

The PIM block’s MAC array supports a variety of low precision formats. Numbers from Samsung’s presentation suggest each PIM block’s MAC array can sustain four INT8 or FP8 MAC operations per data clock, or eight per cycle when not counting the double data rate. Throughput doubles for 4-bit input weights, bringing package-wide compute throughput to 2.4 TOPS.

This isn’t a very high figure, but an implementation with many LPDDR5X chips will have higher aggregate throughput. For example, eight LPDDR5X chips together would have 9.6 INT8 TOPS, which just about matches the NPU in Intel’s Meteor Lake. That would also be an expensive setup, because eight 16 GB LPDDR5X chips would correspond to 128 GB of system memory.

Accessing Compute with Standard DDR Commands

One highlight of LPDDR5X-PIM is that it stays within the standard LPDDR5X protocol while exposing compute capabilities that aren’t part of the memory standard. Samsung achieves this by setting aside special row addresses, which act like MMIO addresses of sorts. Each channel has a pair of predefined rows for mode control. Activating one of those rows sets the chip to single-bank mode, while the other sets the chip to multi-bank mode. Single-bank is the regular mode, while multi-bank applies commands across all 16 banks to exploit the chip’s internal bandwidth.

Special per-bank rows change how read and write commands behave. Activating one of these special rows makes read and write commands access PIM registers instead of regular DRAM bank contents (PIM Registers Activated mode). Samsung envisions a ML use case where software loads model weights into DRAM while the chip is in normal single-bank mode. Then, software switches into multi-bank mode and enters PIM Registers Activated mode. This lets code write activation values into PIM source registers, set scale factors in PIM scale registers, and specify an operation that’s filled into PIM instruction registers.

Because the chip is in multi-bank mode, each PIM register write gets broadcast across all 16 banks. PIM compute therefore works like a very constrained SIMD processor, where the operation, scale factor, and one source operand are the same across all banks. Samsung does allow writing PIM registers in single-bank mode, but that functionality is meant for debugging purposes. Each DRAM packet is 256 bits (BL=16) Filling each source register takes 16 write commands. Doing that one bank at a time across each of the 16 banks would mean 256 write commands, turning host to PIM register write bandwidth into the limiting factor.

After priming PIM registers, software switches back into multi-bank mode and issues read commands. Instead of reading DRAM contents, these read commands initiate computations and get results accumulated into PIM vector register files. Then, write commands tell PIM blocks to write VRF contents back into the DRAM banks.

PIM has to handle reordering that a normal memory controller might carry out. When code sets up PIM by activating the bank, PIM conventionally sets up its instruction register files so that instructions sequentially access each source register element. For instance, the first instruction would reference the first source register element, the second instruction would reference the second source register element, and so on. However, that falls apart if the memory controller reorders accesses. Samsung gets around this with an Address Align Mode (AAM), which makes each instruction infer its source register index from the column address being accessed.

When the host finishes using in-memory compute and wants to read results, it switches the DRAM chip back into single-bank mode. Then, regular DRAM reads and writes will start accessing DRAM contents as normal.

Software Headaches?

Samsung internally achieved huge performance gains when taking advantage of LPDDR5X-PIM, compared to using standard LPDDR5X. The chip’s ability to operate with a standard memory controller is impressive, and Samsung has been very creative in how they approached the problem.

Repurposing standard DRAM commands should simplify hardware, but software challenges look steep. Because PIM modes change the meaning of DRAM access commands, software can’t use PIM and carry out regular memory accesses at the same time. That applies even across threads, because memory controllers and DRAM chips are oblivious to what thread an access is for. If a non-PIM thread reads from memory while another is using PIM, the first thread could cause an unintended computation and get incorrect results into the PIM VRFs. A write from the non-PIM thread could cause PIM blocks to write VRF data back to the wrong address.

Samsung deals with this by having the host isolate a PIM region in memory. I can’t think of an easy way to do this in a typical system without compromising memory bandwidth and PIM performance. Hardware normally interleaves addresses across channels, which lets common access patterns naturally utilize bandwidth across those channels. PIM uses per-channel rows to control single/multi-bank mode changes, so dropping interleaving and designating memory channels as PIM-only would be the only reasonable way to create a PIM region. Then, non-PIM applications wouldn’t be able to take advantage of bandwidth from channels reserved for PIM. PIM code would miss out on bandwidth and compute from non-PIM channels. The latter could be a significant issue because per-chip compute throughput isn’t that high.