Samsung's LPDDR5X-PIM at Hot Chips 2026: In-Memory Compute with Standard DRAM Interfaces

At Hot Chips 2026, Samsung presented its continued push into processing-in-memory (PIM) with LPDDR5X-PIM, a design that embeds MAC (multiply-accumulate) units directly into LPDDR5X DRAM chips. The goal is to bypass the traditional bottleneck of moving data between DRAM and compute dies, exploiting the much higher internal bandwidth of the memory chip itself.

In a standard DRAM access, the memory controller activates a bank and reads data via column access strobe commands, limited by the chip's external interface. Even with multiple banks, the external bus caps total bandwidth. Samsung's PIM design places a compute block at each of the 16 banks, allowing them to access their attached DRAM without external bus constraints. This unlocks an internal bandwidth of 614 GB/s per chip, compared to 76.8 GB/s for regular accesses.

The PIM blocks consist of a MAC tree with register files for instructions, source data, and scale factors. They support INT8/FP8 and lower precision formats, achieving up to 2.4 TOPS per package. While modest, eight chips together deliver 9.6 TOPS, comparable to Intel's Meteor Lake NPU, but at a cost of 128 GB of system memory.

A key highlight is compatibility: LPDDR5X-PIM stays within the standard LPDDR5X protocol while exposing extra compute capabilities through reserved row addresses. Special per-bank rows switch the chip between single-bank mode (normal) and multi-bank mode (broadcasting commands across all banks to execute PIM operations). This allows the chip to work with standard memory controllers, simplifying hardware integration.

However, software challenges loom large. Because PIM modes reinterpret DRAM commands, the chip cannot handle regular memory accesses while in PIM mode. This creates concurrency issues: a non-PIM thread reading or writing memory could trigger unintended computations or corrupt PIM register files. Samsung suggests isolating PIM memory regions, but that requires dropping channel interleaving, forcing dedicated channels for PIM only—sacrificing bandwidth and compute utilization for both PIM and non-PIM workloads.

Multitasking remains problematic. Using PIM with multithreading would require locking PIM region accesses, and in a full OS, processes could collide. One workaround is to run PIM code with interrupts disabled and all other threads blocked—a severe limitation for general-purpose use.

Despite these hurdles, Samsung's approach shows creativity in reusing standard DRAM commands. For developers, this underscores the trade-off in PIM: impressive bandwidth gains but significant software complexity. As AI workloads grow, expect more efforts to make in-memory compute practical, whether through better OS support or more flexible hardware.

Samsung's LPDDR5X-PIM is a promising step toward making PIM a reality in mainstream systems, but its adoption will hinge on solving the software ecosystem challenges that currently restrict its applicability.