Samsung's zHBM 3D Memory Concept Forecasts 8X Speed Over HBM5
In conventional high-performance computing (HPC) and AI server designs, High Bandwidth Memory (HBM) packages sit alongside the processor on a silicon interposer. While fast, this horizontal layout still forces data to travel across a significant physical distance between the RAM and the GPU or accelerator. zHBM changes the geometry by vertically stacking HBM directly on top of the AI accelerator, hence the "Z-axis" naming. By minimizing the physical distance data has to travel, Samsung’s proposed interface drastically cuts latency and boosts bandwidth. The company also claims that zHBM "reduces thermal resistance by more than half," although it's not clear how this impacts heat dissipation for AI chips that are already exceeding 1kW power draw.
Samsung's concept model projects massive architectural leaps over the upcoming HBM5 baseline using next-generation wafer bonding technology. The company claims up to eight times the bandwidth of HBM5, and more than ten times the density, which means ten times the capacity in the same space. Keep in mind that HBM4 still isn't deployed commercially yet, so these are quite the claims versus a far-future technology.
It's also not really something that most consumers, including DIY PC enthusiasts, will ever likely see. Samsung's zHBM, like current HBM, is purpose-built for enterprise AI data centers and large-scale HPC infrastructure. Samsung further noted that the architecture supports customer-specific designs. While that sounds obvious based on the nature of zHBM, what it really means is that enterprise clients can integrate customized IP into the interlayer between the memory and the AI accelerator to expand capacity or accelerate specific workloads.
Alongside zHBM, Samsung also showcased two major developments on the flash storage front. The first is "zNAND-O", which is a concept high-performance NAND architecture built on Samsung's V-NAND technology, currently in development in four- and eight-layer versions. Samsung says zNAND-O was designed specifically for edge AI environments, and thus prioritizes "spatial efficiency, low latency, and high I/O performance" to help hardware handle "real-time, data-intensive inference tasks" locally.
The other flash storage technology was "V10 BV-NAND", the 10th generation of Samsung's V-NAND, dubbed BV-NAND for "Bonding V-NAND". It apparently features over 400 individual flash layers, as, for the first time in the industry, Samsung applied wafer bonding technology to stack NAND memory cells. This reportedly yields a ~58% increase in storage density alongside tangible gains in read, write, and I/O speeds over the previous V9 generation.
To be clear, of these technologies, only V10 BV-NAND will be coming to the market anytime soon, and in this case "soon" is still "maybe next year." Otherwise, zHBM and zNAND-O are simply forward-looking concept models for future hardware cycles, and may never even actually make it to the product stage. Still, the fact that Samsung is crowing about them this early clearly reflects that scaling AI performance is no longer just about building faster processors. The spatial relationship between compute and memory has become a major bottleneck, and every vendor is pursuing its own solutions. Once the dust settles, the datacenter is going to look very different in a few years than it does even now.
