SanDisk's High Bandwidth Flash Memory Could Equip GPUs With 4TB Of VRAM
The rise of massive AI models has put unprecedented strain on GPU memory, and as a memory company, SanDisk is in a prime position to address it. The company unveiled a new product called High-Bandwidth Flash (HBF), a NAND-based memory solution designed to work alongside the widely used High Bandwidth Memory (HBM) found in AI accelerators.
SanDisk describes HBF as stacking up to 16 layers of BiCS 3D NAND with through-silicon vias (TSVs). A logic layer enables massively-parallel access to the flash memory sub-arrays. This allows for both staggering density—between 8x and 16x the capacity of current HBM solutions—as well as impressive throughput.
Indeed, despite being flash-based, HBF reportedly matches HBM in raw throughput, though SanDisk admits that latency remains significantly higher. However, AI accelerators may be able to mix HBM and HBF on the same bus, leveraging HBM's speed for active computation while offloading less latency-sensitive storage to HBF. HBF isn't exactly a drop-in replacement for HBM—protocol differences mean it's not fully compatible—but it does use the same electrical interface, making integration more feasible.
There's also an obvious caveat in that NAND flash wears out over time, unlike DRAM. SanDisk hasn't shared details on how it plans to mitigate this issue, but it emphasized that HBF is optimized for read-heavy AI inference workloads. That's a big deal, since test-time scaling—where models perform additional computations to refine their outputs—is expected to be the next major advancement in AI performance. The recently buzzworthy DeepSeek-R1 model is an early example of this trend.
SanDisk is positioning HBF as an open standard, and if it delivers as promised, it could be a game-changer. AI workloads are only getting more demanding, and the ability to pack multiple terabytes of high-bandwidth memory into a GPU could reshape AI processing as we know it.
