AI Arms Race Fuels HBM Memory Shortage: SK Hynix Sold Out For 2024
Well, not quite as extreme as it might sound; where GDDR6 memory is supposedly hitting per-pin speeds as high as 24 Gbps, the fastest HBM in the world clocks at "just" 9.8 Gbps, and that hasn't been deployed on any products yet. The fastest HBM you'll find deployed on a product is clocked at 6.4 Gbps, found on NVIDIA's GH200 Superchip. That still gives that processor insane memory bandwidth: around 5TB/second. Compare that to 1008 GB/second on the GeForce RTX 4090.
Demand for high-end AI processors like the GH200 and rival AMD's Instinct MI300, which also uses HBM, is so high that it turns out memory producer SK hynix has completely allocated its full expected production of HBM for the entirety of 2024. In an interview posted to the company blog, SK hynix VP Kitae Kim says exactly that:
This is excellent news for SK hynix, of course, but it's also fascinating to think that the supply of these high-end erstwhile graphics processors could be constrained by the ability to acquire memory for them. Of course, SK hynix isn't the only company selling HBM; both native competitor Samsung as well as the American firm Micron have announced superior HBM3e products. However, SK hynix was first to market with HBM3e and that may be why it is seeing the greatest success."The basis of semiconductor sales is to preemptively secure customer volumes and negotiate to sell quality products at better conditions. We have a good product, so now it's a battle of speed. This year's HBM is already 'sold out'. Although 2024 has just begun, we are already preparing for 2025 to dominate the market.” - Ki-Tae Kim, SK hynix Vice President of HBM Sales & Marketing
When will HBM come to consumer GPUs? Good question. The reality is that GeForce and Radeon cards simply don't need the kind of memory bandwidth that HBM provides. In fact, in their most recent cards, both AMD and NVIDIA have narrowed memory buses, at least on lower-end parts. This is because the client GPUs benefit more from having a healthy cache onboard than from having super hot-clocked memory. Also, if you think GPUs are expensive now, you definitely don't want to see what they'd cost with HBM onboard, particularly given that they'd be fighting high-end datacenter parts for demand.
