Intel Says Memory Bandwidth Drives AI Performance More Than Compute
Intel's Srini Krishna penned the "opinion" post, which argues that as AI models continue to grow, processors are increasingly limited by how quickly they can move data rather than how quickly they can perform arithmetic. To that end, Intel is touting support for DDR5 running at up to 8,000 MT/s on "select" Xeon 6 SKUs, along with plans to support 8800 MT/s MRDIMMs on Xeon 6900P systems next year. No word on exactly which SKUs those are, though.
The underlying premise is difficult to argue with to the point of being obvious. AI inference, and to a lesser extent, training, is often constrained by the rate at which model weights can be streamed into compute units. Feeding ever-faster CPUs and GPUs has become at least as important as making them faster in the first place, and Intel is far from alone in reaching this conclusion.
AMD's upcoming EPYC "Venice" processors are expected to increase bandwidth substantially through a combination of an insane 16-channel memory controller as well as support for second-generation MRDIMMs capable of reaching 12800 MT/s. NVIDIA has taken an even more radical approach with its Arm-based Vera CPU, pairing it with LPDDR5X memory via the new SOCAMM2 form factor at up to 9600 MT/s, trading some of the traditional advantages of server DDR for significantly higher bandwidth and better power efficiency.
So the industry is clearly converging on the same idea. Intel's announcement does raise one eyebrow, though. The company promises support for Gen 2 MRDIMMs operating at 8800 MT/s on Xeon 6900P chips at the beginning of 2027. That's great, but as we just noted, second-generation JEDEC MRDIMMs are normally associated with the 12800 MT/s speed class. Intel is almost assuredly intending to support newer-generation modules operating at a lower validated speed, but 8800 MT/s MRDIMMs aren't exactly a brand-new milestone; Granite Rapids already supports first-generation MRDIMMs at that speed, making Intel's framing sound more like a platform-validation update than the arrival of an entirely new class of memory technology.
None of this should come as much of a surprise to anyone who has followed the server market over the past several years. Analysts and OEMs have repeatedly warned that per-core memory bandwidth has been shrinking as each CPU generation packs in more cores without a proportional increase in memory channels. AI workloads simply expose that long-running trend more dramatically than traditional enterprise applications ever did.
The actual story out of Mr. Krishna's blog post is that folks who have already bought into Granite Rapids will be able to upgrade their memory to faster 8000 MT/s RDIMMs later this year. That's a solid upgrade for those users, offering a flat 25% boost in memory bandwidth over the previous DDR5-6400 top spec that could offer a similar benefit to bandwidth-bound workloads. Of course, there's one "small" catch: you have to buy the new RAM, and RAM ain't cheap, but you probably don't need us to tell you about the ongoing RAMpocalypse.
Fortunately for Intel, the company's next-generation Xeon 7 "Diamond Rapids" chips are expected to match EPYC Venice in memory configuration, including support for 16 64-bit memory channels and 12,800 MT/s MRDIMMs. The release window for those is currently just "2027", but the parts aren't expected before the second half of the year, so the real battle next year is going to be Venice vs. Vera. We can't wait to see the fireworks.
