Micron Samples Industry's Highest Capacity 192GB SOCAMM2 Memory For AI Servers
Micron is laying claim to the largest-capacity SOCAMM2 modules on the planet with its new 192GB product, which it has begun sampling to customers. According to Micron, "unprecedented AI innovation and growth" is fueling the need for increasing amounts of memory in the data center, and its low-power 192GB SOCAMM2 module is optimized for large data center clusters.
"As AI workloads become more complex and demanding, data center servers must achieve increased efficiency, delivering more tokens for every watt of power," said Raj Narasimhan, senior vice president and general manager of Micron’s Cloud Memory Business Unit. "Micron’s proven leadership in low-power DRAM ensures our SOCAMM2 modules provide the data throughput, energy efficiency, capacity and data center-class quality essential to powering the next generation of AI data center servers."
Compared to its first-generation LPDRAM SOCAMM product, the new 192GB SOCAMM2 module delivers 50% more capacity in the same small footprint to reduce time to first token (TTFT) by more than 80% in real-time inference workloads, Micron says. Micron's also touting a 20% improvement in power efficiency. It's also based on the same advanced 1-gamma DRAM process that Micron employed on some DDR5 products earlier this year.
While that may seem like no big deal on the surface, Micron points out that the power savings become "quite significant" at scale, with full-rack AI installations now tapping more than 40 terabytes (TB) of CPU-attached lower-power DRAM main memory.
SOCAMM2 products have a heritages in low-power DRAM that was initially designed for mobile phones (which Micron still produces). To get to the point where it became feasible for large-scale data centers, Micron said it employed "specialized design features and enhanced testing" to transform those earlier memory products into data center-class solutions. Micron also claims its SOCAMM2 modules are better suited for data centers than RDIMMs.
"SOCAMM2 improves power efficiency by more than two-thirds compared with equivalent RDIMMs, while packing its performance into a module one-third the size, optimizing data center footprint and maximizing capacity and bandwidth. SOCAMM’s modular design and innovative stacking technology improves serviceability and aids the design of liquid-cooled servers," Micron says.
The 192GB capacity module is not the only one available, but it is the highest. Micron says it's sampling now in speeds up to 9.6Gbps.
