NVIDIA Infuses Grace Hopper Superchip With HBM3e To Supercharge AI Data Center Workloads

hero nvidia gh200 photo
NVIDIA's CEO Jensen Huang donned his signature leather jacket to take the stage at SIGGRAPH 2023 and deliver the opening keynote for today's sessions at the famed rendering conference. Most of his talk today was about generative AI and how it is changing the world, largely powered by his company's GPUs. He did have some hardware news to share, though, including a new release: the Grace Hopper GH200 with HBM3e.

It's a little bit unusual, to tell the truth. The Grace Hopper Superchip isn't a new product in and of itself; NVIDIA's been selling them since earlier this year. It combines a 72-core Grace CPU (based on ARM Neoverse V2 processor cores) and its 500GB of LPDDR5X memory with an H100 Hopper GPU to provide Super Saiyan God levels of compute on a single PCB.

nvidia nighttime rtx racer
NVIDIA also showed off a nighttime scene for RTX Racer. C'mon, release it already!

The original Grace Hopper superchip came with up to 96GB of HBM3 for its Hopper GPU, but the new version instead packs on 141 GB of the latest and greatest HBM3e RAM. 141GB is a weird number, but we expect that it originates from five 32GB stacks (160GB) with ECC enabled, reducing the capacity to 7/8ths of its physical size. That gives a capacity of roughly 141 GB if you round things off with SI units instead of binary (Gibibytes) units. Whatever the case, it is an incredible amount of memory for a GPU.

jensen eight gh200s
Jensen Huang poses with a DGX GH200 board sporting eight Hopper GPUs.

Besides the extra capacity, NVIDIA says that the HBM3e version of the Hopper GPU offers a 50% bandwidth improvement, too. That gives the massive parallel processor some five terabytes per second of memory bandwidth, which is a number that boggles the mind. The H200 GPU and Grace CPU talk to each other over a 900 GB/sec coherent NVLink interconnect, and each Superchip can be linked to additional chips in the same way, allowing them to act fundamentally like a single unit.

nvlink topology

Interestingly, despite the "GH200" name, NVIDIA hasn't talked about any changes to the GPU die itself. Historically, NVIDIA's model numbers categorically denoted changes to the product and architecture, with the rare "2" in the first numeric position indicating a full architectural revision such as with the "Maxwell 2" architecture that underpinned the GeForce 900 series. All of those parts were code-named "GM2xx". It's possible that the Hopper chip itself has seen a silicon revision with this release, but if so, NVIDIA isn't talking about it.

If you're someone whose work involves systems at the exaflop scale, we're very pleased that you're reading HotHardware for your GPU news. However, you'll have to wait before you can get your hands on the new GH200. NVIDIA says that "leading system manufacturers" should have machines sporting the new hardware in Q2 of 2024, with pricing no doubt being in the "if you have to ask..." territory.