IBM's Artificial Intelligence Unit Packs 23B Transistors And It Installs Just Like A GPU

IBM is showing off its first complete system-on-chip (SoC) designed to run and train deep learning models at a faster and more efficient clip than what general-purpose CPUs can achieve. Big Blue is calling it an Artificial Intelligence Unit (AIU) and one of the notable features is that clients can install it just as they would a standard graphics card.

"It can be plugged into any computer or server with a PCIe slot," IBM Research notes in a blog post announcing the card.

Be that as it may, the AIU is not a graphics accelerator like NVIDIA's GeForce RTX 4090 or Intel's Arc A770 and A750 cards. It's considered an application-specific integrated circuit (ASIC), which essentially means it's built for a specific task rather than general purpose workloads. For example, there are ASICs designed to mine Bitcoin much more efficiently than a GPU can.

In this case, IBM's AIU is designed for deep learning workloads. IBM says it can be programmed to run any type of deep-learning task, whether that's analyzing images on a screen or processing spoken languages and words. It's also said to streamline AI workflows by sending data directly from one compute engine to the next.

Chip layout of IBM's Artificial Intelligence Unit (AIU)

IBM didn't build its AIU from scratch. Instead, it's based on the AI accelerator used in the company's 7-nanometer Telum chip that powers its z16 system. The AIU is a scaled version built on a 5-nanometer node and features a 32-core design with 23 billion transistors.

"The AI cores built into Telum, and now, our first specialized AI chip, are the product of the AI Hardware Center’s aggressive roadmap to grow IBM’s AI-computing firepower. Because of the time and expense involved in training and running deep-learning models, we’ve barely scratched the surface of what AI can deliver, especially for enterprise," IBM Research says.

IBM's goal is to improve AI hardware efficiency by a factor of 2.5X each year, and by 2029 it anticipates running AI models 1000X faster than what was possible just three short years ago. The Artificial Intelligence Unit is an early step in that direction.