Intel's Loihi 2 Processor Gets Even Faster At Computing AI Workloads Like A Human Brain
With the proliferation of artificial intelligence in recent years, the term "neuromorphic" is being used much more often in the tech sector. If you're a native English speaker, you can probably surmise that neuromorphic means something along the lines of "brain-like." Indeed, the buzzword of the day is "neuromorphic processing," and it refers to computers—previously called "cognitive computers"—designed to mimic the function of the human brain.
The reason that's the buzzword of the day is because Intel just announced its second-generation neuromorphic processor, Loihi 2. If you never heard of the original Loihi, you probably aren't involved in bleeding-edge artificial intelligence research. Intel developed Loihi in 2017 and released it to research partners in early 2018. The company says it has drawn upon the last few years of using those chips in building its successor.
The idea of neuromorphic computing isn't completely new. The concept of building a computer that "works like a brain" existed in science fiction in the early part of the last century. The real origins of modern neuromorphic engineering date back to the 1980s, however. Still, it wasn't until 2006 that someone tried to actually create a neuromorphic processor, and it wasn't until IBM's TrueNorth in 2014 that anyone succeeded in scaling out such a chip to a semblance of usable densities.
The first-generation Loihi processor was actually the fifth-generation neuromorphic chip designed by Intel Labs, Intel's research arm. It was initially produced in low quantities on the 14nm node and offered in units as small as a two-chip USB stick to researchers. Eventually, Intel scaled Loihi 1 out to 768-node racks, with each chip containing a trio of low-power Lakefield x86 cores and 128 "neuron cores". Each of those represents 131,072 artificial neurons with 208 bytes of memory apiece, all connected together with some 128 million artificial synapses.
Loihi (and its successor, which we'll get to in a moment) is quite different from a standard Intel CPU. It doesn't have an external memory interface, and it doesn't have a "clock" in the traditional sense. Instead, Intel's neuromorphic processors are designed around "spiking neural networks" (SNNs). That means that instead of the whole chip being active and refreshing on a synchronized clock signal, it's fully asynchronous and event-driven. When a charge accumulates, the neurons send "spikes" along active synapses. These spikes are highly time-based; time is encoded as a part of the data itself. When spikes accumulate in a neuron for a certain period of time and reach a set threshold, the core fires off its own spikes to its connected neurons.
It's a fundamentally different approach to computer processing than what we're used to, but it's really rather similar to how the brain works. That similarity can give neuromorphic computers a tremendous advantage in certain types of workloads, much like—but wholly unrelated to—the way quantum computing can make quick work of otherwise-impossible computations.
If we take Intel and its partners at their word, the first-generation Loihi chips have already proven the potential of this technology. They say that Loihi is 3000x as data-efficient at odor recognition and learning compared to a traditional deep autoencoder, 44x faster than a CPU at combinatorial optimization, and that it offers 150% the performance of GPU inferencing when performing adaptive robotic arm control. Oh, and it does all this while drawing "far less than 1 watt" of power.
Loihi 2, then, builds upon the successes of its predecessor and, if Intel's claims prove true, could be quite a step forward for the machine intelligence space. Loihi 2 is drastically denser than the last-generation part, with up to 1 million neurons per chip. Despite that, the new chip is barely half the die area (31mm²) of the original, thanks to being fabricated on the Intel 4 process. That's right: Loihi 2 will be the first chip manufactured on Intel 4 destined for external use. Intel is quick to admit that while Loihi 2 confirms the viability of its next-generation fabrication process, it's still in the pre-production phase, which means quantities of Loihi 2 will be pretty limited for a while.
It's not just about density, though. Intel's new neuromorphic chip has considerable architectural improvements over the last-generation part, too. The company says that its new baby supports integer magnitudes on the spikes whereas Loihi only supported binary spikes; that allows for greater workload precision. The new chip's neurons also support a handful of microcode instructions that give it more direct programmability than the original part. Plus, it's just faster at doing the same things: depending on what specific part of the circuit we're talking about, Loihi 2 operates between twice and ten-times as fast as the original part. This accelerates everything the chip does in a similar fashion to a clock speed ramp on a regular CPU.
Loihi 2 can also scale out further and more easily than the original. It supports 3D tiling with quadruple the interlink bandwidth of the first-generation part, and it also implements some pretty fancy compression technology to squeeze more performance out of said interlinks. Even though it has slightly less memory per neuron core (192 KB), allocation is flexible on the new part where it was fixed on the original, and Intel says the "effective capacity" is "significantly higher." The three Lakefield x86 cores from the original have undergone mitosis and are now six of the same cores, programmed with conventional C or Python code and used for encoding and decoding data across the neuromorphic domain.
Intel's other big reveal for the day is the all-new Lava open-source neuromorphic computing framework. One of the big challenges that Intel and its research partners kept butting into with the original Loihi chips was the lack of software that could work with the the highly-proprietary and experimental nature of the hardware. Hoping to advance the field of neuromorphic computing as a whole, Intel has specifically designed its Lava framework to be platform-agnostic; the company says it has intentionally structured the code so that it is not tied to its own neuromorphic chips.
Additionally, Lava is fully open-source. Intel says it wants to encourage convergence in the field to avoid the current situation with traditional artificial neural networks that comprises an arbitrary mess of frameworks and tools that have varying feature sets and capabilities. It's also extensible; it can be interfaced with extant frameworks like YARP, TensorFlow, PyTorch, and more. Lava itself is entirely used with Python from the user side of things, but Intel does note that its libraries use optimized C, CUDA, and OpenCL code where suitable.
Perhaps the coolest part of Lava is that it is designed with heterogeneous computing in mind. It fully supports execution platforms that include both conventional CPUs and GPUs as well as neuromorphic processors. Intel says that this offers "an overall programming paradigm ... that supports extreme levels of parallelism." Finally, even though neuromorphic processors and spiking neural networks specialize in "learning on the fly," Intel notes that Lava supports tools such as SLAYER for offline training with backpropagation as well.
For now, Intel only offers Loihi 2 hardware to researchers, and only in two forms: the Oheo Gulch single-chip add-in-card that comes with an Arria 10 FPGA for interfacing with Loihi 2, and soon, the stackable Kapoho Point board that mounts eight Loihi 2 chips in a 4x4-inch form factor. Kapoho Point will have GPIO pins as well as "standard synchronous and asynchronous interfaces" that will allow it to be used with things like sensors and actuators for embedded robotics applications, as one example. The company says larger-scale systems remain in development.
The way Intel expects most partners to use Loihi 2, at least for now, is through the Neuromorphic Research Cloud, or NRC—not to be confused with the Neuromorphic Research Community, also called "NRC". To be clear, you have to join the NRC before you can use the NRC. Fortunately, membership is free and "open to all qualified academic, corporate, and government research groups around the world." No doubt due to the substantial research investment Intel has poured into these parts, the company says its goal is to "nurture a commercial neuromorphic ecosystem." Due to that, members are encouraged to share their work with the NRC, but it's not a hard requirement.
If you're an artificial intelligence researcher and you want to sign up for the NRC, head to this page and click the "please email us" link at the bottom. And if you have any thoughts on Loihi 2, we'd love for you to comment below.
