Qualcomm’s NPU Upgrade Unlocks Massive 30B Parameter AI Models For Next-Gen Phones

Over the past several weeks, Qualcomm has been revealing key details about the future of its next-generation mobile platform, including a blistering 5GHz Oryon CPU architecture and a new Adreno Neural Fusion graphics engine. Following up on those announcements, Qualcomm has now detailed the third major pillar of its next Snapdragon flagship: a redesigned Hexagon NPU built specifically for agentic AI and massive parameter models.

Qualcomm's updated Hexagon NPU introduces two notable hardware upgrades to handle persistent, multi-modal workloads without draining smartphone batteries, including an Element Accelerator and an expanded shared memory pool.

The Element Accelerator combines high-throughput vector extensions for raw matrix math with scalar extensions to accelerate execution logic, decision-making, and tool routing. Qualcomm says it's purpose-built for the kind of transformer workloads that power today's generative and agentic AI, and the end result is a mechanism that helps AI respond faster and more efficiently.

"Just as important, our Hexagon NPU significantly expands its large shared memory. By keeping more model state, context, and KV-cache data closer to the accelerators, the platform reduces memory bottlenecks and keeps agents responsive as they juggle longer context, more tools, and more concurrent tasks," Qualcomm says.

According to Qualcomm, its upgraded NPU packs 50% more shared memory versus the NPU in the Snapdragon 8 Elite Gen 5. Additionally, Qualcomm says the built-in KV-cache acceleration supports context lengths up to 32,000 tokens.

Part of the challenge with running large language models (LLMs) locally on mobile devices is trying to overcome limits to memory bandwidth. To help circumvent this, Qualcomm is shifting toward Mixture-of-Experts (MoE) model architectures. Instead of running an entire dense network for every request, a 30-billion-parameter MoE model keeps its massive knowledge base available on-device while activating only about 3 billion routed parameters per token generation step. 

"MoE dynamically selects specialized experts based on the input, reducing active compute and memory bandwidth demands. Combined with intelligent flash-to-memory expert management and caching techniques, this approach enables larger-model-class AI experiences with low power consumption and memory requirements, bringing responsive, private generative AI to phones," Qualcomm says.