What the heck is an NPU, anyway? Here’s everything you need to know

CPUs and GPUs are old news. These days, the cutting edge is all about NPUs, and hardware manufacturers are talking up NPU performance.

The NPU is a computer component designed to accelerate AI tasks in a power-efficient manner, paving the way for new Windows desktop applications with powerful AI features. That’s the plan, anyway.

All PCs will eventually have NPUs, but at the moment only some PCs have them. Here’s everything you need to know about NPUs and why they’re such a hot topic in the computer industry right now.

What is an NPU?

NPU stands for neural processing unit. It’s a special kind of processor that’s optimized for AI and machine learning tasks.

The name comes from the fact that AI models use neural networks. A neural network is, in layman’s terms, a vast mesh of interconnected nodes that pass information between them. (The whole idea was modeled after the way our own human brains work.)

Related: How neural networks are finally maturing

An NPU isn’t a separate device that you buy and plug in (as you would with a GPU, for example). Instead, an NPU is “packaged” as part of a modern processor platform — like Intel’s Core Ultra, AMD’s Ryzen AI, and Qualcomm’s Snapdragon X Elite and Snapdragon X Plus. These platforms have a CPU along with an integrated GPU and NPU.

NPU vs. CPU vs. GPU: What’s the difference? Explained

For many years now, computers have been running tasks on either the central processing unit (CPU) or graphics processing unit (GPU.) That’s still how it works on AI PCs (i.e., computers with NPUs).

The CPU runs most of the tasks on the computer. But the GPU, despite its name, isn’t just for graphics and gaming tasks. The GPU is actually just optimized for a different type of computing task, which is why GPUs have been critical for non-gaming endeavors like mining cryptocurrency and running local AI models with high performance. In fact, GPUs are very good at such AI tasks — but GPUs are awfully power-hungry.

That’s where NPUs come into play. An NPU is faster than a CPU at computing AI tasks, but not as fast when compared to a GPU. The trade-off is that an NPU uses far less power than a GPU when computing those same AI tasks. Plus, while the NPU handles AI-related tasks, the CPU and GPU are both freed up to handle their own respective tasks, boosting overall system performance.

Why use an NPU instead of a CPU or GPU?

If you’re running AI image generation software like Stable Diffusion (or some other AI model) on your PC’s hardware and you need maximum performance, a GPU is going to be your best bet. That’s why Nvidia advertises its GPUs as “premium AI” hardware over NPUs.

But there are times when you want to run AI features that might be too taxing for a regular CPU but don’t necessarily need the top-tier power of a GPU. Or maybe you’re on a laptop and you want to take advantage of AI features but don’t want the GPU to drain your battery down.

With an NPU, a laptop can perform local (on-device) AI tasks without producing a lot of heat and without expending inordinate battery life — and it can perform those AI tasks without taking up CPU and GPU resources from whatever else your PC might be doing.

And even if you aren’t interested in AI per se, you can still take advantage of NPUs for other uses. At CES 2024, HP showed off game-streaming software that uses the NPU for video-streaming tasks, freeing up the GPU to run the game itself. By using the NPU’s extra computing power, the streaming software doesn’t take up any GPU resources… and it’s much faster than using the CPU for the same type of task.

But what can Windows PC software do with an NPU, really?

With an AI PC, the NPU can be used by both the operating system and the apps that reside on the system.

For example, if you have laptop with Intel Meteor Lake hardware, the built-in NPU will let you run Windows Studio Effects, which are AI-powered webcam effects that provide features like background blur and forced eye contact in any application that uses your webcam.