Meteor Lake, sold as Intel Core Ultra (Series 1) from December 2023, was Intel's first PC processor with a built-in neural processing unit (NPU). It spreads AI work across three engines: the CPU for short, latency-sensitive tasks, the integrated Arc GPU for heavy parallel throughput, and the NPU for sustained, low-power AI such as background blur, noise suppression and always-on assistants. In this Intel Technology "Talking Tech" video, two Intel engineers explain how the NPU works and why software and APIs matter as much as the silicon.
Key takeaways
- The NPU is a dedicated accelerator for dense, sustained AI inference at low power, so the CPU and GPU stay free for other work.
- Meteor Lake is a tiled (chiplet) design; the NPU sits in the SoC tile alongside media and display blocks.
- Its compute engines combine fixed-function multiply-accumulate (MAC) arrays with programmable DSP cores.
- Developers reach the NPU through standard APIs and frameworks, such as OpenVINO, ONNX Runtime and DirectML, rather than programming it directly.
About the video
The PC industry is at a significant inflection point, and Intel presents Meteor Lake as bringing AI to client computing at scale. In the video, Darren Crews, Senior Principal Engineer for AI software architecture, and Intel Fellow Tom "TAP" Petersen explain how Intel Core Ultra delivers AI acceleration across the CPU, GPU and the new NPU. They cover how the NPU is optimised for dense and sustained AI workloads, and why a friendly developer experience depends on focusing on software and APIs.
Talking Tech goes in depth on Meteor Lake with Intel architects and engineers: https://intel.ly/3MmhpRl. VP of Design Engineering Group Tim Wilson explains Meteor Lake in Architecture All Access: https://intel.ly/3Spf6ku.
Chapters
- 00:00 Intro
- 00:33 Purpose and role of the NPU
- 01:45 APIs
- 02:24 NPU compute engines explained
- 05:06 Drivers and software
- 06:56 Bringing AI to client
- 09:25 Conclusion
About Intel Technology: the Intel Technology YouTube channel covers emerging technologies, data centre servers, business transformation, memory and storage, security and graphics, with tips, news and product demos from Intel and its partners. Website: https://intel.ly/IntelTechnologies. Intel Technology on X (Twitter): https://intel.ly/3MmhqET.
Why put an NPU in a PC?
AI features on a laptop tend to fall into two groups. Some are bursty and need an answer immediately, such as a quick text prediction. Others run continuously for minutes or hours, such as background blur and eye-contact correction on a video call, live captions or noise suppression. Running the continuous kind on the CPU or GPU works, but it costs battery life and steals cycles from the applications the user is actually running.
An NPU is built for exactly that continuous kind of work. It is a block of hardware that does little except the matrix multiply-accumulate operations at the heart of neural networks, with local memory to keep data close, so it delivers more inference per watt than a general-purpose core.
CPU, GPU and NPU: who does what
| Engine | Strength | Typical AI use on Meteor Lake |
|---|---|---|
| CPU | Lowest latency, flexible, runs any code | Small models, light or one-off inference, pre- and post-processing |
| GPU (integrated Arc) | Highest parallel throughput | Large models, image generation, content creation in bursts |
| NPU (Intel AI Boost) | Best efficiency for sustained inference | Video call effects, background AI, always-on features |
Inside the NPU
Intel describes the Meteor Lake NPU as two neural compute engines. Each engine pairs an inference pipeline, a large array of MAC units for the matrix and convolution work, with programmable SHAVE DSP cores for operations that do not map neatly onto the MAC array, such as activation functions and data reshaping. A DMA engine and local scratchpad memory move tensors in and out efficiently, and a memory management unit lets the NPU share system memory safely with the rest of the chip.
The NPU sits in the SoC tile of Meteor Lake's tiled design, which Intel builds with Foveros 3D packaging. Because it is outside the compute tile, the NPU can stay active while the main CPU cores power down, which helps battery life for always-on features.
A rough sense of scale: Intel quoted the Meteor Lake NPU at around 10 to 11 TOPS (trillions of INT8 operations per second), with the CPU, GPU and NPU together quoted at up to about 34 TOPS for the platform. These are vendor peak figures; real throughput depends on the model, precision and memory traffic.
Why software matters as much as silicon
A new accelerator is only useful if developers can reach it without rewriting their applications. Intel's approach is to expose the NPU through standard routes:
- OpenVINO, Intel's toolkit for optimising and deploying models on CPU, GPU and NPU.
- ONNX Runtime with execution providers that target Intel hardware.
- DirectML and Windows ML on Windows, so applications can use the NPU through Microsoft's APIs.
- A driver stack that handles scheduling, memory and power management, so the operating system can treat the NPU like other devices.
This is the theme of the APIs and drivers chapters of the video: the hardware must be paired with stable software interfaces before independent software vendors will build on it.
Common misunderstandings
- "The NPU replaces the GPU for AI." It does not; for large or bursty models the GPU often has more raw throughput. The NPU wins on sustained efficiency.
- "More TOPS always means faster AI." Peak TOPS ignores memory bandwidth, supported operators and precision; two NPUs with similar TOPS can perform very differently.
- "Applications use the NPU automatically." Only if they, or the framework they use, target it.
FAQ
What is Intel AI Boost?
Intel's brand name for the NPU in Core Ultra processors.
Which processors have the Meteor Lake NPU?
Intel Core Ultra (Series 1) mobile processors, launched in December 2023. Later generations such as Lunar Lake have larger NPUs.
Do I need special software to use the NPU?
You need an up-to-date NPU driver and an application or framework that targets it, such as OpenVINO, ONNX Runtime or a Windows feature built on DirectML.
Your comments will be moderated before it appears here.