Glossary
Neural processing unit (NPU)
A neural processing unit (NPU) is a specialized chip for AI-intensive processes, such as real-time translation and image generation. Microsoft says many new Windows AI features require an NPU that can run at 40 or more trillion operations per second, and a speech model can run faster on one.
Updated October 16, 20264 cited facts, each traced to its source
Also called NPU.
A neural processing unit (NPU) is a specialized computer chip for AI-intensive processes, in Microsoft's words, such as real-time translation and image generation 1. It sits beside the main processor (CPU) and the graphics chip (GPU). Its job is the arithmetic that AI models repeat many times, so tasks like speech recognition and text generation can run on the device.
What an NPU does that a CPU does not
A CPU is built to run many kinds of programs. An NPU is built for one kind of work: the large batches of multiplication and addition that a neural network uses. Running that work on a dedicated chip frees the CPU for other tasks. Microsoft says its Phi Silica model is optimized for efficiency and performance on Copilot+ PCs, where it runs on the NPU 2.
Chips are often rated in TOPS, which stands for trillions of operations per second. Microsoft says many of the new Windows AI features require an NPU that can run at 40 or more TOPS 3. That is the threshold for the Copilot+ PC label. A TOPS rating is a peak figure, the most the chip can reach, so it does not tell you how fast a particular app will respond.
Example: running speech recognition on an NPU
The open-source whisper.cpp project runs OpenAI's Whisper speech model on a PC. Its README says that with a dedicated NPU, it can run the Whisper encoder on the NPU through VitisAI, a vendor-specific runtime, which gives a significant speedup over CPU-only inference 4. The encoder is the part of the model that reads the audio, so moving it to the NPU speeds up the first step of turning speech into text. The README does not give a single speed figure, and the result depends on the chip and the model size.
Why it matters for dictation
For dictation, the NPU is about speed and about running the speech model on the PC. The NPU helps only where a dictation app runs its speech model locally. Check whether an app does that before you expect the NPU to change anything.
An NPU does not make a recognizer more accurate. A faster chip finishes the same work sooner, and the words it produces depend on the model, not the chip.
Related terms
- Copilot+ PC: the Windows hardware class defined by the 40 TOPS NPU.
- Phi Silica: a Microsoft model that runs on the NPU of Copilot+ PCs.
- Neural Engine: Apple's equivalent block in Apple silicon.
Guides that cover it
Evidence
Every number on this page, with where it comes from. Open a fact to see the exact words in its source.
- 1.
- 2.
- 3.
- 4.
Related terms
- Dictation on Windows: Voice Typing, Voice Access, AppsThe topic this term belongs to
- Azure AI SpeechAzure AI Speech is Microsoft's cloud speech service. Microsoft now calls it Azure Speech in Foundry Tools, and its speech to text powers the online recognition behind Windows voice typing, which needs an internet connection.
- Copilot+ PCA Copilot+ PC is a class of Windows 11 hardware with a neural processing unit (NPU) that Microsoft says can run at 40 or more trillion operations per second. Some newer Windows AI features, including fluid dictation, require this hardware.
- Fluid dictationFluid dictation is a Windows feature on Copilot+ PCs that corrects grammar, punctuation and filler words as you speak. Microsoft says it is powered by small language models that run on the device, and that you can undo a correction by saying Revert or Undo that.
- Number and grid overlays (Voice Access)Number and grid overlays are Voice Access views for clicking things you cannot name. Show numbers puts a number on each clickable item so you can say it, and Show grid divides the screen into numbered tiles that narrow the mouse pointer step by step.
- Online speech recognitionOnline speech recognition sends your voice to a cloud service that turns it into text, instead of processing it on your PC. Microsoft says voice typing on Windows uses online speech recognition, which Microsoft describes as giving more accurate recognition.
- Phi SilicaPhi Silica is Microsoft's on-device language model, which runs on the neural processing unit of Copilot+ PCs. Microsoft says it is being replaced by Aion Instruct, and its voice typing help names it as the download to watch while fluid dictation gets ready.
- Speech pack (Windows)A speech pack is the set of on-device language files that Windows needs before it can recognize speech in a language offline. Microsoft says you must download a speech pack for a language in Voice access before you can use it, and the first download needs an internet connection.
- Voice Access (Windows)Voice Access is the Windows 11 feature that lets you control the whole PC and write text by voice, not only dictate. Microsoft says it works without an internet connection after setup, and it replaces Windows Speech Recognition.
- Voice typing launcherThe voice typing launcher is the Windows voice typing setting that lets you start dictation quickly when the cursor is in a text box. Microsoft lists it among the settings you open from the gear icon in the voice typing flyout.
- Windows key + H (Win+H)Windows key + H, written Win+H, is the keyboard shortcut that starts voice typing in Windows, so you can dictate into the text box where your cursor is. Microsoft says it does not start voice typing while Voice Access is running.
- Windows on ArmWindows on Arm is Windows running on processors that use the Arm instruction set instead of x86 or x64 chips. It runs native Arm apps and many unmodified x86 and x64 apps, and Microsoft says Arm-native apps give the best performance and battery life.
- Windows Speech API (SAPI)The Windows Speech API (SAPI) is Microsoft's interface that lets Windows apps use speech recognition and text-to-speech without building the speech engine themselves. Windows.Media.SpeechRecognition is a separate API for command and control in Windows Runtime apps.