On-Device vs Cloud Speech Recognition: The Tradeoffs

On-device recognition runs the speech model on your own phone or computer. The audio does not have to leave the device, so dictation keeps working without a network. Cloud recognition uploads the audio to a server, which can run much larger models and cover more languages, and the service charges by the minute. Accuracy depends on the speech, the background noise and the microphone ¹, so the decision usually comes down to privacy, connectivity, cost per minute and the kind of speech you need to recognize. For the basics of how the model turns sound into text, see how speech recognition works.

On-device recognition runs the model on your hardware, while cloud recognition sends your audio to a server

The difference is where the computation happens. An on-device speech recognition engine loads the model onto the phone or laptop and turns sound into text there. A cloud dictation service sends the audio over the internet and returns the transcript from a data center. Each design has consequences for privacy, for whether dictation works offline and for who pays for the computing.

Apple's newest engine is fully local. Apple says transcription with its SpeechAnalyzer API is entirely on the device, although the language models must first be downloaded ². Apple's older SFSpeechRecognizer class could use Apple's servers on devices with limited resources, and Apple says it worked well for short dictation ³. So the words "on-device" depend on which API an app calls, not only on the app's name.

The setting you choose matters too. On a Mac, Apple says voice inputs and transcripts for general text Dictation are processed on your device and not sent to Siri servers, and you can check this in Keyboard settings ⁴. Google's App Store listing for Gboard on iPhone describes a different path: if you use Gboard's microphone, voice input is sent to Google for transcription ⁵. Our overview of what Google's Speech Recognition & Synthesis app does covers a related Google app. Check each app's settings and privacy listing rather than assuming a default.

Compressed and quantized models fit speech recognition onto phones and laptops

Shrinking a model is what made local recognition practical. A neural network stores millions of numbers, and storing each one as an 8-bit integer instead of a 32-bit floating-point number cuts the memory it needs. Jacob and colleagues report that quantizing both activations and weights to 8-bit integers gives a close to 4x memory footprint reduction compared with 32-bit floating-point numbers ⁶. Their tests used image models such as MobileNets, so the 4x figure shows the principle, not a speech benchmark.

Speech shows the same pattern. Google's older server recognizer used a search graph of almost 2 GB, which Google said was too large for a phone and was why it needed a connection ⁷. Google's 2019 Gboard recognizer moved voice typing onto the phone with an 80 MB speech recognizer that works even when the phone is offline ⁸. Google said the new design offered the same accuracy as its traditional server-based models in a much smaller model ⁹. That is Google's own claim, made at launch, under its own test conditions.

Open-source tools follow the same route. The whisper.cpp project says its quantized Whisper models require less memory and disk space, and can be processed more efficiently depending on the hardware ¹⁰. The README gives no percentage, so read the saving as a direction and measure it on your own machine.

Quantization is also a trade. A lower-precision model can lose some accuracy, and the loss depends on the model and the audio. Test a quantized model on recordings like yours before you rely on it.

Neural engines and NPUs in modern chips run speech models at low power

Dedicated neural hardware is the second half of the story. Apple says the Neural Engine in the M4 chip is capable of up to 38 trillion operations per second, which it describes as faster than the neural processing unit of any AI PC at the time ¹¹. Apple describes the Apple Neural Engine as the energy-efficient, high-throughput engine for machine learning inference on Apple silicon ¹². A neural engine and a neural processing unit are both built for this kind of work.

Those are peak figures, and they do not show how much energy a speech model uses. The battery section below gives a measured example.

Cloud models still lead on the largest models, rare languages and long recordings

Cloud servers are not limited by a phone's memory, so they can run the largest models. The Whisper family shows how quickly the size grows. The small model has 244 million parameters and needs about 2 GB of memory ¹³. The medium model has 769 million parameters ¹⁴, and the large model has 1,550 million parameters and needs about 10 GB of memory ¹⁵.

Size is not the whole story. On the Open ASR Leaderboard's English short-form track, Parakeet TDT 0.6B v2 had an average word error rate (WER) of 6.05 percent ¹⁶. Whisper Large v3 had 7.44 percent on the same track ¹⁷. The smaller model, at 0.6 billion parameters ¹⁸, was the more accurate one there. Bigger is not automatically better. The leaderboard compares systems on shared test sets, so a different test or language can change the order.

Language coverage is where the cloud advantage is most visible. Apple lists 48 language and regional variants for on-device and modeless Dictation on the Mac ¹⁹. Apple lists 63 language and regional variants for Dictation on the Mac ²⁰, and 15 of them are not on the on-device list ²¹. For those 15, Apple's list does not show on-device dictation, so check before you rely on it offline. If you dictate in a less common language, check the list for your language before you choose an offline tool.

Long recordings are a different problem. Whisper was trained on 30-second audio chunks and cannot take a longer file as a single input ²². Whether the model runs locally or in the cloud, a long file has to be split into chunks by the software. A tool that handles chunking well matters more than where the model happens to run.

The evidence we found does not show on-device dictation to be as accurate as cloud dictation in general. Accuracy depends on the engine: in one 2025 test of a single 7-minute-31-second episode, Apple's speech API had a word error rate of 10.3 percent, against 1.5 percent for Whisper Large V3 Turbo ²³. The question on this point has its own answer: is on-device speech recognition as accurate as cloud. The size, language and recording-length limits above are where the cloud still has the edge.

On-device recognition wins on latency, offline use, cost per minute and privacy

On-device recognition wins on four counts. Latency comes first. The audio never crosses the network, so no round trip is added to each phrase. We did not find an independent measurement of that round trip in this research, so this guide does not put a number on it.

Offline use is the second. Google said in 2019 that its on-device recognizer is always available, even when the phone is offline ²⁴. Apple says SpeechAnalyzer transcribes entirely on the device once its models are downloaded ².

Privacy is the third. Apple's general statement is that on-device machine learning benefits privacy because the data for inference stays on the device rather than on a server ²⁵. Recognition on the device means the audio itself is not sent to a provider for transcription. It does not mean an app never sends data elsewhere, so read its privacy policy.

Cost per minute is the fourth, and it is the one with a clear price tag. The next section compares the bills.

Apple, Google and Microsoft each mix on-device and server recognition

No major platform is purely one or the other. Each mixes the two by feature, by language and by app, and the table shows the split as the sources describe it.

Apple moved its newest engine fully to the device, and Google put an on-device engine into Gboard while the iPhone app still sends microphone input to Google. In these sources, the newest system APIs are the ones that run on the device, while app makers still choose their own path. Our overview of offline speech to text on Windows and offline voice typing on phones covers the platform-specific details.

Cloud recognition bills per minute, while on-device recognition costs memory, battery and download size

Cloud recognition is billed by usage. Google's V1 list price for standard recognition is $0.016 per minute with data logging, after the first 60 minutes each month ²⁷. Once the first 60 free minutes each month are used, one hour of audio works out to about $0.96 ²⁸. Without data logging, the same hour costs about $1.44 once the free minutes are used ²⁹. AssemblyAI lists its Universal-2 model at $0.15 per hour of audio ³⁰.

Prices change, so treat these as list prices on the dates we checked them, and check the provider's page before you budget. The Google per-hour figures are the per-minute prices times 60. They apply once the free first 60 minutes each month are used.

Built-in on-device dictation has no per-minute bill. It has a different cost, paid in device resources. Memory is the first. Whisper large, an open model, needs about 10 GB of memory ¹⁵, while Google's 2019 Gboard recognizer is an 80 MB model ⁸. Download size is the second, and it is paid once. Battery is the third. Li and colleagues report that changes to how their on-device streaming speech model is built can cut its energy use by up to 47 percent while keeping accuracy comparable ³². That is a best case against their own baselines, measured on their model, not a battery-life figure for a phone.

Small embedded boards handle wake words and short commands, not open dictation

Small boards sit at the far end of the same trade-off. Espressif says its MultiNet model supports up to 300 Chinese or English speech commands, which is a list of short phrases rather than open dictation ³³. That is the right tool for a light switch or a timer, and the wrong tool for an email.

Larger open models run on small computers too, but slowly. Home Assistant's documentation says Whisper takes around 8 seconds to process a voice command on a Raspberry Pi 4, and under a second on an Intel NUC ³⁴. Home Assistant's Speech-to-Phrase is a closed-ended model that transcribes only what it knows, and it transcribes in under one second even on a Raspberry Pi 4 ³⁵. The trade is clear: a small command set is fast on modest hardware, and open dictation needs a larger model and more time. For the Pi details, see the question on running speech recognition on a Raspberry Pi.

How to choose between on-device and cloud recognition

Start with the built-in options if they cover your language and privacy needs. They are the simplest place to start. Our guide to running Whisper locally covers the open option. If you need a language that the built-in list does not offer, or long files with speaker labels, a cloud service may be worth its per-minute price. Check the provider's data-retention settings before you send audio that contains personal or client information.

To estimate a monthly cloud bill, multiply your recorded minutes by the provider's per-minute price, then subtract any free allowance. A cloud service is a poor fit for constant background transcription, because the bill grows with every minute. On-device recognition costs the same however long you use it, but it draws on battery and memory while it runs. Test both on a week of your own audio before you commit to either.

Yap, which we make, is an on-device dictation app for Mac and Windows. You can see what it does on the download page, and the offline dictation page for Mac explains the on-device setup. For the terms used here, see our glossary entries on word error rate and wake words. To go further, read our guides to speech recognition accuracy, speech recognition models and Apple dictation privacy, or return to the speech recognition overview.