Can speech to text work without an LLM?

Yes. Speech to text does not need a large language model. A recognizer turns sound into words with its own models, and an LLM is an optional second step that can correct or rewrite what the recognizer produced. Vosk, an open-source engine, describes itself as an offline speech recognition API that runs on small devices. The trade-off is mostly accuracy on hard audio, and whether your audio stays on your device depends on the tool, not on whether an LLM is involved.

Recognition and language modeling are separate steps, so one can run without the other

A recognizer has two jobs. One part maps sound onto speech sounds, and another part scores which word sequences are likely. Before 2012, most systems used hidden Markov models to handle the timing of speech and Gaussian mixture models to score how well each state fits a short window of sound ¹. A language model was a separate component that scored word sequences on top of that, and it was never the part that heard the audio.

Some systems now work without any separate language model. Google's Listen, Attend and Spell (LAS) model reached a word error rate of 14.1 percent on a subset of its voice search task without a dictionary or a language model ². That figure comes from short voice search queries, so it does not describe long dictation. It does show the principle: the recognizer can work alone, and a language step can be added later.

Offline engines run without an LLM, and their accuracy is the trade-off

Vosk says it works offline even on lightweight devices such as the Raspberry Pi, Android and iOS ³. Its small English model, vosk-model-small-en-us-0.15, is 40 MB and had a word error rate of 9.85 percent on the LibriSpeech test-clean set, a read-speech test, under the Apache 2.0 license ⁴. Vosk gives a typical figure of about 50 MB on disk and about 300 MB of memory while a small model runs ⁵.

Those numbers are the project's own measurements on clean, read speech. They say nothing about a noisy office, a fast conversation or a heavy accent, and a small model trades accuracy for size. So a no-LLM engine is a real option for offline work, but you should test it on your own voice and your own microphone before you rely on it for a long document.

An LLM can improve a recognizer, but it is an extra step

Much of the interest in this question comes from research showing that an LLM can make a recognizer's output better. In the HyPoradise benchmark, LLM-based correction of a recognizer's output reduced word error rate by 77.1 percent relative on the ATIS test set ⁶. Those are relative reductions on two test sets from a 2023 study, not absolute error rates, and they describe a research setup, not a dictation app.

The order matters. The recognizer produces text first, and the LLM edits that text second. A tool can stop after the first step, which is the verbatim case, or add the second. Both are speech to text, but only the second involves an LLM.

Where the audio is processed decides privacy, and that is separate from the LLM question

Some tools that do not use an LLM still send your speech to a service. Microsoft says that Windows voice typing converts speech to text using online speech recognition ⁷. If you need offline dictation on Windows, check the tool's own settings instead of assuming that voice typing works without a connection. Our answer to whether Windows voice typing works offline covers the steps.

On Apple platforms, a developer can require on-device recognition. Apple says that this setting stops the request sending audio over the network, and that on-device requests are less accurate ⁸. That is a developer setting, not a switch in the Dictation menu, but it shows the trade-off clearly: keeping audio on the device can cost some accuracy. For the iPhone side, see whether iPhone dictation works offline.

What to do if you want speech to text with no LLM

1. Name the setting you need. No LLM rewrite, offline recognition and no stored audio are three separate settings, and a tool can have one without the others.
2. Find a verbatim mode. A tool that types what the recognizer hears has no second pass that rephrases your words. The dictation app without AI question covers how to check a tool.
3. Check where recognition runs. Look for a statement that recognition is on-device, and confirm it in the tool's own documentation for your operating system version.
4. Test on your own speech. A word error rate from one test set does not carry over to your accent, your room or your microphone. Dictate a page you already know and count the errors yourself. The word error rate entry explains how that count works.
5. Accept the accuracy trade-off. A small local model may make more errors than a large cloud model, so proofread the output before you send it.

Yap, which we make, is an on-device dictation app for Mac and Windows, so you can check its settings for where recognition runs on the download page. For the whole subject, start with speech to text without AI, or read the voice and AI overview. If you want the related questions, see whether Whisper is an LLM, whether LLMs can do speech recognition and whether LLMs improve speech recognition accuracy.