Can large language models do speech recognition?

Yes, in part. A large language model can do speech recognition when it receives audio in a form it can read. A text-only LLM cannot hear anything, so the audio has to pass through a speech encoder first. Alternatively, an LLM can work after an ordinary recognizer, correcting its text. Both approaches exist. For dictation that appears as you speak, start with a streaming speech model.

A text-only LLM cannot hear, so audio must be encoded first

An LLM reads tokens, so audio has to be turned into vectors that the model can process. That is what a speech encoder does, and it is why the newest systems are built from two parts: an encoder that hears and a language model that writes. The SALMONN authors describe a model that "enables the LLM to directly process and understand general audio inputs" and reaches competitive performance on speech tasks ¹.

NVIDIA's Canary Qwen 2.5B is a speech encoder with a language-model decoder. On the Open ASR Leaderboard's English short-form track, it had a 5.63% average word error rate and an inverse real-time factor of 418 ². On the same track, Parakeet TDT 0.6B v2 had a 6.05% average word error rate and ran much faster ³. An LLM-based design can lead on accuracy, but it does not automatically lead on speed, and the leaderboard is one benchmark. The guide to speech recognition models puts these families side by side.

An LLM does its best work after the recognizer, as a corrector

The second approach keeps an ordinary recognizer and lets a language model fix its output. The recognizer produces a list of candidate transcripts, called an N-best list. The HyPoradise benchmark shows that a language model can do more than pick the best candidate. It can correct words that are missing from the N-best list, because it generates text rather than only choosing among candidates ⁴.

HyPoradise is a research benchmark built from recognizer output, so it shows the capability exists, not how often a particular app changes your words. For everyday cleanup, such as removing fillers and adding punctuation, read AI cleanup for dictation.

Whisper is a speech model, not a GPT chatbot, and it can add sentences

People often ask whether Whisper is a GPT model. It is not. OpenAI's Whisper is a speech model. The authors built it on an encoder-decoder Transformer, and they chose that design because they judged it "well validated to scale reliably" ⁵. It is trained to transcribe audio, not to hold a conversation.

Its decoder writes text, so a transcript can also contain fluent phrases the audio never held. In a 2024 study of Whisper, roughly 1 percent of audio transcriptions contained entire hallucinated phrases or sentences ⁶. Check any Whisper transcript against the recording before you rely on it. For a full comparison of Whisper and the other options, read the guide to Whisper AI.

ChatGPT dictation sends your audio to OpenAI's models

When you press the microphone icon in ChatGPT, OpenAI says that the recorded audio is sent to its models to be transcribed ⁷. For privacy, that means your speech leaves your device, so check that before you dictate anything sensitive. The question on what speech to text ChatGPT uses covers the details.

LM Studio runs language models, and we did not find speech-to-text in it

We searched LM Studio's documentation on October 17, 2026, including its app docs and its developer docs. We found no speech-to-text feature for its model runtime. The docs do list a Bionic page on voice input with local transcription, which we have not read in full, so check that page before you rely on it. To transcribe a file with a local tool, use a speech model such as Whisper run locally, then give the text to a language model if you want it cleaned up.

Benchmark scores for LLM systems need care

Test sets can leak into training data. A 2025 study found that a substantial amount of the LibriSpeech and Common Voice evaluation sets appear in public large language model pretraining corpora ⁸. A system that has seen a test set can post a low error rate on it without doing better on your audio. Test on recordings you control, as the guide to speech recognition benchmarks explains.

What to do: four steps

1. For live text, start with a streaming speech model. LLM-based systems differ in whether they stream, so check that before you rely on one for live dictation.
2. For a recording, transcribe with a speech model first. Then review the transcript against the audio.
3. Use a language model afterward for cleanup only. Compare the cleaned text with the raw transcript, and reject any edit that changes the meaning.
4. If you need an LLM that listens, read its license and terms. A model's name does not tell you whether you can use it commercially.

For a fuller picture of how speech models fit with language models, read speech to text and LLMs. For how the networks differ, read which neural network is best for speech recognition.