How do you connect speech to text to an LLM?

To connect speech to text to a large language model (LLM), you run two steps in order. First, a recognizer turns your audio into text. Second, your app sends that text to the model as its message, together with your instructions, and then shows or speaks the reply. This is a chain. OpenAI's voice agent guide says to use its chained path when you want to inspect or transform the text between speech recognition, your agent and speech generation ¹. The text sits in the middle, where you can read it, log it and correct it.

Step 1: Transcribe the audio before the model sees it

Choose the recognizer for your job first. The choice of speech to text to pair with an LLM depends on whether the speech is live, recorded or private. For a recording you already have, a batch call is the simplest route. OpenAI's speech-to-text guide shows its transcription quickstart as sending the audio file to /v1/audio/transcriptions with the model gpt-transcribe ². The text that comes back is what you pass on to the model.

Batch services have size limits. OpenAI's Transcriptions API accepts files up to 25 MB, and larger recordings must be compressed or split into chunks ³. When you split a recording, cut at a pause between sentences rather than in the middle of a sentence, since a cut there separates words that the recognizer and the model need together.

Step 2: Send the final transcript, not the partial text

Live speech arrives in pieces. A streaming recognizer shows a partial transcript while you talk, then revises it. The partial text is a draft. In Artificial Analysis's 2026 streaming benchmark, OpenAI's GPT Realtime Whisper went from a 7.5 percent word error rate on its first partial transcript to 5.1 percent on the final transcript ⁴. Send the model the final text, which means waiting until the recognizer has committed to it.

That wait needs a signal for when you have finished. This is called turn detection. OpenAI's Realtime API uses periods of silence to split the audio into turns by default ⁵. A short silence sends the turn early, so the model may answer half a thought. A long silence waits longer, which adds delay before the model starts. Set the silence length by testing your own pauses, not by copying a default.

Step 3: Put the rules in the instructions and the speech in the message

Keep the two roles apart. Your instructions go in the system message, for example "remove filler words and keep names and numbers exactly as spoken". The transcript goes in the user message. Keeping them apart makes it possible to log the transcript, the instructions and the reply separately. If the reply goes wrong, you can tell which of the three caused it.

Step 4: Stream the reply and time the wait

Stream the model's reply as it is generated, so the first words appear before the answer is finished. The delay you feel is the time to the first token of the reply, plus the recognizer's wait for the final transcript. In VoiceAgentBench, a pipeline of Whisper large v3 followed by the Qwen3 8B language model had a 90th percentile time to first token of about 800 milliseconds ⁶. Measure your own chain the same way, from the moment you stop speaking to the first word of the reply, and add the figures for each stage.

The chain has a weakness: errors pass from one stage to the next

A chain is only as good as its first stage. The SeamlessM4T paper lists error propagation from speech recognition into the translation model as a known weakness of cascaded systems ⁷. The same risk applies to an LLM that follows a recognizer. A misheard name becomes a wrong name in the answer, and the model may write it so fluently that the error is hard to spot. This is why the transcript should be kept and checked, and why the model's own changes should be compared with it.

The alternative is one model that hears and answers

A speech-to-speech path removes the middle step. OpenAI describes its speech-to-speech path as a single model that takes audio in, decides what to do and responds in speech ⁸. You gain a shorter path and a natural voice, and you lose the text you could inspect. For a voice assistant that must answer quickly, that trade can be the right one. For dictation, where the words are the product, the chain is usually the safer choice. The difference between the two designs is covered in cascaded versus speech-to-speech voice AI, and the model type is described in the speech-to-speech model entry.

It depends on where the words go

• Dictating into a document: you may not need an LLM at all. If plain text is enough, built-in dictation needs no LLM step you add yourself. If you add an LLM for cleanup, run it on finished text, where its delay matters less.
• Asking an assistant or agent: use a chain with streaming, and measure the delay in your setup. The reply is the product, so every stage counts.
• Audio that must stay private: a cloud recognizer sends your audio out of the device, and a cloud model sends the text out too. Yap, which we make, is an on-device dictation app for Mac and Windows, and its speech recognition runs on your device, so you can dictate privately before any text reaches a model. You can download it and test it on your own speech.

What to do

1. Name the job. Decide whether the speech is live, recorded or private, and choose the recognizer that fits.
2. Build the chain with a saved transcript. Store the raw text and the model's reply together, so you can check every change.
3. Send only the final transcript. Wait for the recognizer to commit to the text, and set your turn detection by testing your own pauses.
4. Test the output on your own speech. Record 20 to 30 sentences, and compare the recognizer's text with the model's version, counting the words the model changed that were already right. The guide to speech to text and LLMs has the full test.
5. Time the wait. Measure from the end of your speech to the first word of the reply, and find which stage takes the most time.

Related questions are can LLMs do speech recognition and is Whisper an LLM. For the wider subject, see speech to text and LLMs and Voice and AI.