Glossary
Turn detection
Turn detection is the step that decides when a speaker has finished a turn, so a voice system knows when to respond. A pause can be a hesitation rather than an ending, so the decision trades speed against cutting the speaker off.
Updated October 17, 20264 cited facts, each traced to its source
Also called end-of-turn detection, semantic endpointing, barge-in handling.
Turn detection (also called end-of-turn detection, semantic endpointing or barge-in handling) decides when a speaker has finished a turn, so the system knows when to respond. It also decides what to do when someone interrupts while the system is talking. Both decisions depend on the same question: is this pause the end of the thought, or a breath in the middle of it?
Silence is the simplest signal
The simplest design waits for silence. OpenAI's Realtime API voice activity detection, which is its default server-side turn detection for models that support it, uses periods of silence to split the audio into turns 1. A longer silence threshold makes the system wait longer before it decides you are finished.
Research systems often call this part an endpointer. A 2023 study describes an endpointer as a model that detects whether the user has finished speaking 2. Newer designs try to use the content of the words as well as the sound, so that "I want to book a flight to" is not treated as complete.
Waiting longer costs speed, and cutting in costs accuracy
Turn detection is a trade-off. Wait too long and the reply feels slow. Human conversation is quick, so a system that takes much longer than a person would to reply can feel unresponsive.
Cutting in too early has its own cost. Sainath et al. report that, with an acoustic endpointer alone, their on-device model reached a median endpointer latency of 450 ms, and its word error rate rose from 6.2 to 7.4 percent 3. The endpointer's timing changed the accuracy, so the choice of threshold is not free.
A fixed silence rule has no way to tell a planned pause from a finished sentence.
Why it matters for dictation
Dictation usually avoids automatic turn detection. The 2023 study notes that in dictation, the user finishes by clicking a button again, instead of relying on an endpointer 4. The person decides when the text is done, and a pause does not end it.
That is a sound default for writing. You think in pauses, and a tool that ends your draft at the first quiet moment wastes words. When you use a voice agent or an assistant, turn detection decides when your request is complete, and a bad threshold sends a half-sentence to the system. For dictation, look for an explicit way to end the session.
Related terms
- Voice activity detection: detecting whether anyone is speaking, the signal many turn detectors build on.
- Voice agent: a conversational system that depends on turn detection to reply at the right time.
- Push-to-talk: starting and ending input with a button instead of a timer.
- Automatic speech recognition: the recognizer that turns each detected turn into text.
Guides on this topic
Evidence
Every number on this page, with where it comes from. Open a fact to see the exact words in its source.
- 1.
- 2.
- 3.
- 4.
Related terms
- Speech to Text AI: Dictation, Assistants and Voice AgentsThe topic this term belongs to
- AI text detectorAn AI text detector is software that estimates whether a passage was written by a language model, usually from the words alone. Detectors make mistakes in both directions, so a score is weak evidence about how a text was made.
- Generative AIGenerative AI is the class of AI models that create new content, such as text, audio or images, from patterns in their training data. The NIST profile defines it as models that emulate the structure and characteristics of input data to generate derived synthetic content.
- Generative error correction (GER)Generative error correction (GER) passes a speech recognizer's output, often its N-best list of alternative transcripts, to a large language model, which rewrites the transcript to fix errors. It can recover words the recognizer never offered, but it can also change words that were already right.
- Intent recognitionIntent recognition is the step that decides what a spoken request is asking for, such as setting a timer or sending a message, and which details the action needs. Voice assistants and voice agents rely on it before they act.
- Large language model (LLM)A large language model (LLM) is a language model with a very large number of parameters, trained on huge amounts of text to predict and generate language. Google's glossary defines the minimum as a language model having a very high number of parameters.
- Machine learningMachine learning is a method in which a program learns its behavior from example data rather than from rules written by hand. Google's glossary defines it as a program or system that trains a model from input data.
- Prompt (AI)In AI, a prompt is the input you give a model to get a response, such as a question, an instruction or pasted text. Google's glossary defines it as any text entered as input to a large language model to condition the model to behave in a certain way.
- Speech language modelA speech language model (speech LLM) is a large language model that accepts audio as input, so it can process speech directly instead of working only on a text transcript. The 2023 SALMONN model is one example that lets an LLM process general audio inputs.
- Speech-to-speech modelA speech-to-speech model takes audio in and produces audio out with one model, without writing the conversation out as text in between. Most voice systems instead chain speech recognition, a language model and speech synthesis.
- Text to speech (TTS)Text to speech (TTS), also called speech synthesis, is software that converts written text into spoken audio. Modern systems use neural networks, and they can run on the device or in the cloud.
- Voice agentA voice agent is a voice system that listens, reasons with a language model, uses tools, and speaks back across several turns to finish a task, such as checking an order and rescheduling it. Benchmarks show that agents still fail many tasks on realistic audio.
- Voice assistantA voice assistant is software that listens for a spoken request, works out what you want, and then acts or answers, such as setting a timer or turning off a light. Dictation is different: it turns your speech into text for you to edit.
Pages that use this term
- Speech to Text AI: Dictation, Assistants and Voice AgentsTopic
- Voice Assistants vs Dictation: Siri, Alexa, GeminiGuide
- AI Voice Agents: How They Work and Where They FailGuide
- How do you connect speech to text to an LLM?Question
- How fast does a voice AI agent need to respond?Question
- Which speech to text should you pair with an LLM?Question
- Speech-to-speech modelGlossary
- Voice agentGlossary
- Wake wordGlossary