Glossary
Voice activity detection (VAD)
Voice activity detection (VAD) is a component that decides whether a stretch of audio contains speech. Speech recognizers use it to split audio into turns, skip silence and decide when a speaker has finished.
Updated October 17, 20267 cited facts, each traced to its source
Also called VAD, speech activity detection.
Voice activity detection (VAD) decides, for each short piece of audio, whether it contains speech or only silence and background noise. Speech recognizers use this decision in three ways: they skip silence to save computing, they mark where a spoken phrase begins, and they decide when the speaker has finished. Some APIs call the last job endpointing or turn detection. In all of these, the VAD sets the timing of the text you see.
How VAD splits speech into turns
Most real-time systems run a small VAD model on every short chunk of audio, separately from the large recognizer. A modern open model, Silero VAD, processes a chunk of 30 milliseconds or more in less than 1 millisecond on a single CPU thread 1. That speed is why VAD can run continuously at no noticeable cost.
Cloud APIs expose the same idea as settings. OpenAI's Realtime API uses periods of silence to split the audio into turns 2. A shorter silence setting means a turn is detected as ended more quickly 3. Google Cloud's voice activity timeouts must be greater than 500 milliseconds and less than 60 seconds 4, so a developer cannot make the system cut off faster than that.
A concrete example: the pause before a sentence ends
Say you dictate "Send the report to Dana" and pause for half a second to think. A short silence setting may end the turn there, and the recognizer finalizes the sentence before "by Friday" arrives, so the next phrase starts a new line. A long setting waits, and the text appears more slowly after you stop. The right value is a trade-off between cut-off errors and lag, so it is worth testing with your own speaking pace.
Why VAD matters for dictation
VAD sets how quickly your words settle into the document. OpenAI notes that a higher detection threshold requires louder audio to activate the detector, which may work better in noisy environments 5. A threshold that is too high can drop soft speech, while one that is too low can treat fan noise or a keyboard as speech.
Batch tools make a similar choice. The default voice activity filter in faster-whisper removes only silences longer than 2 seconds 6. Measurement is also tricky: Artificial Analysis defines time to first partial transcript from the moment its Silero detector finds the end of speech 7, so a benchmark's latency depends partly on the VAD it uses. Our guide to real-time speech recognition covers the latency side in more detail.
Related terms
- Automatic speech recognition (ASR): the recognizer that VAD feeds.
- Real-time factor (RTF): the speed measure for the whole system, including VAD.
- Word error rate (WER): the accuracy measure that a cut-off word will increase.
Guides that cover it
Evidence
Every number on this page, with where it comes from. Open a fact to see the exact words in its source.
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
Related terms
- Speech Recognition: What It Is and How Well It WorksThe topic this term belongs to
- Acoustic modelAn acoustic model is the part of a speech recognizer that scores how well a short window of audio matches the sounds of speech. Since about 2012 most acoustic models are deep neural networks, and their training data strongly affects accuracy.
- Automatic speech recognition (ASR)Automatic speech recognition (ASR) is software that converts spoken audio into written words. Modern systems are neural networks trained on large amounts of transcribed speech, and they run in the cloud or on the device.
- ConformerA Conformer is a neural encoder for speech recognition that combines self-attention, which sees the whole input, with convolution, which captures local sound patterns. FastConformer is NVIDIA's faster variant of the same design, used in streaming models.
- Connectionist temporal classification (CTC)Connectionist temporal classification (CTC) is a way to train a neural network to turn audio into text without knowing in advance which moment of sound matches which letter. The network labels every short slice of audio with a character or a blank, and the blanks and repeated labels are then removed to leave the transcript.
- Hidden Markov model (HMM)A hidden Markov model (HMM) is a statistical model in which a sequence of hidden states produces the observations you can measure, and each state changes to the next with set probabilities. In speech recognition, the hidden states are units of sound and the observations are short frames of audio.
- Inverse text normalization (ITN)Inverse text normalization (ITN) is the step that converts a speech recognizer's spoken-form output into written form, so that 'two hundred' becomes 200. It changes how the text reads, not which words the recognizer heard.
- Language model (in speech recognition)In speech recognition, a language model (LM) scores how probable a word sequence is in a language. The recognizer combines that score with the acoustic evidence to choose between words that sound alike, such as homophones.
- LibriSpeechLibriSpeech is a free corpus of about 1,000 hours of read English speech, taken from LibriVox audiobooks, and it is one of the most widely used benchmarks for English speech recognition. Its scores describe clean, read speech, which differs from everyday dictation.
- Mel spectrogramA Mel spectrogram is a grid of sound energy over time, with frequency bands spaced to match human hearing. Many speech recognizers compute its logarithm, the log-Mel spectrogram, and feed it to a neural network instead of the raw waveform.
- Real-time factor (RTF)The real-time factor (RTF) is the time a system takes to process audio divided by the length of that audio. An RTF of 1 or lower means the system keeps up with real-time speech. Some leaderboards report the inverse, RTFx, where higher is faster.
- RNN transducer (RNN-T)An RNN transducer (RNN-T) is a neural speech model that reads audio in order and emits words as it goes, so text can appear while the speaker is still talking. It has an encoder for the sound, a prediction network for the words already written, and a joint network that chooses the next output.
- Speaker diarizationSpeaker diarization is the task of working out who spoke when in a recording, by splitting the audio into turns and grouping turns that come from the same voice. It is separate from transcribing the words, and a transcript gains speaker labels only when diarization is run alongside recognition.
Pages that use this term
- Speech Recognition: What It Is and How Well It WorksTopic
- Open-Source Speech Recognition: The Best ModelsGuide
- Voice Assistants vs Dictation: Siri, Alexa, GeminiGuide
- How to Run Whisper Locally on Your Own ComputerGuide
- Transcribe Audio With Whisper, ChatGPT or GeminiGuide
- Real-Time Speech Recognition: Streaming and LatencyGuide
- AI Voice Agents: How They Work and Where They FailGuide
- How do you use Whisper to transcribe audio?Question
- How does speech recognition know when you stop talking?Question
- How fast does a voice AI agent need to respond?Question
- What is Kaldi speech recognition?Question
- Automatic speech recognition (ASR)Glossary
- False accept (accidental activation)Glossary
- Mel spectrogramGlossary
- Real-time factor (RTF)Glossary
- Speaker diarizationGlossary
- Turn detectionGlossary
- Wake wordGlossary