Glossary
Real-time factor (RTF)
The real-time factor (RTF) is the time a system takes to process audio divided by the length of that audio. An RTF of 1 or lower means the system keeps up with real-time speech. Some leaderboards report the inverse, RTFx, where higher is faster.
Updated October 17, 20264 cited facts, each traced to its source
Also called RTF, RTFx, inverse real-time factor.
The real-time factor (RTF) measures how fast a speech model processes audio. It is the processing time divided by the length of the audio. Microsoft's guide to running speech models on a device defines it this way: the ratio of the processing time to the audio length 1. The number is a measure of computing cost, not of accuracy. A model can have an excellent RTF and a poor word error rate, or the reverse.
Lower RTF means faster processing
Because the ratio divides processing time by audio length, a smaller number is faster. Microsoft's worked example makes this concrete: if a device processes a one-minute audio file in 30 seconds, the RTF is 0.5 2. The device spends half the audio's duration on the work.
The threshold for live speech is 1. Microsoft says a device should have an RTF of 1 or lower to support real-time and interactive speech experiences, and an RTF above 1 means the device cannot keep up with the audio 3. Below 1, the system can finish each chunk before the next one arrives. Above 1, a queue of unprocessed audio builds up, and the text lags further behind the speaker.
RTFx is the inverse, and higher is faster
Many benchmark tables report the inverse instead. The Open ASR Leaderboard reports the inverse real-time factor (RTFx) rather than the real-time factor 4. Under that convention, an RTFx of 100 means the system processes 100 seconds of audio for every second of computing. The two numbers carry the same information, but they point in opposite directions. Always check which one a table uses before comparing values. Our guide to speech recognition benchmarks explains how the leaderboards are built.
Why it matters for dictation
For dictation, RTF decides whether the words appear while you speak or only after you stop. A model that is too slow on your hardware will lag, and the lag grows with the length of the sentence. Microsoft recommends measuring RTF on files rather than live microphone input, and taking several samples so you can see the spread 1. The figure is device-specific: the same model can have a very different RTF on a laptop and on a phone. Our guide to real-time speech recognition covers streaming and latency, which are related but different measures.
Related terms
- Word error rate (WER): the accuracy measure that usually appears beside RTF in benchmarks.
- Automatic speech recognition (ASR): the task whose speed RTF measures.
- Voice activity detection (VAD): decides when to start and stop processing audio.
Guides that cover it
Evidence
Every number on this page, with where it comes from. Open a fact to see the exact words in its source.
- 1.
- 2.
- 3.
- 4.
Related terms
- Speech Recognition: What It Is and How Well It WorksThe topic this term belongs to
- Acoustic modelAn acoustic model is the part of a speech recognizer that scores how well a short window of audio matches the sounds of speech. Since about 2012 most acoustic models are deep neural networks, and their training data strongly affects accuracy.
- Automatic speech recognition (ASR)Automatic speech recognition (ASR) is software that converts spoken audio into written words. Modern systems are neural networks trained on large amounts of transcribed speech, and they run in the cloud or on the device.
- ConformerA Conformer is a neural encoder for speech recognition that combines self-attention, which sees the whole input, with convolution, which captures local sound patterns. FastConformer is NVIDIA's faster variant of the same design, used in streaming models.
- Connectionist temporal classification (CTC)Connectionist temporal classification (CTC) is a way to train a neural network to turn audio into text without knowing in advance which moment of sound matches which letter. The network labels every short slice of audio with a character or a blank, and the blanks and repeated labels are then removed to leave the transcript.
- Hidden Markov model (HMM)A hidden Markov model (HMM) is a statistical model in which a sequence of hidden states produces the observations you can measure, and each state changes to the next with set probabilities. In speech recognition, the hidden states are units of sound and the observations are short frames of audio.
- Inverse text normalization (ITN)Inverse text normalization (ITN) is the step that converts a speech recognizer's spoken-form output into written form, so that 'two hundred' becomes 200. It changes how the text reads, not which words the recognizer heard.
- Language model (in speech recognition)In speech recognition, a language model (LM) scores how probable a word sequence is in a language. The recognizer combines that score with the acoustic evidence to choose between words that sound alike, such as homophones.
- LibriSpeechLibriSpeech is a free corpus of about 1,000 hours of read English speech, taken from LibriVox audiobooks, and it is one of the most widely used benchmarks for English speech recognition. Its scores describe clean, read speech, which differs from everyday dictation.
- Mel spectrogramA Mel spectrogram is a grid of sound energy over time, with frequency bands spaced to match human hearing. Many speech recognizers compute its logarithm, the log-Mel spectrogram, and feed it to a neural network instead of the raw waveform.
- RNN transducer (RNN-T)An RNN transducer (RNN-T) is a neural speech model that reads audio in order and emits words as it goes, so text can appear while the speaker is still talking. It has an encoder for the sound, a prediction network for the words already written, and a joint network that chooses the next output.
- Speaker diarizationSpeaker diarization is the task of working out who spoke when in a recording, by splitting the audio into turns and grouping turns that come from the same voice. It is separate from transcribing the words, and a transcript gains speaker labels only when diarization is run alongside recognition.
- Voice activity detection (VAD)Voice activity detection (VAD) is a component that decides whether a stretch of audio contains speech. Speech recognizers use it to split audio into turns, skip silence and decide when a speaker has finished.
Pages that use this term
- Speech Recognition: What It Is and How Well It WorksTopic
- Speech-to-Text APIs Compared: Price, Accuracy, SpeedGuide
- Speech Recognition Models: How Neural ASR WorksGuide
- Human vs AI Transcription: Accuracy, Cost, SpeedGuide
- Open-Source Speech Recognition: The Best ModelsGuide
- Speech Recognition Benchmarks and LeaderboardsGuide
- Real-Time Speech Recognition: Streaming and LatencyGuide
- Can Whisper transcribe in real time?Question
- Can you run voice recognition software on a Raspberry Pi?Question
- How long does it take to transcribe one hour of audio?Question
- What are the best alternatives to Whisper?Question
- What is NVIDIA Parakeet, and is it better than Whisper?Question
- What is the fastest speech-to-text API?Question
- Automatic speech recognition (ASR)Glossary
- RNN transducer (RNN-T)Glossary
- Voice activity detection (VAD)Glossary
- Word error rate (WER)Glossary