Glossary
Conformer
A Conformer is a neural encoder for speech recognition that combines self-attention, which sees the whole input, with convolution, which captures local sound patterns. FastConformer is NVIDIA's faster variant of the same design, used in streaming models.
Updated October 17, 20264 cited facts, each traced to its source
Also called FastConformer.
A Conformer is a neural network design for speech recognition that combines two kinds of processing. Gulati and colleagues proposed "the convolution-augmented transformer for speech recognition, named Conformer" 1. It is an encoder: it turns the audio frames into a representation that a decoder then converts into words.
Attention sees the whole input, convolution sees the detail
A transformer uses self-attention, which lets every frame look at every other frame. That is useful for long-range context, such as knowing that a word is part of a sentence about a medical chart. Convolution works on a small neighborhood of frames, and it is good at local patterns such as the short burst of a consonant.
A Conformer block uses both. It runs a feed-forward module, then multi-head self-attention, then a convolution module, then a second feed-forward module, and it adds the two feed-forward outputs at half weight. The result is passed through a normalization step. The combination matters because speech needs both the local sound and the wider context.
The design lowers word error rates on standard tests
On the LibriSpeech benchmark, the Conformer model reached word error rates of 2.1 percent on test-clean and 4.3 percent on test-other without a language model, and 1.9 and 3.9 percent with one 2. Lower is better. Word error rate counts substitutions, deletions and insertions against a reference transcript, so a result of 2.1 percent means about two errors in every hundred words on that clean test set.
The Open ASR Leaderboard paper observes that Conformer-based encoders paired with transformer-based decoders achieve the best average word error rate among the systems it compared 3.
FastConformer: the same design, built for streaming
NVIDIA's FastConformer is a Conformer variant designed for speed. Its Nemotron Speech Streaming EN 0.6B model uses a cache-aware FastConformer encoder with 24 layers and an RNN-T decoder 4. "Cache-aware" means the model keeps what it has already computed from earlier audio, so it does not reprocess the whole recording each time new sound arrives. That is what makes streaming practical.
Why it matters for dictation
When you dictate, the words should appear quickly and be right the first time. The encoder decides how much the model understands from the sound, and the decoder decides how quickly the words are written. A Conformer encoder gives strong accuracy on clean speech, and a streaming variant lets the same design run as you talk.
Check the model card rather than the brand name. A product can say it uses a modern neural model without naming the encoder, and two products with Conformer encoders can still differ in language coverage, noise handling and speed.
Related terms
- RNN transducer (RNN-T): a decoder design often paired with a Conformer encoder for streaming.
- Connectionist temporal classification (CTC): an alternative output method that some Conformer models use.
- Word error rate (WER): the score used to compare these models.
- LibriSpeech: the benchmark corpus behind the results above.
Guides that cover it
Evidence
Every number on this page, with where it comes from. Open a fact to see the exact words in its source.
- 1.
- 2.
- 3.
- 4.
Related terms
- Speech Recognition: What It Is and How Well It WorksThe topic this term belongs to
- Acoustic modelAn acoustic model is the part of a speech recognizer that scores how well a short window of audio matches the sounds of speech. Since about 2012 most acoustic models are deep neural networks, and their training data strongly affects accuracy.
- Automatic speech recognition (ASR)Automatic speech recognition (ASR) is software that converts spoken audio into written words. Modern systems are neural networks trained on large amounts of transcribed speech, and they run in the cloud or on the device.
- Connectionist temporal classification (CTC)Connectionist temporal classification (CTC) is a way to train a neural network to turn audio into text without knowing in advance which moment of sound matches which letter. The network labels every short slice of audio with a character or a blank, and the blanks and repeated labels are then removed to leave the transcript.
- Hidden Markov model (HMM)A hidden Markov model (HMM) is a statistical model in which a sequence of hidden states produces the observations you can measure, and each state changes to the next with set probabilities. In speech recognition, the hidden states are units of sound and the observations are short frames of audio.
- Inverse text normalization (ITN)Inverse text normalization (ITN) is the step that converts a speech recognizer's spoken-form output into written form, so that 'two hundred' becomes 200. It changes how the text reads, not which words the recognizer heard.
- Language model (in speech recognition)In speech recognition, a language model (LM) scores how probable a word sequence is in a language. The recognizer combines that score with the acoustic evidence to choose between words that sound alike, such as homophones.
- LibriSpeechLibriSpeech is a free corpus of about 1,000 hours of read English speech, taken from LibriVox audiobooks, and it is one of the most widely used benchmarks for English speech recognition. Its scores describe clean, read speech, which differs from everyday dictation.
- Mel spectrogramA Mel spectrogram is a grid of sound energy over time, with frequency bands spaced to match human hearing. Many speech recognizers compute its logarithm, the log-Mel spectrogram, and feed it to a neural network instead of the raw waveform.
- Real-time factor (RTF)The real-time factor (RTF) is the time a system takes to process audio divided by the length of that audio. An RTF of 1 or lower means the system keeps up with real-time speech. Some leaderboards report the inverse, RTFx, where higher is faster.
- RNN transducer (RNN-T)An RNN transducer (RNN-T) is a neural speech model that reads audio in order and emits words as it goes, so text can appear while the speaker is still talking. It has an encoder for the sound, a prediction network for the words already written, and a joint network that chooses the next output.
- Speaker diarizationSpeaker diarization is the task of working out who spoke when in a recording, by splitting the audio into turns and grouping turns that come from the same voice. It is separate from transcribing the words, and a transcript gains speaker labels only when diarization is run alongside recognition.
- Voice activity detection (VAD)Voice activity detection (VAD) is a component that decides whether a stretch of audio contains speech. Speech recognizers use it to split audio into turns, skip silence and decide when a speaker has finished.