Fact · Speech recognition
The Whisper authors chose an encoder-decoder Transformer as the architecture for their speech recognition models because they judged it well validated to scale reliably.
Checked against its source on October 17, 2026
In the source's own words
“We chose an encoder-decoder Transformer (Vaswani et al., 2017) as this architecture has been well validated to scale reliably”
Robust Speech Recognition via Large-Scale Weak Supervision (full text), Alec Radford et al., OpenAI (arXiv), 2022, Section 2, Approach.
Open the original (DOI 10.48550/arXiv.2212.04356)- What it measures
- The encoder reads the spectrogram and the decoder writes the text, one token at a time, in the same network.
- What it does not show
- A design choice reported by the authors for their own models. Other end-to-end systems use transducers or CTC heads instead.
Source
Cited on
- How Speech Recognition Works, From Sound to TextGuide
- What Is Whisper AI? OpenAI's Speech Model ExplainedGuide
- Speech to Text and LLMs: What Language Models ChangeGuide
- Speech Recognition Models: How Neural ASR WorksGuide
- Human vs AI Transcription: Accuracy, Cost, SpeedGuide
- Can large language models do speech recognition?Question
- Is speech recognition AI?Question
- Is Whisper a large language model?Question