Fact · Speech recognition
The Careless Whisper study evaluated Whisper as it stood in 2023, so its 1 percent hallucination figure describes that version of the model.
Checked against its source on October 17, 2026
In the source's own words
“We evaluate Open AI’s Whisper, a state-of-the-art automated speech recognition service outperforming industry competitors, as of 2023.”
Careless Whisper: Speech-to-Text Hallucination Harms, Allison Koenecke et al., arXiv (ACM FAccT 2024), 2024, Abstract.
Open the original- What it measures
- The authors ran their tests in 2023. The paper's full text says the 1.4 percent figure for hallucinations came from experiments in April and May 2023.
- What it does not show
- OpenAI has changed the model since then, and the paper does not measure the current version.
Source
Cited on
- Transcription: How to Turn Audio and Video Into TextTopic
- Meeting Transcription: How It Works and What to UseGuide
- Video to Text: Transcribe Videos and YouTubeGuide
- How Accurate Is Speech Recognition? The Real NumbersGuide
- Speech to Text and LLMs: What Language Models ChangeGuide
- Speech to Text Without AI: Dictation With No AI RewriteGuide
- How to Run Whisper Locally on Your Own ComputerGuide
- Transcribe Audio With Whisper, ChatGPT or GeminiGuide
- AI Voice Agents: How They Work and Where They FailGuide
- How do you translate a video into English subtitles?Question
- How do you turn off AI rewriting in a dictation app?Question
- Why does Whisper make up words nobody said?Question