Preprint
Careless Whisper: Speech-to-Text Hallucination Harms
Allison Koenecke et al., arXiv (ACM FAccT 2024), 2024
Read the originalRead on October 16, 2026
- What it is
- A study of OpenAI's Whisper on recordings of speakers with aphasia and a control group, measuring how often it invents phrases that were never spoken and what those inventions say.
- How much weight it carries
- Peer-reviewed at FAccT 2024 by researchers from Cornell, the University of Washington and others. It tested one Whisper version in 2023 on one speech dataset, so rates may differ for newer versions and other audio.
Facts taken from this source
- The study's thematic analysis found that 38 percent of Whisper's hallucinations included explicit harms such as perpetuating violence, making up inaccurate associations or implying false authority.
“38\% of hallucinations include explicit harms such as perpetuating violence, making up inaccurate associations, or implying false authority” (Abstract)
- The Careless Whisper study evaluated Whisper as it stood in 2023, so its 1 percent hallucination figure describes that version of the model.
“We evaluate Open AI’s Whisper, a state-of-the-art automated speech recognition service outperforming industry competitors, as of 2023.” (Abstract)
- In a 2024 study, roughly 1 percent of Whisper audio transcriptions contained entire hallucinated phrases or sentences that did not exist in the audio.
“roughly 1\% of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio” (Abstract)
Pages that rest on this source
- Speech Recognition: What It Is and How Well It WorksTopic
- Transcription: How to Turn Audio and Video Into TextTopic
- Speech to Text AI: Dictation, Assistants and Voice AgentsTopic
- Is Whisper AI Safe? Privacy and Security ExplainedGuide
- Legal Dictation Software for Lawyers and Law FirmsGuide
- Dictation for Journalists and Content CreatorsGuide
- Why does Whisper make up words nobody said?Question
- Meeting Transcription: How It Works and What to UseGuide
- Video to Text: Transcribe Videos and YouTubeGuide
- How Accurate Is Speech Recognition? The Real NumbersGuide
- Speech to Text and LLMs: What Language Models ChangeGuide
- Speech to Text Without AI: Dictation With No AI RewriteGuide
- How to Run Whisper Locally on Your Own ComputerGuide
- Transcribe Audio With Whisper, ChatGPT or GeminiGuide
- AI Voice Agents: How They Work and Where They FailGuide
- How do you translate a video into English subtitles?Question
- How do you turn off AI rewriting in a dictation app?Question
- Dictation for Work: Doctors, Lawyers, Writers, StudentsTopic
- Speech to Text Privacy: Where Your Voice GoesTopic
- How Speech Recognition Works, From Sound to TextGuide
- How to Transcribe Audio to TextGuide
- What Is Whisper AI? OpenAI's Speech Model ExplainedGuide
- Speech-to-Text APIs Compared: Price, Accuracy, SpeedGuide
- Best Dictation Apps for iPhone and AndroidGuide