What Is Whisper AI? OpenAI's Speech Model Explained

Whisper is a speech recognition model, not an app. OpenAI released it as open-source software in September 2022 ¹, and its code and model weights carry the MIT License ². That means you can download it for free, run it on your own computer, or pay to send audio to OpenAI's hosted API. Whisper turns recorded speech into text and can translate that speech into English. It does not label who is speaking, and it does not stream text as it hears audio.

Whisper is a free, open-source speech recognition model from OpenAI, not an app or a company

Whisper is the name of a set of learned model weights and the code that runs them. OpenAI's repository describes it as a general-purpose speech recognition model ³. Because it is a model, a person who wants to dictate usually needs one of three things: a program that runs the model, OpenAI's API, or an app built on top of it. Is Whisper AI free? The answer depends on which of these you choose, and the cost section below shows each path.

OpenAI's own voice features in ChatGPT are separate from the model. Apps that look similar to Whisper, and apps that use it underneath, are covered in our question on whether there is a Whisper app. Who owns Whisper is answered separately, because the model's authors and the companies that sell products built on it are different parties.

It learned from 680,000 hours of audio with transcripts, which made it robust across conditions

Whisper was trained on 680,000 hours of multilingual and multitask audio with transcripts ⁴. The scale is the point of the design. The authors report that, compared with humans, their models approach human accuracy and robustness ⁵.

The next section explains the model's structure, and the section after it explains where accuracy falls.

One encoder-decoder Transformer transcribes, translates and handles several audio tasks in one model

The model is an encoder-decoder Transformer. The authors chose this architecture because it "has been well validated to scale reliably" ⁶. In plain terms, the encoder reads the audio and the decoder writes the text, one token at a time.

One model handles several jobs. The paper says many different tasks can be performed on the same audio, and it names transcription, translation, voice activity detection and alignment among them ⁷. For a user, the two jobs that matter most are these:

• Transcription writes the speech in its original language.
• Translation writes English text from speech in another language, through the translations endpoint of OpenAI's API, which uses whisper-1 ⁸.

Check the model you choose before you rely on it for translation.

Several model sizes trade accuracy for speed, from tiny to large-v3 and turbo

The released models form a ladder. Bigger models have more parameters, need more memory, and run more slowly, but they make fewer errors on hard audio. The repository's model table lists these sizes:

Memory is the practical limit for most people. The large model needs about 10 GB of video memory ¹¹. Smaller models need less, so check the memory figure against your computer before you install anything.

Turbo sits outside this ladder. The repository describes it as an optimized version of large-v3 that transcribes faster, with minimal degradation in accuracy ¹². Our guide to running Whisper locally walks through setting it up.

Whisper is accurate on clean English but uneven across languages and recording conditions

On clean, read English, Whisper is accurate. On the LibriSpeech Clean test set, Whisper Large V2 had a word error rate (WER) of 2.7 percent ¹³. LibriSpeech is a standard benchmark of read audiobook speech, so this number describes the easy case.

Accuracy falls on harder audio. On the CHiME6 test set, the same model had a WER of 25.5 percent ¹⁴. A single clean-audio number therefore overstates what you will see on a noisy call or a meeting.

Language changes the result a great deal. OpenAI says Whisper's performance varies widely depending on the language ¹⁵. The chart uses the FLEURS benchmark results for Large V2, the figures in the paper.

English reached 4.2 percent ¹⁶, while Hindi reached 21.5 percent ¹⁷ and Swahili 39.3 percent ¹⁸. Accuracy differs widely between these languages, and a result for one language does not transfer to another. A dictation user who speaks a lower-resource language should test the model on their own speech before relying on it.

For how these numbers are measured and what a WER figure does and does not tell you, read our guide to speech recognition accuracy. The question of how Whisper compares with a cloud service from Google is covered in the question on whether Whisper is more accurate than Google Speech-to-Text, because the result depends on the test set.

Its documented weaknesses are invented text, no speaker labels and no native streaming

Invented text. The most serious documented weakness is hallucination, where the model writes words that were never spoken. In a 2024 study of one set of recordings, roughly 1 percent of Whisper's audio transcriptions contained entire hallucinated phrases or sentences that did not exist in the audio ¹⁹. For any use where a wrong sentence matters, such as medical or legal records, review the transcript against the audio before you rely on it.

No speaker labels. Whisper's transcripts do not say who is speaking. OpenAI's hosted guide reserves speaker labels for a separate model: gpt-4o-transcribe-diarize, which should be used only when you need to identify who speaks during different parts of a recording ²⁰. If you need labels, plan a speaker diarization step as well as transcription. Our question on whether Whisper can identify speakers covers the options.

No native streaming. Whisper's models are trained on 30-second audio chunks, and they cannot take longer audio as one input ²¹. Longer recordings are split and processed in windows, so you get text after each window rather than word by word as you speak. On OpenAI's hosted side, file streaming works with the newer transcription models, while whisper-1 does not support it ²². For the live case, see our guide to real-time speech recognition.

You can use Whisper three ways: run it yourself, call OpenAI's API, or use an app built on it

Run the open-source model on your own computer

The model is free to run, and the only costs are hardware and your time. The memory limits above decide which size you can run. Our guide to running Whisper locally has the full installation steps, and it explains how to test a model on your own recordings before you commit to one.

Call OpenAI's hosted API

Hosted transcription also has file size limits, so check them before you send long recordings. OpenAI bills transcription by the minute. Its price list shows gpt-transcribe at $0.0045 per minute ²³. Those are newer OpenAI models, and the guide lists whisper-1 as a separate model, so check the model name before you budget. Our comparison of speech-to-text APIs covers prices from several providers.

Use an app built on Whisper

Many apps run Whisper for you. Buzz describes itself as transcribing and translating audio offline on a personal computer ²⁴. Some apps send audio to a server, so the audio may leave your computer. Each app sets its own price, privacy policy and limits, so read those before you send sensitive audio through one.

If you want to skip setup, free online transcription of audio files runs in your browser. Yap, which we make, is an on-device dictation app for Mac and Windows, so you can dictate without running a model yourself. Our overview of the best dictation apps compares the options by need.

Newer open models from NVIDIA and others now beat Whisper on many English benchmarks

Whisper is no longer the most accurate open model on every English test. On the Open ASR Leaderboard's English short-form track, Whisper Large v3 had an average WER of 7.44 percent ²⁵. NVIDIA's Parakeet TDT 0.6B v2 had 6.05 percent ²⁶, and NVIDIA's Canary Qwen 2.5B had 5.63 percent ²⁷.

The speed gap is larger than the accuracy gap. Parakeet processes audio about 23 times faster than Whisper Large v3 on that leaderboard ²⁸. The leaderboard's normalized scores also strip punctuation and casing, so they measure the words rather than the presentation. Our guide to speech recognition benchmarks explains how these leaderboards are built.

The result is not uniform. In a 2025 9to5Mac test of one episode, Whisper Large V3 Turbo had a WER of 1.5 percent ²⁹, and NVIDIA's Parakeet v2 had 12.3 percent ³⁰. One episode is a small sample. Test the candidates on audio that sounds like yours before you choose.

Whisper only turns speech into text: it does not speak or generate voices

Whisper is a speech recognition model. Its repository describes it as a general-purpose speech recognition model ³, and it takes audio in and writes text out. It cannot read text aloud or produce a synthetic voice. Those tasks belong to text-to-speech, the reverse process, and people often confuse the two when they search for both. Speech to text and text to speech are different technologies, and a model built for one does not do the other.

For the basics of how the input becomes text, see how speech recognition works. The general term for this technology is automatic speech recognition.

Whisper powers several apps, which is why its name appears everywhere

Many transcription and keyboard apps use Whisper, so the name shows up in app listings and product pages. Some of those apps run the model on the device, and others send audio to a server. Other names that contain "Whisper" may be different products, so check the developer and the underlying model before you buy.

Two questions are worth reading in full. Is Whisper AI the same as Wispr Flow? explains how a model and a dictation product differ. Is there an official Whisper AI app? lists the options by platform.

Whisper suits offline transcription of recordings, and it struggles with live dictation and speaker labels

Whisper fits well when you have recordings to transcribe, you control the computer that runs the model, and you can wait for the results. Interviews, lectures and voice memos are common examples. Because the code is open, developers also build their own tools on it, which is one reason so many apps use the model. Running it on your own computer also keeps the audio on that machine, which matters for private recordings.

It fits less well when you need text as you speak. The model works on 30-second windows ²¹, so text appears after each window rather than at the cursor word by word. Dictating straight into a document, an email or a chat depends on that word-by-word feedback. For that job, check whether an app streams text to the place you are typing, and read our guide to real-time speech recognition. Speaker labels are the other gap. OpenAI reserves speaker identification for a separate model ²⁰, so a meeting with several people may need a separate diarization step on top of transcription, which adds setup and cost.

Language is the third limit. Accuracy depends on how much audio the model saw in your language, and the results above show how far that can vary. If your work mixes languages or uses a lot of specialist vocabulary, expect to correct more of the output.

Testing Whisper on your own audio tells you more than any published benchmark

Published benchmarks use their own recordings, so they can only suggest how a model will do on yours. A short test on your own audio is more useful, and it takes less time than most people expect. The steps are these:

1. Record a few minutes of the audio you will actually use, with the microphone, room and speaking style you use day to day.
2. Write a reference transcript by hand, or correct a machine transcript while listening to the audio.
3. Run each candidate model or app on the same recording, with the same settings, so the comparison measures the models rather than the settings.
4. Compare the word error rate (WER). Count the substituted, deleted and inserted words, and divide by the number of words in your reference transcript.
5. Check the errors that matter most to you, such as names, numbers, dosages or case citations, rather than relying on the overall score alone.

Audio quality changes results more than model choice for many people. A microphone close to your mouth in a quiet room helps any model, and the noisy-audio results above show what happens when those conditions are missing. Our guide to the best microphone for dictation covers how to improve the input before you compare models.

Where to go next

• For the wider subject, read the speech recognition overview.
• To compare the models you would run, read open-source speech recognition and speech recognition models.
• To see how the model fits a dictation workflow, read our guide to dictation on Mac, or how to use Whisper for dictation on a Mac.