Which Whisper model should I use?

Use the turbo model for English. It is the best balance of accuracy, speed and memory for most people. It needs about 6 GB of video memory ¹, and its English error rate on a public leaderboard is close to the largest model's. Switch to medium or large when you need translation, and to a smaller model only when your computer cannot run turbo. The right size depends on three things: the language you speak, the task, and the memory you have.

Turbo gives up little English accuracy for much less memory

On the Open ASR Leaderboard's English short-form tests, OpenAI's Whisper Large v3 had an average word error rate of 7.44 percent, and Whisper Large v3 Turbo had 7.83 percent ^{2 3}. Turbo also ran faster on the same track, with an inverse real-time factor of 200 ³ against 146 for Large v3 ², which counts seconds of audio processed per second of compute.

The large model needs about 10 GB of video memory ⁴. Turbo fits in about 6 GB ¹, so it runs on many more graphics cards. For a graphics card with less memory than that, the smaller models are the next step, and our guide to running Whisper locally has the memory and speed of every size.

The leaderboard uses English short-form test sets. Your recordings can differ in noise, accent and speaking speed, so the gap you hear may be larger or smaller. The only way to know is to measure the word error rate on a few of your own files.

Choose medium or large for translation, because turbo cannot translate

Turbo is not the model for translation. The Whisper repository says the turbo model is not trained for translation tasks, and that you should use a multilingual model to translate non-English speech into English ⁵. Those multilingual models are tiny, base, small, medium and large.

For translation, use the largest multilingual model your memory allows. The Whisper README recommends the medium or large model for the best translation results ⁶. Large is the one to try first if your graphics card has the roughly 10 GB it needs ⁴. Medium is the step down: it needs about 5 GB of video memory ⁷ and runs about 2 times as fast as large ⁸. Test a short clip first, because performance varies widely by language ⁹.

English-only models help only the smallest sizes

The .en models are trained for English only. OpenAI's README says the English-only models tend to perform better for English, especially the smallest ones ¹⁰. The README adds that the difference becomes less significant for small.en and medium.en.

This matters if you run Whisper on weak hardware and transcribe only English. A tiny.en or base.en model can be a better choice than the multilingual model of the same size. If you ever need another language, switch to the multilingual model of that size.

Accuracy depends on the language, so a size that works in English may not work in yours

Accuracy changes sharply from one language to another. In the FLEURS benchmark, which reads texts aloud in good audio, the Whisper large-v2 model had a word error rate of 4.2 percent in English ¹¹. The same model reached 21.5 percent in Hindi ¹².

Those figures are for large-v2, not turbo, and they describe read speech in good audio. They do not tell you how turbo performs in Hindi, or how the models compare on your phone recordings. They suggest that a model's English score is a poor guide to other languages. If your language is not English, test the candidate models on a few minutes of your own audio before you commit.

What to do: test two sizes on your own audio

Work through these steps in order.

1. Start with turbo for English, or for any language when you do not need translation.
2. Transcribe five or ten minutes of your own audio with turbo, and again with the size one step larger if your hardware allows it.
3. Compare the two transcripts against a correct version you check by hand. Count the errors in each.
4. Keep the smaller model if its errors are acceptable for your use, since it runs faster and uses less memory.
5. Move to medium or large only for translation, or when the larger model's transcript is clearly better on your recordings.

If your computer cannot run turbo, read whether Whisper can run without a GPU before you change the model, because the processor route can change which sizes are practical. For the choice between implementations rather than sizes, see whisper.cpp versus faster-whisper. If you would rather not install anything, running Whisper in Google Colab gives a notebook route with a free GPU, within its limits.

For the full picture of the subject, see our speech recognition overview, and the guide to Whisper AI.