What are the best Whisper settings for accuracy?

For most recordings, the best settings are the defaults plus three deliberate choices: the largest model your hardware runs comfortably, the language set by hand, and a short prompt of names and jargon. Leave the temperature fallback and the no-speech threshold alone until a test on your own audio shows a problem. Each setting changes results, so test one change at a time on a short file with a known transcript.

Start with turbo, or large if your hardware has the memory

The turbo model is an optimized version of large-v3 that transcribes faster, with a minimal degradation in accuracy ¹. On the Open ASR Leaderboard's English short-form tests, Whisper Large v3 Turbo had an average word error rate of 7.83 percent ². That is a benchmark on standard test sets, not on your recordings. Large is the choice when you want the most accuracy and your card has the memory for it. On the same leaderboard track, Whisper Large v3 had an average word error rate of 7.44 percent ³, and our answer on the best GPU for Whisper covers the memory it needs.

Accuracy also depends on the language, so the same model can score very differently on different speech. Measure your own results with the word error rate, which compares the output with a correct transcript. Our answer on which Whisper model is best compares the models in more detail.

Set the language yourself

In openai-whisper, the language is left unset by default, and the program detects the spoken language itself ⁴. When you know the language, set it with --language, for example whisper japanese.wav --language Japanese. Setting it removes one thing the model has to guess, which is why it is the first setting to fix. If the output still comes out in the wrong language, see why Whisper picks the wrong language.

Add a short prompt for names and jargon

The --initial_prompt option can give the model custom vocabularies or proper nouns, which makes those words more likely to be transcribed correctly ⁵. Put the terms that matter, such as product names, people's names and technical words, in the prompt. Keep the prompt short and accurate, and check the result on a sample.

Keep the temperature fallback unless you see failures

When a decode fails its quality checks, Whisper retries with higher temperatures. The openai-whisper command line builds that list from its starting temperature, up to 1.0, in steps set by the fallback increment ⁶. Each retry adds time, so a run with fallback takes longer on difficult audio. Keep the default unless a test shows that a few segments are badly wrong.

Leave previous text on unless the output loops

The --condition_on_previous_text option feeds the previous output into the next window. The source says that turning it off can make the text inconsistent across windows, but makes the model less prone to getting stuck in a failure loop ⁷. Change it only if you see the same words repeated over and over.

Use the no-speech threshold to tune silence, not as a quick fix

The no-speech threshold defaults to 0.6 in the command line ⁸. A segment counts as silent when its no-speech probability is above the threshold and its average log probability is below the log-probability threshold ⁹. Raising the threshold skips fewer segments, which keeps more quiet speech but can let silence produce text. Lowering it does the reverse. Move it in small steps, and only if a test shows dropped speech or invented text. Our answer on why Whisper hallucinates covers the second problem.

Test one change at a time

1. Pick a clip of one or two minutes with a correct transcript.
2. Run it with the defaults and measure the word error rate.
3. Change one setting, run again, and compare.
4. Keep the change only if the error rate drops on your audio, and check a second clip before you rely on it.

The guide to running Whisper locally lists these settings alongside the install steps, and our overview of speech recognition covers accuracy across the subject.