Whisper or Parakeet: which is more accurate?

Parakeet TDT v3 is slightly more accurate overall, and Whisper large-v3-turbo is better in several languages. Across the languages each engine supports, Parakeet has a median word error rate of 10.11 percent ¹, and Whisper has 11.34 percent ². The gap is about one percentage point, which is small next to the gaps between languages. If you dictate in one language, look up that language below instead of trusting the median.

Both engines ran on the same Mac, on the same 100 recordings per language from the FLEURS test set, with nothing sent over the network. Our on-device benchmark dataset holds 56 data points ³, each traced to its source, and the transcripts are published.

The median hides a split: each engine wins in different languages

The medians are close because each engine wins roughly where the other loses. Whisper has the lower error rate in English (3.92 percent against 4.48 for Parakeet) ^{4 5} and in Spanish (2.53 against 2.79) ^{6 7}. Parakeet is ahead in German (4.84 against 5.08) ^{8 9}, in French (5.29 against 6.62) ^{10 11} and in Italian (2.52 against 2.73) ^{12 13}.

In those six large languages the differences are a few tenths to about one and a half points, so either engine gives a usable draft.

Parakeet is clearly better in the smaller European languages

The larger gaps appear in languages with less training audio. In Lithuanian Parakeet reaches 17.75 percent against 25.03 for Whisper ^{14 15}. In Estonian it is 14.17 against 19.91 ^{16 17}. The widest gap in the whole dataset is Maltese: Whisper had a word error rate of 72.35 percent ¹⁸, which means most words were wrong, while Parakeet had 22.02 percent ¹⁹.

Whisper is much better in Greek

The reverse happens in Greek. Parakeet had a word error rate of 31.66 percent ²⁰, the highest of any Parakeet language we measured, while Whisper had 11.96 percent ²¹. Whisper also leads in Danish, Finnish, Croatian, Polish, Russian and Swedish, by margins from a fraction of a point to about two and a half points. Both engines are close in Dutch, and Parakeet is ahead in Ukrainian, Czech, Slovak, Hungarian and Romanian.

Every row for each engine

Whisper large-v3-turbo, all languages we tested:

Parakeet TDT v3, all languages we tested:

What this means if you dictate in one language

• English, Spanish, Portuguese: Whisper has the lower error rate, by a margin you will rarely notice in daily use.
• German, French, Italian, Dutch and most Central and Eastern European languages: Parakeet is the safer choice.
• Greek: choose Whisper.
• Baltic languages, Slovenian, Maltese: error rates are high for both engines (Parakeet is the lower of the two in each). Expect to correct a noticeable share of words.

Accuracy is not the only difference. Speed and hardware needs differ too, which we cover in the question on the fastest on-device engine for a Mac.

What the data does not show

• Read sentences, not conversation. FLEURS is read speech recorded in quiet conditions. Spontaneous speech, accents and background noise usually raise error rates.
• One Mac, 100 recordings per language. A different sample or machine could move a figure by a few tenths of a point. Differences under one point between engines should be treated as a tie.
• Words only. Word error rate ignores punctuation and capital letters. Our dataset also holds a formatted error rate that counts them, and it is higher for both engines.
• One Whisper model. We tested large-v3-turbo. Other Whisper sizes behave differently, as covered in which Whisper model is best.

Related reading

For the full picture of how these engines work, see the guide to on-device speech recognition and our overview of speech recognition. The Apple comparison adds Apple's model, which languages work best ranks languages, and what NVIDIA Parakeet is explains the model. Speeds are in the processing speed dataset.