Fact · Languages and accents
37.0percent character error rate
On the Fleurs benchmark, the smallest Whisper model, tiny, had a character error rate of 37.0 percent for Japanese, against 5.3 percent for large-v2 on the same test.
Checked against its source on October 17, 2026
In the source's own words
“Model Croatian Hungarian Armenian Indonesian Icelandic Italian Japanese Javanese Georgian Kazakh Khmer Kannada Korean Luxembourgish ... Whisper tiny 79.0 83.8 118.6 51.7 113.3 29.8 37.0 107.3 123.0 165.2 100.6 100.7 36.1 99.1”
Robust Speech Recognition via Large-Scale Weak Supervision (full text), Alec Radford et al., OpenAI (arXiv), 2022, Table 13 (Fleurs), row Whisper tiny, column Japanese.
Open the original (DOI 10.48550/arXiv.2212.04356)- What it measures
- Table 13 lists all five Whisper model sizes on Fleurs. Japanese is scored by characters in the paper, as explained in the Japanese character error rate fact for large-v2. The tiny model is the smallest model in the family.
- What it does not show
- A single read-speech benchmark. Tiny models are much less accurate than the larger models in every language in this table, so the gap shows the cost of model size, not a fixed penalty.