Fact · Languages and accents
5.3percent character error rate
On the Fleurs benchmark, the Whisper large-v2 model had a character error rate of 5.3 percent for Japanese, because the paper scores Japanese by characters rather than words.
Checked against its source on October 17, 2026
In the source's own words
“Model Croatian Hungarian Armenian Indonesian Icelandic Italian Japanese Javanese Georgian Kazakh Khmer Kannada Korean Luxembourgish ... Whisper large-v2 13.4 17.0 44.6 7.1 38.2 4.0 5.3 nan 105.0 37.7 99.7 37.0 14.3 88.0”
Robust Speech Recognition via Large-Scale Weak Supervision (full text), Alec Radford et al., OpenAI (arXiv), 2022, Table 13 (Fleurs), column Japanese; Appendix text on character-level scoring.
Open the original (DOI 10.48550/arXiv.2212.04356)- What it measures
- Table 13 is the paper's Fleurs error table. The paper says it puts a space between every letter for Chinese, Japanese, Thai, Lao and Burmese, effectively measuring character error rate instead. The Japanese column is therefore character-level and cannot be compared with the word error rates for English or Spanish.
- What it does not show
- Read speech in good audio, from one benchmark. Character error rate counts single characters, so it is not the same measure as word error rate. Real Japanese dictation with accents, noise or specialist vocabulary will score higher.