Fact · Languages and accents
23.0percent word error rate
In Dolev and colleagues' tests, Whisper large-v3 used without any training on Swiss German had a word error rate of 23.0 percent on the STT4SG-350 test set.
Checked against its source on October 17, 2026
In the source's own words
“Whisper large-v3 zero-shot 0.230 63.1 This work”
Does Whisper Understand Swiss German? An Automatic, Qualitative and Human Evaluation, Eyal Liron Dolev et al., Proceedings of the 11th Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2024), 2024, Table 2, STT4SG-350 row.
Open the original- What it measures
- The reference text is Standard German. The table gives word error rate as a fraction, 0.230, which is 23.0 percent.
- What it does not show
- The test measures Standard German output against Swiss German speech. Verbatim dialect spelling would score higher.