Fact · Speech recognition
13.88percent word error rate
A 2026 study says published Swiss German speech recognition results are inflated by benchmark contamination, because a Whisper model trained on the test set with no Swiss German data reached 13.88 percent WER.
Checked against its source on October 17, 2026
In the source's own words
“published state-of-the-art Swiss German ASR results (17.1-17.5% WER) are inflated by benchmark contamination: a vanilla Whisper model self-trained on the ASGDTS test set with zero Swiss German data achieves 13.88% WER, surpassing all published systems”
Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6% WER, Felix Akeret, arXiv (Computation and Language), 2026, Abstract.
Open the original- What it measures
- The example shows how a test set can be learned and then scored well, without any real gain in the language itself.
- What it does not show
- A single preprint on one benchmark. The 13.88 percent figure is the contaminated baseline the author trained, not a new model for dictation.