Preprint
Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6% WER
Felix Akeret, arXiv (Computation and Language), 2026
Read the originalRead on October 17, 2026
- What it is
- A 2026 preprint that fine-tunes Whisper large-v3 for Swiss German and argues that published Swiss German results were inflated because their test data leaked into training. It re-scores the systems on a held-out split.
- How much weight it carries
- A single-author preprint that has not been peer reviewed. Its claim about published results is a critique of specific systems, so it should be read as one careful case, not a settled verdict on the whole field. The numbers come from one dialect benchmark.
Facts taken from this source
- A 2026 study says published Swiss German speech recognition results are inflated by benchmark contamination, because a Whisper model trained on the test set with no Swiss German data reached 13.88 percent WER.
“published state-of-the-art Swiss German ASR results (17.1-17.5% WER) are inflated by benchmark contamination: a vanilla Whisper model self-trained on the ASGDTS test set with zero Swiss German data achieves 13.88% WER, surpassing all published systems” (Abstract)