Fact · Languages and accents
29.1BLEU
The Whisper paper reports a zero-shot score of 29.1 BLEU on the CoVoST2 speech to English translation benchmark, without training on CoVoST2 data.
Checked against its source on October 17, 2026
In the source's own words
“We achieve a new state of the art of 29.1 BLEU zero-shot”
Robust Speech Recognition via Large-Scale Weak Supervision (full text), Alec Radford et al., OpenAI (arXiv), 2022, Speech translation results on CoVoST2.
Open the original (DOI 10.48550/arXiv.2212.04356)- What it measures
- This is the average over the CoVoST2 benchmark, and the paper notes that accuracy differs widely between languages. A BLEU score is not the share of words that are right.
- What it does not show
- The figure is an average over the CoVoST2 benchmark, not one language pair. The paper reports that results vary by language, so an average hides the weaker ones.