Fact · Speech recognition
0of 84 readings
In our October 2026 test, Whisper large-v3-turbo left out a passage of 10 or more words in a row in 0 of 84 English readings of 3 to 4 minutes handed over cut at pauses into pieces of 10 to 20 seconds.
Measured by us on October 25, 2026, from recordings
Our own measurement
This number comes from our benchmark, not from another publication. The full record (the setup, every recording's checksum, every transcript and timing) is published, so you can recompute it.
- What it measures
- OpenAI Whisper large-v3-turbo, 5-bit quantized (ggml-large-v3-turbo-q5_0.bin), run through whisper.cpp 1.9.4 with Metal on the GPU, with the language given. 84 examples of 3 to 4 minutes read by 40 speakers from public-domain audiobooks (LibriSpeech-Long test-clean, Google DeepMind, from LibriSpeech), clean recordings of read English.
- How it is measured
- Aligning each transcript with the reference (minimum edit distance), the longest run of consecutive reference words with no word in the transcript; a reading counts when that run is 10 words or more. The longest such run in any reading: 3 words.
- What it does not show
- A run of left-out words can also be a passage the reader skipped in the audiobook while the transcript kept it; such a run would count against every engine alike.