Fact · Speech recognition
17.71percent formatted error rate
In our October 2026 benchmark, counting punctuation and capital letters as a reader sees them, Whisper large-v3-turbo's text differed from the written Bulgarian sentences by 17.71 errors per 100 tokens, against 15.87 for the words alone.
Measured by us on October 19, 2026, from 100 recordings
Our own measurement
This number comes from our benchmark, not from another publication. The full record (the setup, every recording's checksum, every transcript and timing) is published, so you can recompute it.
- What it measures
- OpenAI Whisper large-v3-turbo, 5-bit quantized (ggml-large-v3-turbo-q5_0.bin), run through whisper.cpp 1.9.4 with Metal on the GPU, with the language given. The reference is FLEURS's written sentence, with its punctuation and capitals. The gap to the plain word error rate is the editing a dictated text needs for punctuation and capitals alone.
- How it is measured
- Formatted error rate = 456 errors / 2,575 reference tokens = 17.71 percent, over 100 recordings; tokens are words with their capitals plus each sentence mark (. , ! ? ; :) on its own.
- What it does not show
- FLEURS's sentences follow their source's punctuation, and a comma can be optional, so part of the gap is style rather than error, for every engine alike. Numbers written as digits against words count as errors.
In the data
A row of Dictation Punctuation and Capitalization Accuracy, as Whisper large-v3-turbo, Bulgarian: 17.71 percent.
- Engine
- Whisper large-v3-turbo
- Maker
- OpenAI
- Language
- Bulgarian
- Words only (WER)
- 15.87
Used to calculate
- In the formatted accuracy of on-device speech recognition: punctuation and capitals counted dataset, the 25 published figures for “Whisper large-v3-turbo” have a median of 12.94 percent, ranging from 5.68 to 71.96.
- The formatted accuracy of on-device speech recognition: punctuation and capitals counted dataset holds 56 data points, each a published figure traced to its source.