Fact · Speech recognition
15.87percent word error rate
In our October 2026 benchmark on a Mac with an M2 Pro, Whisper large-v3-turbo had a word error rate of 15.87 percent on 100 Bulgarian recordings from the FLEURS test set.
Measured by us on October 19, 2026, from 100 recordings
Our own measurement
This number comes from our benchmark, not from another publication. The full record (the setup, every recording's checksum, every transcript and timing) is published, so you can recompute it.
- What it measures
- OpenAI Whisper large-v3-turbo, 5-bit quantized (ggml-large-v3-turbo-q5_0.bin), run through whisper.cpp 1.9.4 with Metal on the GPU, with the language given. The recordings are read sentences (FLEURS, Google, 2022), selected by a fixed rule before any engine ran. Run on macOS 26.6.2, 16 GB of memory.
- How it is measured
- Word error rate = 357 word errors / 2,250 reference words = 15.87 percent, over 100 recordings; the engine's text and FLEURS's reference sentence are normalized the same way (NFKC, lowercase, words as runs of letters and digits).
- What it does not show
- Read Wikipedia sentences in a quiet setting, scored without any extra text normalization for numbers or spelling variants, so a correct '25' against a reference 'twenty-five' counts as an error for every engine alike. Dictated speech, accents and noise differ.
In the data
A row of On-Device Speech Recognition Benchmark: 25 Languages, as Whisper large-v3-turbo, Bulgarian: 15.87 percent.
- Engine
- Whisper large-v3-turbo
- Maker
- OpenAI
- Language
- Bulgarian
A row of Speech Recognition Accuracy: Word Error Rates, as Whisper large-v3-turbo, Bulgarian (our test): 15.87 percent.
- System
- Whisper large-v3-turbo
- Kind
- Open model
- Test set
- FLEURS
- Language
- Bulgarian
- Speech
- Clean read speech
- Year
- 2026
Used to calculate
- The speech recognition word error rates dataset holds 653 data points, each a published figure traced to its source.
- The speech recognition word error rates dataset draws on 38 distinct sources.
- In the on-device speech recognition word error rates on FLEURS dataset, the 25 published figures for “Whisper large-v3-turbo” have a median of 11.34 percent, ranging from 2.53 to 72.35.
- The on-device speech recognition word error rates on FLEURS dataset holds 56 data points, each a published figure traced to its source.