Fact · Speech recognition
11.34percent word error rate
In our October 2026 benchmark on a Mac with an M2 Pro, Whisper large-v3-turbo had a word error rate of 11.34 percent on 100 Croatian recordings from the FLEURS test set.
Measured by us on October 19, 2026, from 100 recordings
Our own measurement
This number comes from our benchmark, not from another publication. The full record (the setup, every recording's checksum, every transcript and timing) is published, so you can recompute it.
- What it measures
- OpenAI Whisper large-v3-turbo, 5-bit quantized (ggml-large-v3-turbo-q5_0.bin), run through whisper.cpp 1.9.4 with Metal on the GPU, with the language given. The recordings are read sentences (FLEURS, Google, 2022), selected by a fixed rule before any engine ran. Run on macOS 26.6.2, 16 GB of memory.
- How it is measured
- Word error rate = 217 word errors / 1,913 reference words = 11.34 percent, over 100 recordings; the engine's text and FLEURS's reference sentence are normalized the same way (NFKC, lowercase, words as runs of letters and digits).
- What it does not show
- Read Wikipedia sentences in a quiet setting, scored without any extra text normalization for numbers or spelling variants, so a correct '25' against a reference 'twenty-five' counts as an error for every engine alike. Dictated speech, accents and noise differ.
In the data
A row of On-Device Speech Recognition Benchmark on a Mac, as Whisper large-v3-turbo, Croatian: 11.34 percent.
- Engine
- Whisper large-v3-turbo
- Maker
- OpenAI
- Language
- Croatian
A row of Speech Recognition Accuracy: Word Error Rates, as Whisper large-v3-turbo, Croatian (our test): 11.34 percent.
- System
- Whisper large-v3-turbo
- Kind
- Open model
- Test set
- FLEURS
- Language
- Croatian
- Speech
- Clean read speech
- Year
- 2026
Used to calculate
- The speech recognition word error rates dataset holds 597 data points, each a published figure traced to its source.
- The speech recognition word error rates dataset draws on 32 distinct sources.
- In the on-device speech recognition word error rates on FLEURS dataset, the 11 published figures for “Whisper large-v3-turbo” have a median of 5.08 percent, ranging from 2.53 to 23.51.
- The on-device speech recognition word error rates on FLEURS dataset holds 26 data points, each a published figure traced to its source.