Fact · Speech recognition
12.44percent word error rate
In our October 2026 benchmark on a Mac with an M2 Pro, Parakeet TDT 0.6B v3 had a word error rate of 12.44 percent on 100 Croatian recordings from the FLEURS test set.
Measured by us on October 19, 2026, from 100 recordings
Our own measurement
This number comes from our benchmark, not from another publication. The full record (the setup, every recording's checksum, every transcript and timing) is published, so you can recompute it.
- What it measures
- NVIDIA Parakeet TDT 0.6B v3 converted to Core ML (Parakeet Ultra, FluidInference, model revision 95eaa59a39d4), run through FluidAudio 0.17.4 on the CPU and GPU, with the language given. This is the engine the Yap app uses on Mac. The recordings are read sentences (FLEURS, Google, 2022), selected by a fixed rule before any engine ran. Run on macOS 26.6.2, 16 GB of memory.
- How it is measured
- Word error rate = 238 word errors / 1,913 reference words = 12.44 percent, over 100 recordings; the engine's text and FLEURS's reference sentence are normalized the same way (NFKC, lowercase, words as runs of letters and digits).
- What it does not show
- Read Wikipedia sentences in a quiet setting, scored without any extra text normalization for numbers or spelling variants, so a correct '25' against a reference 'twenty-five' counts as an error for every engine alike. Dictated speech, accents and noise differ.
In the data
A row of On-Device Speech Recognition Benchmark: 25 Languages, as Parakeet TDT v3, Croatian: 12.44 percent.
- Engine
- Parakeet TDT v3
- Maker
- NVIDIA
- Language
- Croatian
A row of Speech Recognition Accuracy: Word Error Rates, as Parakeet TDT v3, Croatian (our test): 12.44 percent.
- System
- Parakeet TDT 0.6B v3
- Kind
- Open model
- Test set
- FLEURS
- Language
- Croatian
- Speech
- Clean read speech
- Year
- 2026
Used to calculate
- The speech recognition word error rates dataset holds 653 data points, each a published figure traced to its source.
- The speech recognition word error rates dataset draws on 38 distinct sources.
- In the on-device speech recognition word error rates on FLEURS dataset, the 25 published figures for “Parakeet TDT v3” have a median of 10.11 percent, ranging from 2.52 to 31.66.
- The on-device speech recognition word error rates on FLEURS dataset holds 56 data points, each a published figure traced to its source.