Why doesn't voice recognition understand my accent?

Dictation often misreads accented speech because the system learned from speech that did not sound like yours. Google says that speech recognition is most often trained on typical speech, so people with heavy accents do not get the same benefit ¹. The gap can be measured. In a 2023 test on conversations between friends, the best English model averaged a word error rate of 19.7 percent across many accents ². Accent is not the only cause, though, and the other causes are often easier to fix.

The training data is the main reason, and you cannot change it

A system makes fewer errors on speech that resembles its training data. A 2020 study of commercial speech recognition systems traced the gap in error rates between speaker groups to the acoustic models those systems use ³. The acoustic model is the part that maps sound to speech units, so when it has heard little speech like yours, it guesses more often.

The gap also shows up in tests built to include accents. On Svarah, a 2023 benchmark of Indian English, the authors report a gap of 5.65 to 43.03 percentage points between the error rate on Svarah and on native English read speech, across the open source models they tested ⁴. Even the smallest gap in that range, 5.65 points, means every open source model in the test did worse on Indian English than on native English read speech.

It is not always your accent, and the evidence is mixed

The evidence does not always point to accent. In a study of an oral-history archive with noisy, elderly and accented speech, the authors found no evidence that accent or speaker age affected recognition performance ⁵. The same study found that signal-to-noise ratio and syllable rate, which measures how fast someone speaks, were the two dominant factors behind the overall error rate ⁶.

This matters for you. If your dictation is poor in a noisy room or when you speak quickly, the cause may be the room and the pace, not the accent. Test the same passage in a quiet room, at a steady pace, before you decide that accent is the problem.

The microphone and the room cause errors too

The effect of distance can be large. In a test with the same Whisper model, the single distant microphone recording of AMI meetings had about 2.15 times the word error rate of the headset recording ⁷. The same meeting was recorded both ways, so the difference comes from the microphone setup.

What to do: a short plan

1. Measure first. Dictate a passage of about 200 words, count the errors and divide by the number of words. That is your word error rate, and it gives you a baseline.
2. Fix the microphone. Use a headset or a close microphone, and dictate in a quiet space. Repeat the passage and compare.
3. Set the variant. Choose the regional variant closest to your speech. Trying a different variant costs nothing, so it is a quick test to run. The question on which English variant to use covers the choice.
4. Add your words. Names, product terms and local place names are the errors you will correct most often. Apple's Voice Control can import custom vocabulary from a text file with one term on each line ⁸, which is a practical way to keep the list.
5. Consider a voice profile. Some tools can adapt to your voice. The question on training dictation to your accent explains what your app may offer and what it does not change.

If you already know the tool is weak for your speech, compare two or three tools on the same passage. The question on which speech to text works best with an Indian accent explains that test in detail, with the figures for Indian English.

Where the evidence is thin

The studies above test a set of speakers rather than one person, and their averages hide individual differences. The published figures are for particular test sets, years and models. They show the direction of the problem, not the error rate you will get today. Your own passage is the only result that answers the question for you.

For the wider picture of the research, read our guide to speech recognition and accents. The languages overview links to the other guides in this series, including multilingual dictation.