How good is Whisper at Korean?

Whisper is usable in Korean, but it is clearly weaker than it is in English. On the FLEURS benchmark, the large-v2 model from 2022 had a word error rate of 14.3 percent in Korean ¹. Its English rate on the same test was 4.2 percent ², so Korean errors run about 3.4 times higher ³. Roughly one word in seven is wrong on read Korean. Expect more errors on everyday speech, and set the language yourself.

Korean is about 3.4 times worse than English on the same test

The ratio puts the gap in plain terms. Dividing 14.3 by 4.2 gives about 3.4, so a Korean word is about 3.4 times as likely to be wrong as an English word on this test ³. Korean is better than Hindi on the same benchmark, where the large-v2 model reached 21.5 percent ⁴, but it is still far from the English result.

The gap follows the training data. The paper estimates that a language's word error rate halves for every 16-fold increase in its training data ⁵. English has far more training audio than Korean in the model, so the higher Korean rate fits that estimate. The guide to Whisper's languages and accuracy sets Korean next to the other languages in the same test.

Korean is scored by word, which makes spacing matter

Korean is one of the languages where the score depends on how the transcript is set up. Korean spacing is inconsistent, and the FLEURS paper notes that Japanese and Korean transcripts may or may not contain spaces, depending on the transcriber ⁶. The benchmark had to handle that itself.

The Whisper paper scores Korean by word, not by character. Its list of languages scored by character is Chinese, Japanese, Thai, Lao and Burmese, and Korean is not on it ⁷. So the 14.3 percent figure counts whole words, and a missing or extra space can add an error even when the sounds are right. Read it as a fair guide to word accuracy, but not as a precise measure of spelling.

Automatic detection can miss Korean at the start

If you do not set a language, Whisper decides from the audio, and it looks at only the first 30 seconds to do so ⁸. A recording that opens with a short English remark or silence can be detected as the wrong language, and the rest of the recording is then transcribed that way. The question on why Whisper transcribes in the wrong language covers the common causes.

Mixed Korean and English speech can be harder again, since the model has to switch languages mid-sentence. Set the language to Korean for Korean recordings, and expect more errors on English words inside them.

What to do if you dictate or transcribe in Korean

1. Set the language to Korean rather than leaving detection on.
2. Use the largest model your app offers, and check whether the app uses a smaller default.
3. Test with two or three minutes of the speech you actually produce, such as meetings, lectures or dictation. Count the wrong words on a short passage rather than judging the whole transcript at a glance.
4. Check spacing and English words separately, because those are the two places where Korean output is most likely to differ from what a careful writer would type.

The guide to Japanese, Chinese and Korean speech to text covers the tools that handle these languages, and the question on how good Whisper is at Japanese gives the comparison for Japanese. For a different case, the question on Whisper's translation mode explains that Whisper translates only into English. The overview of languages and accents covers the wider subject.