Japanese, Chinese and Korean Speech to Text

When you type Japanese or Chinese on a keyboard, you do not type the characters themselves. You type a spelling of the sound, and an input method converts it into characters. Dictation skips that conversion. The recognizer outputs kanji, hanzi or hangul directly. In a controlled phone study, Mandarin entered by speech ran at 123 words per minute, against 43 with a pinyin keyboard ^{1 2}.

The errors also change. The main risk is choosing which written form a sound should become, and Japanese and Chinese are usually measured in characters rather than words. This guide covers how dictation works in each language, the built-in options, and the published accuracy figures, with the sources for each number.

In Japanese, Chinese and Korean, dictation skips the slowest part of typing: character conversion

Most people who type Japanese or Chinese already do a conversion step. Japanese typists enter kana or romaji and then choose the kanji. Chinese typists enter pinyin and pick the hanzi from a list. Korean typists build hangul syllables from letters as they type. Each step takes time, and the choices take attention.

Dictation replaces that whole sequence with one step: you say the sentence and the recognizer returns the written text. The chart below shows the gap in the controlled study of Mandarin, where the participants used a phone.

The same study also measured English ³. It used short messages ¹ and an older recognizer, so treat the gap as a floor for modern engines rather than a precise number ⁴.

Each language has its own input method, which dictation replaces:

Speech goes straight to kanji, hanzi and hangul, so homophones are the main error

A homophone is a word that sounds the same as another word but differs in meaning or spelling ⁵. Japanese has many. The sound kōka can be written 効果 (effect), 高価 (expensive) or 硬貨 (coin). A keyboard user chooses one of these from a list. A dictation engine has to choose one from the sound and the context, and it sometimes picks the wrong one.

Mandarin has the same problem on a larger scale, because many syllables map to several hanzi. The recognizer must split the audio into words, and the boundaries between words are not marked in Mandarin writing. An early study of Mandarin word boundaries found that native speakers agreed on where the boundaries fall only about 75 percent of the time ⁶. That is one reason researchers score Mandarin by characters.

Korean has a different main error. Its sounds map cleanly to hangul, so the hard part is spacing and the spelling of borrowed words, covered in the Korean section below.

In practice, check the words your dictation produces most often. A recognizer that gets the sound right but picks the wrong homophone looks accurate at first glance, so read each dictated sentence before you send it.

Accuracy is measured in characters, not words

Word error rate counts a mistake per word, which works for English because words are separated by spaces. Japanese and Chinese have no spaces between words, so a word error rate depends on how the text is split, and the split is itself uncertain. Researchers therefore count characters. The character error rate (CER) is the edit count for characters: the minimum number of insertions, substitutions and deletions needed to turn the recognizer's output into the reference text. The 2024 Mandarin stuttering challenge scores it this way ⁷.

The Whisper paper, which describes an open speech model, states this choice in its scoring method. It puts a space between every letter for five languages, which makes the score a character error rate for them ⁸. The five are Chinese, Japanese, Thai, Lao and Burmese.

Korean is not on that list ⁹. Korean writes spaces between words, so the paper scores it by word. The benchmark FLEURS adds a further complication: the authors note that Japanese and Korean spacing depends on the transcriber ¹⁰. A word error rate for Korean therefore also counts spacing decisions.

When you compare two numbers, check the metric first. A character error rate of 7 percent and a word error rate of 7 percent measure different things, and they are not comparable.

Japanese: built-in dictation, Whisper and Japanese-tuned models

Apple. On the Mac, Apple's feature page lists Japanese (Japan) under On-Device and Modeless Dictation ¹¹. The same page lists Japanese for Auto-Punctuation, which inserts commas and full stops as you speak ¹². Apple's page notes that these on-device languages require a download of speech models ¹³. Check the Keyboard settings on your own Mac, because the lists change between macOS releases.

Windows. Microsoft describes Windows voice typing as online speech recognition powered by Azure Speech ¹⁴. It also needs an internet connection, a working microphone and the cursor in a text box ¹⁵. If you dictate Japanese on a Windows PC without a connection, the built-in option will not work, and you will need an offline app.

Cloud and open models. Google Cloud's Speech-to-Text lists Japanese (Japan) with the code ja-JP for its chirp_2 model ¹⁶. Whisper's language table includes Japanese with its own code ¹⁷, so Whisper can transcribe it on your own computer.

How accurate are these models? The published numbers come from research papers, and they are the most reliable guide we have. The table below collects the ones we could open. They use different test sets, so compare the rows only roughly.

The first three rows show what size and training do for a small model. The Whisper-Tiny result drops by more than half after training on Japanese data ¹⁸, and the fine-tuned tiny model beats the untuned base model, 14.7 percent ¹⁸ against 20.2 percent ¹⁹. The last row is a harder test: spontaneous speech, with fillers and hesitations, not the short, deliberate sentences of everyday dictation. A large model's score on that test does not predict its score on your messages.

For typing speed in Japanese, the best device comparison we found is an older study of 24 university students ²¹. On a two-minute copying task, they typed Japanese at 23 words per minute on desktop PCs ²¹ and at 17 on cellphones ²². The study is dated and small, so it shows the shape of the comparison, not current speeds. The question on typing speed covers what is known about current figures.

Chinese: Mandarin is well supported, Cantonese and other varieties less so

Apple. Apple's on-device list for Mac Dictation includes Mandarin Chinese (China mainland) ²³. Its general Dictation list also includes Mandarin (Taiwan) ²⁴. Cantonese appears in the on-device list for Hong Kong and for Simplified Chinese on the mainland ²⁵. For punctuation, Apple lists Chinese (Simplified), Chinese (Traditional) and Cantonese (Taiwan) as separate entries for Auto-Punctuation ²⁶.

The practical point is that the on-device list includes Mandarin Chinese (China mainland) but not Mandarin (Taiwan) ²³. If you write for Taiwan or Hong Kong, check each variant in your own settings.

Measured accuracy. One open benchmark for Mandarin is AISHELL-1. Its best 2017 baseline, a model called LFMMI, reached a character error rate of 7.62 percent on the test set ²⁷. That is a read-speech benchmark from an older model, so it shows the floor of what is possible, not what a current dictation tool will do.

Cantonese and other varieties. We did not find a published character error rate for Cantonese dictation in the sources we opened, so we cannot rank it against Mandarin. Apple's lists show Cantonese support ²⁵, which is a different thing from a measured accuracy. If your work is in Cantonese, test a few paragraphs of your own speech before relying on it.

Voice input compared with typing. In the Mandarin study, the phone speech speed was 123 words per minute, 2.87 times the 43 of the pinyin keyboard ¹. The question on whether speaking Chinese is faster than typing pinyin covers that comparison in more detail.

Korean: word spacing and loanwords are where dictation slips

Accuracy. The KoSpeech toolkit, an open-source Korean recognizer, reached a character error rate of 10.31 percent on the KsponSpeech corpus of spontaneous Korean speech, using the acoustic model alone ²⁸. That is a 2020 baseline without a language model, so current systems are likely better, but no newer Korean figure from an open paper is in the sources we opened.

Spacing. Korean writes spaces between words, and the spacing rules leave room for choice. The FLEURS authors note that Korean transcripts may or may not contain spaces, depending on the transcriber ¹⁰. A dictation engine chooses its own spaces, so you may see text that is correct in its letters but spaced differently from the style you want. Because the Whisper paper scores Korean by word, a spacing choice counts as an error in that measure ⁹.

Loanwords. Many modern Korean words are borrowed from English and written in hangul. A recognizer may write a borrowed term with a spelling that is not the one you expect, or split it into two words. Read proper nouns and technical terms before you send the text.

Built-in support. Apple's Mac Dictation lists Korean (Korea) under On-Device and Modeless Dictation and under Auto-Punctuation ^{11 12}. Google Cloud's Speech-to-Text lists Korean (South Korea) with the code ko-KR for its chirp_2 model ²⁹. Whisper's language table includes Korean ¹⁷.

For a guide to the Korean question, see how to voice type in Korean.

Punctuation by voice produces full-width marks

Japanese and Chinese text uses full-width punctuation. The comma and full stop are wider than the half-width marks used in English and have their own characters. A dictation engine with automatic punctuation should insert these marks in a Japanese or Chinese context, not the English ones, so check the marks in your first test.

Apple lists Japanese (Japan) and Korean (Korea) among the languages with Auto-Punctuation on the Mac ¹², and lists Chinese (Simplified) and Chinese (Traditional) as separate entries ²⁶. For the setting itself, see automatic punctuation.

Spoken punctuation commands, such as saying a word for a comma, depend on the language and the app. Check the command list in your dictation settings before you dictate a long document, because command words that work in English may not exist in Japanese or Chinese.

Choose the built-in option first, then test it on your own speech

For a short message in Japanese, Chinese or Korean, the built-in option on your device is usually enough, provided it is on the language list for your region. For a long document, or for work where a homophone error changes the meaning, you should measure the result yourself.

Three steps are enough to check:

1. Read a passage of about 200 characters aloud. Use the text you normally write, not a sample from a textbook.
2. Count the characters that differ from your intended text. Count substitutions, extra characters and missing characters, then divide by the number of characters in your passage.
3. Repeat with the setting you will use. Test once with the on-device setting, and once with online dictation if your device offers both, because the two can differ.

This is a rough character error rate, not a published benchmark. It will tell you more about your speech and your vocabulary than any average number in this guide.

If you are choosing a dictation app for Japanese, Chinese or Korean, read our overview of the languages dictation apps cover and compare the Mac and Windows guides for Mac dictation languages and Windows dictation languages. Yap, which we make, runs dictation on the device for Mac and Windows, so the on-device status above matters for that choice as well. Our Mac dictation language help shows how to switch languages.

Four terms carry the numbers in this guide

• Character error rate counts the edits needed to correct a recognizer's text, per character, and is the usual metric for Japanese and Chinese. See character error rate.
• Input method is the software that turns keystrokes into characters, such as a pinyin or kana converter. See input method editor.
• Pinyin is the romanized spelling of Mandarin sounds used on Chinese keyboards. See pinyin.
• Word error rate counts errors per word and is the usual metric for English. See word error rate.

Related guides in this section cover multilingual dictation, speech recognition accents, Whisper's supported languages and Spanish speech to text. For the metric itself, see word error rate and how it is measured.

Each common question has a short answer on its own page

• What is the best speech to text for Japanese?
• How do you voice type in Chinese?
• Is speaking Chinese faster than typing pinyin?
• How fast do people type in Japanese?
• How do you voice type in Korean?

Back to the overview of dictation and speech to text in other languages.