Is word error rate the same as accuracy?

Word accuracy is 1 minus WER, so the two numbers carry the same information

Word error rate (WER) counts the errors, and word accuracy counts the words that are right. Both come from the same three counts. WER is (S + D + I) divided by N, where S, D and I are the substitutions, deletions and insertions and N is the number of words in the reference. Word accuracy is (N - S - D - I) divided by N, which is exactly 1 minus WER. The guide to word error rate gives the formula with a worked example.

The equality holds only when accuracy is defined this way. Some tools define it differently, so the two words do not always mean the same thing in a report. The rest of this page explains the differences.

Accuracy can fall below zero because insertions count against it

Insertions are added to the numerator of WER, so a system that writes extra text can have a WER above 100 percent. The same arithmetic puts accuracy below zero. Morris, Maier and Green use a simple example: a system that outputs two wrong words for each input word has a WER of 200 percent ¹. Its word accuracy, 1 minus that WER, is -100 percent ².

A negative accuracy looks absurd, but it is a correct consequence of the formula. It also shows a weakness of both measures. Morris and colleagues note that both systems in their example communicate exactly zero information, yet WER still ranks one as worse than the other. The authors proposed other measures for that reason, which the next section covers.

Percent correct leaves insertions out, so it reads higher than accuracy

Many speech tools report "percent correct" instead of accuracy. The NIST sclite scorer, which many research papers use, is one example. It defines the percent of correct words as the number of correct words divided by the number of reference words, and it reports insertions as a separate percentage ³. Percent correct therefore ignores insertions completely.

The three numbers can differ a lot on the same output, so check which one a report shows:

A system that writes many extra words can therefore show a high percent correct and a low accuracy at the same time. When a vendor or paper reports "accuracy", look for the formula, or compute 1 minus the WER yourself from the counts.

The usual citation for WER is a 2004 review, and the definition is older

The 2004 review by Morris, Maier and Green is the standard citation for the definition of WER. It describes WER as the de facto standard index for ASR assessment, defined as the proportion of word errors to words processed ⁴. The review also proposes match error rate (MER) as an alternative absolute measure ⁵.

For the isolated word case, Morris and colleagues cite a 1982 workshop paper by Woodard and Nelson as the source of the WER definition ⁶. We did not read that 1982 paper for this page, so cite it only as the source Morris and colleagues name. When you cite WER in a report, cite the 2004 review, the scoring tool and its version, because those tell a reader which definition you used.

It depends on the report, the tool and the test

The equality of WER and accuracy is a convention, not a law of nature. Use these checks before you compare numbers:

• If a report gives "accuracy", check whether it is 1 minus WER. If it is percent correct, the number will read higher.
• If a report gives WER only, the accuracy is 1 minus WER, and it can be negative for verbose output.
• If the task is exact matching, such as a command that must be spoken perfectly, sentence-level accuracy is a different measure. The share of sentences with no error is not the same as word accuracy.
• If you average scores across test sets, weight them by reference length, so that a long recording counts for more than a short one. The guide to speech recognition accuracy explains why results vary across audio.

What to do

1. Ask which measure the report uses: WER, 1 minus WER, or percent correct.
2. Compute the three counts (substitutions, deletions and insertions) from your own alignment, so you can derive all three measures.
3. Report WER with the reference word count, the tool and its version, and the normalizer.
4. Say "accuracy = 1 minus WER" whenever you use the word, and avoid it when you mean percent correct.
5. Keep an eye on insertions. If your system adds words, the accuracy will fall faster than the error rate suggests.

For more on reading these numbers, see what counts as a good word error rate, how to calculate word error rate, whether word error rate can go over 100 percent and the difference between WER and character error rate. The term entry for word error rate gives the short definition.