Peer-reviewed research

From WER and RIL to MER and WIL: Improved Evaluation Measures for Connected Speech Recognition

Andrew Cameron Morris et al., Proceedings of Interspeech 2004, ISCA, 2004

Read the original

Read on October 17, 2026

What it is: A 2004 paper that defines the word error rate formula, works through examples where WER gives misleading scores, and proposes match error rate (MER) and word information lost (WIL) as alternatives.
How much weight it carries: Peer-reviewed and widely cited in speech recognition evaluation. It is a 2004 study of connected speech recognition, so its examples predate modern neural recognizers, but the definitions and the alignment argument still hold.

Facts taken from this source

The 2004 review by Morris, Maier and Green cites a 1982 workshop paper by Woodard and Nelson as the source for the definition of WER for isolated word recognition.
“Woodard, J.P. & Nelson, J.T. "An information theoretic measure of speech recognition performance", Workshop on standardisation for speech I/O technology, Naval Air Development Center, Warminster, PA, 1982.” (References, item [10])
Morris et al. state that the usual alignment, with zero weight for hits and equal weight for substitutions, deletions and insertions, is optimal.
“with zero weight for hits and equal weight for substitutions, deletions and insertions, is shown to be optimal” (Abstract)
Morris, Maier and Green propose match error rate (MER), the proportion of input and output word matches that are errors, as an alternative absolute measure.
“MER is the proportion of I/O word matches which are errors” (Abstract)
Morris et al. give an example in which a system that outputs two wrong words for each input word gets a WER of 200 percent, although both systems in their example communicate exactly zero information.
“A similar system which outputs two wrong words for each input word has S=N1, H=D=0, I = N 1, and WER = (N1+N1)/N1 = 200%” (Section 2, worked examples)
Morris, Maier and Green describe word error rate as the de facto standard index for speech recognition assessment, defined as the proportion of word errors to words processed.
“The present de facto standard index for ASR system assessment is the Word Error Rate, which is defined as the proportion of word errors to words processed” (Section 1, Introduction)

Pages that rest on this source

Peer-reviewed research

From WER and RIL to MER and WIL: Improved Evaluation Measures for Connected Speech Recognition

Andrew Cameron Morris et al., Proceedings of Interspeech 2004, ISCA, 2004

Read the original

Read on October 17, 2026

What it is: A 2004 paper that defines the word error rate formula, works through examples where WER gives misleading scores, and proposes match error rate (MER) and word information lost (WIL) as alternatives.
How much weight it carries: Peer-reviewed and widely cited in speech recognition evaluation. It is a 2004 study of connected speech recognition, so its examples predate modern neural recognizers, but the definitions and the alignment argument still hold.

Facts taken from this source

The 2004 review by Morris, Maier and Green cites a 1982 workshop paper by Woodard and Nelson as the source for the definition of WER for isolated word recognition.
“Woodard, J.P. & Nelson, J.T. "An information theoretic measure of speech recognition performance", Workshop on standardisation for speech I/O technology, Naval Air Development Center, Warminster, PA, 1982.” (References, item [10])
Morris et al. state that the usual alignment, with zero weight for hits and equal weight for substitutions, deletions and insertions, is optimal.
“with zero weight for hits and equal weight for substitutions, deletions and insertions, is shown to be optimal” (Abstract)
Morris, Maier and Green propose match error rate (MER), the proportion of input and output word matches that are errors, as an alternative absolute measure.
“MER is the proportion of I/O word matches which are errors” (Abstract)
Morris et al. give an example in which a system that outputs two wrong words for each input word gets a WER of 200 percent, although both systems in their example communicate exactly zero information.
“A similar system which outputs two wrong words for each input word has S=N1, H=D=0, I = N 1, and WER = (N1+N1)/N1 = 200%” (Section 2, worked examples)
Morris, Maier and Green describe word error rate as the de facto standard index for speech recognition assessment, defined as the proportion of word errors to words processed.
“The present de facto standard index for ASR system assessment is the Word Error Rate, which is defined as the proportion of word errors to words processed” (Section 1, Introduction)