Preprint
English Conversational Telephone Speech Recognition by Humans and Machines
George Saon et al., arXiv (cs.CL), IBM Research, 2017
Read the originalRead on October 17, 2026
- What it is
- An IBM study that measured human transcribers on the Switchboard and CallHome telephone tests and reported the word error rate of IBM's own recognizer on the same tests.
- How much weight it carries
- A widely cited preprint from a major research group. The human scores come from three transcribers on a small set of calls, and the tests are old telephone speech, so the numbers show a benchmark, not the accuracy of a modern dictation app.
Facts taken from this source
- In the same IBM study, the best human transcriber reached a word error rate of 6.8 percent on the CallHome telephone test.
“achieved a surprisingly low 6.8% WER for CallHome” (Section 2, human transcription experiments)
- In the IBM study, the best of three human transcribers reached a word error rate of 5.1 percent on the Switchboard telephone test after quality checks.
“could only get down to 5.1% for the best transcriber after quality checks” (Section 2, human transcription experiments)
- IBM's own English conversational telephone recognizer reached a word error rate of 10.3 percent on the CallHome subset of the Hub5 2000 evaluation.
“to the level of 5.5%/10.3% on the Switchboard/CallHome subsets of the Hub5 2000 evaluation” (Abstract)
- IBM's own English conversational telephone recognizer reached a word error rate of 5.5 percent on the Switchboard subset of the Hub5 2000 evaluation.
“to the level of 5.5%/10.3% on the Switchboard/CallHome subsets of the Hub5 2000 evaluation” (Abstract)