What are the most important speech recognition research papers?

No list of papers is final, and the field keeps adding new ones. The selection below is ours. We chose papers that changed the method, or that changed how results are measured, and whose claims you can check in the original text. Each entry says what the paper showed and where its result has limits.

Rabiner's 1989 tutorial sets out the statistical method

Rabiner's 1989 tutorial on hidden Markov models is the best starting point for the theory. It says that before a hidden Markov model can be applied to speech, three fundamental problems must be solved: evaluating the likelihood of an observation sequence, finding the best sequence of model states, and adjusting the model parameters ¹. It is a tutorial, so it teaches the math step by step rather than reporting a new benchmark.

Shannon and colleagues tested speech recognition with primarily temporal cues

"Speech recognition with primarily temporal cues," a 1995 paper by Shannon and colleagues, reports that increasing the number of bands beyond three or four gave no further improvement in their results ².

The limit is important. The test used simulated speech and simple sentences, so the paper shows what temporal cues can carry, not how well a recognizer works on everyday recordings.

Lippmann's 1997 review compares machines with humans

Lippmann's 1997 review compares modern speech recognition systems with humans. Its stated aim is to see how far recent advances have moved the technology toward human-like performance ³. It is a useful single source for the older human comparison, and it describes the systems of its own time, so read its results as a record of that period. Our question on whether speech recognition is more accurate than humans works through the comparison.

Hinton and colleagues' 2012 review brought deep networks into the acoustic model

A 2012 review by a group of authors reported that a deep neural network reduced the word error rate of a baseline system from 27.4 to 18.5 percent on a Switchboard telephone task ⁴. Read the number with care. It summarizes several systems on one benchmark, so it shows a direction, not a precise size of improvement.

Graves and colleagues' 2006 paper trained networks on unsegmented sequences

The 2006 paper introduced the method now called connectionist temporal classification, the training method named in our glossary. Earlier recurrent networks needed pre-segmented training data and post-processing, which limited where they could be used. The paper presented a method that labels unsegmented sequences directly ⁵. Read it for the training method, and see the glossary entry on connectionist temporal classification for a short definition.

The 2022 Whisper paper describes an open model trained on 680,000 hours

The Whisper paper, Robust Speech Recognition via Large-Scale Weak Supervision, is on arXiv under the number 2212.04356. Its authors trained the model on 680,000 hours of multilingual and multitask audio with transcripts ⁶. It is the paper to read if you want to understand the open model that many people run today. The guide to Whisper AI explains what the model does well and where it fails.

Journals to follow for new work

Two journals cover the field. The IEEE Transactions on Audio, Speech and Language Processing is dedicated to theory and methods for processing signals that represent audio, speech and language ⁷. Speech Communication describes itself as an interdisciplinary journal focused on the rapid dissemination and thorough discussion of research results ⁸.

Journal papers go through peer review. arXiv preprints appear first but are not peer reviewed. Use the journal for vetted work and arXiv for the newest results, and check which kind you are reading.

How to read a paper on speech recognition quickly

1. Read the abstract and the results table first, to see which test set and metric the paper uses.
2. Check whether the test set is one you know, such as LibriSpeech or Switchboard. Scores on different test sets cannot be ranked against each other.
3. Compare word error rate only on the same test set and with the same text normalization. See the glossary entry for word error rate for how it is computed.
4. Look for a limitations section, and check that the test audio matches your use. A score on one kind of audio does not predict the score on another.

It depends on your goal

If you want the theory, read Rabiner's tutorial, then the 2012 review. If you want to understand the models you can run today, read the 2022 Whisper paper and then our question on which Whisper model is best. If you want a broad textbook, see our question on how to learn speech recognition. For the story behind these papers, read our history guide, and for every guide in this topic, see the speech recognition overview.