Hidden Markov model (HMM)

A hidden Markov model (HMM) is a statistical model with two layers of data. The hidden states follow one another with set transition probabilities, and each state produces an observation you can measure. You see the observations, not the states, so the states are "hidden". Rabiner's 1989 tutorial lays out the three problems an HMM must solve before it is useful for speech: evaluating how likely an observation sequence is, finding the best sequence of states, and adjusting the model's parameters to fit the signal ¹.

In speech, the hidden states are units of sound

A word is modeled as a sequence of states, each standing for a small unit of sound. Each short frame of audio is an observation. A scoring model estimates how well a given state fits a frame. Older systems used a Gaussian mixture model for that score, and Hinton and colleagues describe the pipeline as HMMs handling the timing of speech while Gaussian mixture models score how well each state fits a short window of acoustic input ².

The transition probabilities handle timing. Take the "k" sound at the start of "cat". The model stays in that state for as many frames as the sound lasts, then moves on to the next. A slow speaker keeps the model in each state longer, and a fast speaker moves through the states more quickly, which is why one word can take a different number of frames each time it is spoken.

Decoding finds the most likely path through the states

To turn audio into words, a decoder searches for the most probable path through the states. The search is usually done with the Viterbi algorithm, which keeps only the best path into each state at each frame. A language model is combined with the search, so that a word which sounds right but makes a poor sentence is less likely to win.

Neural networks improved the scores, but the timing model stayed

In a 2012 review, a deep neural network reduced the word error rate of a GMM-HMM baseline from 27.4 to 18.5 percent on a Switchboard telephone task ³. The result was a turning point. Many systems kept the HMM for timing and replaced the Gaussian mixtures with neural networks.

Later systems, including CTC and RNN transducer models, learn the mapping from audio to text in one network and no longer need a separate HMM. That shift is why older speech documentation talks about acoustic models and decoders, and the newer material often does not.

Why it matters for dictation

Early dictation software was built on HMMs, so older manuals and papers use their vocabulary. Knowing what a hidden state and a transition probability are helps you read those documents and understand why error rates fell so sharply after neural networks arrived. It also explains why a timing model is part of a traditional system, while a modern end-to-end model handles timing inside the network.

If a product describes its acoustic model and language model as separate parts, it is probably following the older design. If it describes one neural network, it is probably not.

Related terms

• Acoustic model: the part that scores sound against speech units.
• Language model (in speech recognition): the part that scores word sequences.
• Connectionist temporal classification (CTC): a training method that replaced the HMM's timing role in many systems.
• Automatic speech recognition (ASR): the field the HMM belongs to.

Guides that cover it

• History of speech recognition: from Audrey to Whisper
• How speech recognition works, from sound to text