What is voice biometrics and how does a voice biometric system work?

Voice biometrics identifies or verifies a person by the characteristics of their voice. A voice biometric system works in two steps. It enrolls a voice to build a stored model, the voiceprint, and then it compares new speech with that model to decide who is speaking. The same technique is called speaker recognition by researchers. It is different from speech recognition, which works out the words. For the wider picture, see our overview of voice privacy and the guide to voice biometric data.

Voice biometrics identifies a person by how their voice sounds, not by what they say

The IEEE Technology Navigator defines speaker recognition as identifying or verifying a person's identity from the characteristics of their voice ¹. The characteristics are the traits of the sound: pitch, timbre and the way a person shapes their words. Two people can say the same sentence, and a voice system is built to tell them apart from those traits.

That is why the question is different from voice recognition versus speaker recognition. A dictation tool that writes down your words does speech recognition. A system that checks who is talking does speaker recognition, and it can use the same audio.

A voice biometric system works in two steps: enrollment, then comparison

Enrollment builds the voiceprint

Before a system can recognize anyone, a person has to enroll. The system records their speech and turns it into a stored model of their voice. That model is the voiceprint. Enrollment is also where the system learns which mode it will use, because the mode decides what the person must say (see the section on modes below).

Whether the system also keeps the original recording is a separate question. Ask any vendor for that answer in writing, because the model and the audio can be stored under different rules.

Comparison decides whether the new voice matches

When someone speaks later, the system measures that speech the same way and compares it with the enrolled voiceprint. Nuance's documentation for its Gatekeeper product puts the step this way: "The biometrics system compares spoken utterances with previously enrolled voiceprints, and determines whether they are spoken by the same speaker" ².

The result is a match score, and the system turns the score into a decision by comparing it with a threshold. A higher threshold rejects more impostors but also rejects more genuine speakers, for example after a cold. The threshold is a product choice, not a fixed rule, so two systems can give different results on the same voice.

Verification and identification answer two different questions

Voice biometrics does two jobs, and the difference matters for privacy.

Verification needs one enrolled voice for each account. Identification needs a database of enrolled voices, and that is why it draws more scrutiny under privacy law.

Text-dependent and text-independent modes change what you say

Systems fall into two modes. The mode is set when the voice is enrolled, and it decides what the speaker must say.

• Text-dependent systems ask for a passphrase. The Nuance documentation says that "Text-dependent verification uses a passphrase to identify the speaker" ⁵. The same words are needed at enrollment and at check time, which makes the input predictable.
• Text-independent systems work from ordinary speech. The same documentation says that "Text-independent verification uses conversational audio of the speaker" ⁶. The system needs enough speech to form a confident match, so a short phrase tells it less.

A phone system that asks you to say a fixed phrase is text-dependent. A meeting tool that labels speakers across a whole call works from ordinary speech, so it is closer to the text-independent model.

A voice biometric system is biometric data, and that changes the law

The UK data protection regulator lists voice recognition among the physical or physiological biometric identification techniques, alongside facial recognition and iris scanning ⁷. A voiceprint built to recognize one person is the kind of data those rules cover. A transcript of what someone said is usually a different matter.

This is the reason to ask whether a product builds a voiceprint at all. The legal consequences are covered in our guide to voice biometric data and in the question on whether speech to text collects biometric data.

A voice match is a probability, so a voice alone is weak proof of identity

A match is a score, not a certainty, and the decision is only as good as the speech the system hears. A short utterance gives the system less to compare, so a short phrase is a weaker basis for a decision than a longer sample. Noise, a poor microphone and a cold can all shift the score.

Voice can also be imitated. Speaker recognition has to contend with spoofing attacks, which can include synthesizing speech with text-to-speech tools ⁸. A voice should therefore not be the only check that approves a payment or an account change. See the question on whether voice authentication is still secure and our guide to voice cloning and deepfakes.

What to do if a product uses voice biometrics

1. Ask whether it builds a voiceprint. Ask for the word itself. A product that says it only detects "speakers" may still match one voice across recordings.
2. Find the mode. Text-dependent enrollment asks you to speak a fixed phrase. Text-independent enrollment uses your normal speech. Both can build a voiceprint.
3. Ask how the voiceprint is stored and deleted. Look for a way to delete enrollment and for the retention period, in the privacy policy or the settings.
4. Give consent before enrollment. Decline enrollment if the product will not explain what it does with the voiceprint.
5. Do not rely on voice alone for approvals. Use a second check that a caller cannot control.

For terms used here, see the entries on voiceprint, speaker verification and biometric data.