What are the best datasets for speech recognition?

There is no single best speech recognition dataset. The right one matches the speech you need to recognize, the language you need it in, and a license that allows your use. For clean English, LibriSpeech is the standard place to start. For more varied English, GigaSpeech is larger but has stricter terms for its audio. For many languages, Common Voice and FLEURS cover the most ground. Choose by the speech first, then check the license before you train on the data or publish a result.

The best dataset depends on the speech you need to recognize

Start from the speech, not from a leaderboard. Read audiobooks teach a system careful, clean reading. Phone calls and meetings bring hesitations, overlapping voices and noise. The Open ASR Leaderboard paper lists the test sets it uses together with each set's license, which is a good place to see how much these sets differ.

LibriSpeech is the most quoted English set, and its license is permissive

LibriSpeech is a corpus of about 1,000 hours of read English speech, and the OpenSLR page for it lists the license as CC BY 4.0 ¹. The clean and other test sets are the ones most papers report. The catch is that the best systems already score near the floor on these sets, so a small gap between two systems tells you little about a phone call or a meeting. Our guide to speech recognition benchmarks explains why in more detail.

GigaSpeech is larger and more varied, but read its audio terms first

GigaSpeech has 10,000 hours of labeled audio ², collected from audiobooks, podcasts and YouTube, so it mixes read and spontaneous speech. Its audio has stricter terms than its size suggests. The dataset card says that its creators do not own the copyright of the audio, and that access is offered for non-commercial research and educational purposes, under conditions set out on the card ³. The same card carries an apache-2.0 license tag. A label and an audio restriction can both apply, so treat the card's terms as the ones that govern the audio. For a commercial product, get written permission or choose a set with a clearer license.

Common Voice and FLEURS cover the most languages

Common Voice is recorded by volunteers, which gives you many accents, microphones and rooms. Its 2020 paper calls it the largest audio corpus in the public domain for speech recognition, by hours and by languages ⁴. Those totals date from the paper's 2019 state, and the project has grown since, so check the current release before you plan a project around a specific size.

FLEURS is built for comparing languages. It is a parallel speech dataset in 102 languages, so the same content is read in each one ⁵. That makes results comparable across languages, which a set of unrelated recordings cannot do. Scores still vary widely between languages, so test the language you care about rather than trusting an average. Our guide to Whisper's supported languages and accuracy shows how much they differ.

VoxPopuli is a useful choice for formal speech and accents. Its dataset card says it contains transcribed speech for 18 languages, and 29 hours of transcribed non-native English for research on accented speech ⁶. The speech comes from parliamentary events, so it is more formal than everyday dictation.

Telephone speech is licensed through a catalog, not downloaded freely

Switchboard is a corpus of spontaneous telephone conversations, and it has been used for telephone speech research for decades. The Linguistic Data Consortium (LDC) distributes it under its User Agreement for Non-Members, and its licensing instructions differ for members and non-members ⁷. Check whether your institution already holds a membership before you buy a copy. Free-looking copies online may not carry the same terms.

A famous test set may already be in the training data

A test set stops being a fair test once a model has seen it. A 2025 study found that a substantial amount of the LibriSpeech and Common Voice evaluation sets appear in public language model pretraining corpora ⁸. A low error rate on a set that may have leaked is weaker evidence than it looks. Our page on why speech-to-text accuracy claims differ covers how to check a claim.

Kaggle and other mirrors: check the original dataset card

Kaggle hosts many speech datasets, and some are uploads by people who did not create the corpus. A mirror can change the version, drop files or state a different license from the original. Before you use a copy, find the original dataset card on its home site or on Hugging Face, confirm that the version and size match, and record the license that applies to the original. Note the version you used, so that someone else can repeat your test.

Research papers describe the method, and the card describes the terms

Read the original paper for how a dataset was built and what it is meant to measure. Read the dataset card for who may use it and how. The papers that most often matter here are the GigaSpeech paper, the Common Voice paper and the Open ASR Leaderboard paper, which lists the test sets and their licenses in one table.

What to do

1. Write down the speech you need: the language, the accent, the setting, and whether it is read or spontaneous.
2. Pick the set that matches it, using the table above as a starting point.
3. Check the license on the original card, including whether you need commercial use, redistribution or a model trained on the data.
4. Check whether the set is likely to be in the training data of the models you are comparing.
5. Test on a small set of your own recordings. The guide to speech recognition benchmarks shows how to build one that predicts your own use.

For the leaderboard that compares systems on these sets, read what the Open ASR Leaderboard is. To see how the scores compare, read the most accurate speech-to-text model. The whole subject is on the speech recognition overview.