What is the Open ASR Leaderboard?

The Open ASR Leaderboard is a public comparison of speech recognition systems, the software that turns speech into text. It is run by Hugging Face and its collaborators and hosted as a Hugging Face Space. Every system on it transcribes the same audio, and the transcripts are scored with the same code. Its 2025 paper covers 86 open-source and proprietary systems across 12 datasets ¹. Because every entry is scored the same way, it is the easiest place to compare a cloud service with an open model.

It does not measure how a system will perform on your voice, microphone and room. It measures how each system performs on the leaderboard's own test sets. Read the scores as a fair comparison on those sets, not as a verdict on every use.

It is a public scoreboard that runs every system on the same audio

Most accuracy claims come from the company that built the system, and each company picks its own test audio and its own scoring rules. The leaderboard removes that choice. It takes a fixed set of audio files, each with a human reference transcript, and feeds the same files to every system. The paper says the full and latest results table is on Hugging Face ², so you can check the rows yourself.

The leaderboard covers several tracks. The English short-form track is the one most people quote. Long-form audio and multilingual speech are scored separately, so a model that leads one track may not lead another. Always check which track a number comes from.

It reports two numbers: word error rate and speed

The leaderboard reports two measures for each system. The paper describes them as "word error rate (WER) for comparing transcription quality, and inverse real-time factor (RTFx) for comparing inference speed" ³.

Word error rate counts the words a system gets wrong, as substitutions, deletions and insertions, divided by the number of words in the reference. A lower number is better. The word error rate entry explains the arithmetic. Inverse real-time factor is the reverse of a time ratio: a higher number means more seconds of audio processed per second of compute. Speed is measured on one fixed test setup, so it describes throughput, not the wait you feel after speaking. Our guide to real-time speech recognition covers that.

The scoring removes punctuation and fillers before counting errors

Before it counts errors, the leaderboard normalizes both texts. For its normalized transcripts, punctuation and casing are removed, along with disfluencies such as fillers, repetitions and repairs ⁴. A system is not penalized for writing "um" or leaving out a comma, and it is not rewarded for adding one.

That is fair for comparing recognizers, but it does not match dictation. If you want a system that keeps your punctuation and drops the filler words you would rather not see, the leaderboard score says nothing about that behavior. Test it yourself.

The leaderboard's order changes between snapshots

In the 2025 paper's latest version, the English short-form table puts Cohere Labs Transcribe first among the open systems, with an average word error rate of 5.42 percent ⁵. The proprietary Zoom Scribe v1 came next, at 5.47 percent ⁶. The two averages differ by 0.05 points, a gap small enough that a different test set could change the order.

A later mirror of the results, dated May 6, 2026, shows a different leader. It reports that "IBM Granite Speech 4.1 2B leads with 5.33% WER, edging out Cohere Transcribe 03-2026 by 0.09 percentage points" ⁷. That mirror is a third-party copy, not the official page, so check the Hugging Face table before you quote a rank.

The lesson is the same in both snapshots. The top systems sit within a few tenths of a point of one another, and the rank can change as new systems are added. A small gap is not proof that one system is better for you.

Private test sets are the leaderboard's answer to contamination

A public test set can leak into the data used to train a model, which makes its score look better than the model's real skill. The paper's authors note that "private evaluation sets could help minimize the risk of test-set contamination" ⁸. A data vendor, Appen, says it provides a suite of new private English-language audio datasets for the leaderboard ⁹. Appen sells that kind of data, so treat its announcement as its own claim.

Contamination is covered in more detail in our guide to speech recognition benchmarks.

How to read one row of the leaderboard

1. Find the track. Short-form English, long-form audio and multilingual results are different tests.
2. Check the date. The same model name can hide a retrained version, and new rows can change the order.
3. Check open or proprietary. An open model can run on your own computer. A proprietary service usually runs on its maker's servers.
4. Check the average. The leaderboard's average covers several test sets. A single average can hide a weak test set, such as accented or noisy speech.

What to do with the leaderboard

Use it to shortlist systems. A system that is far behind the top group on the track that matches your language is worth dropping. Then test the shortlist on recordings of your own speech, in the room where you will use it, with the same scoring rules for every system.

For the most common version of this question, read the most accurate speech-to-text model. For the reasons published scores disagree, read why speech-to-text accuracy claims disagree. For open models you can run yourself, see our guide to open-source speech recognition. The whole subject is on the speech recognition overview.