Open-Source Speech Recognition: The Best Models

Open models can now match proprietary speech-to-text services on English benchmarks, and many of them run at no cost on your own computer. The catch is that licenses, language coverage, streaming support and hardware needs differ a great deal between models. A high score on a public test does not tell you which model will work on your recordings, so start with what you need.

This guide compares the main open model families on dated leaderboard results and license terms. It then covers what you still have to build around a model, which most comparison lists skip. For the basics, see our overview of speech recognition.

Open models now rival proprietary APIs on public English benchmarks

On the Open ASR Leaderboard, which averages word error rate (WER) across several English test sets, 74 of the 86 systems listed on 27 March 2026 were open-source ¹. Lower WER is better: it is the share of words a system gets wrong.

Cohere Labs Transcribe, an open model, had an average WER of 5.42 percent ². Zoom Scribe v1, which the paper's table marks as not open, had 5.47 percent ³. The two scores are so close that one benchmark cannot name a winner.

Other open models sit close behind. NVIDIA's Canary-Qwen 2.5B scored 5.63 percent ⁴. NVIDIA's Parakeet TDT 0.6B v2 scored 6.05 percent and processed audio at 3,390 RTFx, the fastest rate in the chart below ⁵. Speed in that table is shown as inverse real-time factor (RTFx), which is audio duration divided by processing time. RTFx is higher when a system is faster, so 3,390 means roughly 3,390 seconds of audio processed per second of computing.

These numbers are averages over the leaderboard's English tests, which are scored after punctuation, casing and filler words are removed ⁶. Treat them as a starting point, and read the guide to speech recognition benchmarks before you rank models by them. The table is also a snapshot from 27 March 2026 ¹. The paper points to Hugging Face for its full and latest results table ⁷, so check that table for current numbers.

The main families: Whisper, NVIDIA Parakeet and Canary, and a new wave of open models from other labs

OpenAI's Whisper is still the model most people mean when they say open-source speech recognition. NVIDIA's Parakeet TDT 0.6B v2 and Canary-Qwen 2.5B score better than Whisper Large v3 on the English leaderboard. Their average word error rates are 6.05 and 5.63 percent, against 7.44 percent for Whisper Large v3 ^{5 4 8}. Newer open models from Meta and NVIDIA add language coverage or live streaming. The table lists the main options with their language scope and license.

Read the table in a fixed order. Check the license first, because it decides whether you can sell a product built on the model. Check the languages next, since a model that does not cover your language is of no use however low its error rate. Then check streaming and hardware. Accuracy comes last, and only after you have narrowed the list to models you can legally use and that cover your speech.

Whisper remains a sensible default when you need many languages and mature tools. If you are deciding between it and the newer open models, read whether there are better alternatives to Whisper. The NVIDIA Parakeet question, including whether it beats Whisper, is answered in what NVIDIA Parakeet is.

The Whisper family and its tools are covered in our guide to Whisper AI, and the guide to running Whisper locally shows one way to start.

Licenses differ: MIT and Apache 2.0 allow commercial use, while some models restrict it

The license decides whether you can ship a model in a paid product. MIT and Apache 2.0 are the most permissive common licenses. Whisper's code and model weights are MIT licensed ⁹. The Kaldi toolkit is Apache 2.0 ²⁰, and Meta's Omnilingual ASR model and code are Apache 2.0 too ¹⁸.

CC-BY-4.0 models can also be used commercially, but the license requires attribution. NVIDIA's Canary-Qwen 2.5B card says the model "is ready for commercial use" under CC-BY-4.0 ¹⁵. Parakeet TDT 0.6B v3 is also released under CC BY 4.0 ¹¹.

Some models carry a non-commercial license. The NVIDIA Canary 1B card lists CC-BY-NC-4.0 ¹⁶. The NC term means NonCommercial, so you cannot use that model in a paid product unless you obtain another license.

Three checks before you build on any model:

1. Read the license on the model's own card, not in a comparison article. A family can contain models with different licenses.
2. Check the training data terms separately. Meta's model is Apache 2.0, but its training corpus is licensed under CC-BY-4.0 ¹⁸.
3. Keep the attribution and license text with your product when the license requires it.

This is general information about licenses, not legal advice. For a commercial product, have a lawyer confirm the terms.

Older toolkits (Kaldi and Vosk) still fit tiny devices and fixed command sets

Older toolkits still matter. They suit very small devices, fixed command sets, and projects where you want to build the recognizer yourself. They are rarely the first choice for general dictation on a modern laptop.

Kaldi is the classic research toolkit, licensed under Apache 2.0 ²⁰. It is a set of tools for building a recognizer, which is why it suits developers and researchers more than people who only want to dictate.

Vosk is the one most people meet first. Its homepage says it "supports 20+ languages and dialects" ²¹, and it works offline on lightweight devices such as the Raspberry Pi ²². If you want a closer look, see what Vosk is.

The older toolkits trade accuracy for size. For open-ended dictation of long, accented or noisy speech, a neural model such as Whisper or Parakeet is the usual choice. Test Vosk on your own audio before you commit to it.

If your device is a small board that only needs to recognize a short list of commands, a toolkit like Vosk, with its vocabulary limited to those commands, is usually the most practical place to start. Accuracy on that narrow set is what you will measure, not accuracy on general speech.

Hugging Face is where to find open speech models and the leaderboard's full results

Hugging Face hosts a large collection of open speech models. Its speech recognition task page says that "the Hub contains over 17,000 ASR models" ²³. That count covers the whole Hub, so check the original model card rather than assuming each entry is a separate, independently tested system.

The model pages let you test a model in the browser before you install anything ²³. That makes Hugging Face a good place to compare samples of your own recordings.

Hugging Face hosts models for you to download and run, rather than one speech-to-text service. The free route is to download a model and run it on your own machine, which is what the next section covers.

Small models run on a laptop CPU, while the largest need a GPU

Model size sets the memory a model needs, and memory sets the hardware you need. Whisper's large model needs about 10 GB of video memory on a graphics card ²⁴, a demanding requirement for most laptops. Smaller models need much less memory.

NVIDIA says the Parakeet TDT 0.6B v3 model is "designed and/or optimized to run on NVIDIA GPU-accelerated systems" ²⁵. A CPU can still run smaller models for dictation, but expect slower results there. Test on the computer you will actually use.

Speed varies a lot between models. The chart compares the RTFx values from the same leaderboard table, where higher is faster.

Parakeet TDT 0.6B v2 ran at 3,390 RTFx ⁵, and Parakeet TDT 0.6B v3 at 3,330 ²⁶. The Whisper rows are slower: Whisper Large v3 Turbo ran at 200 RTFx ²⁷, and Whisper Large v3 at 146 ⁸. These are measured over the leaderboard's test sets, so your speed on a single live utterance, on your own computer, will differ.

Size, languages, streaming and speed decide the right model, not word error rate alone

The leaderboard ranks models by average WER, but your choice depends on four things that average hides: the audio you record, the languages you speak, whether you need streaming, and the hardware you have.

Audio conditions can reverse a ranking. In a 2025 test by 9to5Mac of one audio episode, NVIDIA's Parakeet v2 had a WER of 12.3 percent, the highest of the three engines tested ²⁸. One episode is not a benchmark, but it shows how a leaderboard can mislead about your own audio.

Whisper can also invent text. A 2024 study found that roughly 1 percent of the Whisper transcriptions it tested contained entire hallucinated phrases or sentences that did not exist in the audio ²⁹. Check any model you choose for this on your own recordings.

Streaming is a separate choice. Live use of Whisper depends on a sliding-window approach. The whisper.cpp streaming example uses one: setting the step to 0 "enables the sliding window mode" and transcribes once voice activity is detected ³⁰. Models built for streaming avoid that workaround. NVIDIA's Nemotron Speech Streaming model uses a cache-aware encoder with an RNN-T decoder ¹⁹, which lets it stream audio as it arrives. For the wider question, see the guide to real-time speech recognition.

Match the model to the job:

• Many languages: choose Omnilingual ASR or Whisper, then test your own language, since language coverage is not the same as accuracy.
• English dictation: the Parakeet and Canary models score well among open models on English benchmarks, and Parakeet v3 also covers 25 European languages ¹⁰.
• Live captions or voice assistants: choose a model designed for streaming, such as Nemotron.
• Tiny devices or fixed commands: Vosk fits where neural models are too large.

How to test a model on your own audio

A leaderboard cannot tell you how a model handles your microphone, your accent or your vocabulary. A short test can, and it takes an afternoon.

1. Collect 20 to 30 short recordings that match your real use: the same microphone, room and speaking style, with the words you actually use.
2. Write the correct transcript for each recording by hand.
3. Run each candidate model on the same recordings, and record how long each one takes.
4. Count the words the model got wrong, meaning substitutions, deletions and insertions. Divide that count by the number of words in your correct transcript. The result is your WER, and lower is better.
5. Compare models on that number and on speed, then check the license for the one you pick.

Keep the test set fixed while you compare models. Changing the recordings between runs makes the comparison unreliable.

A model is not a product: you still need voice activity detection, chunking, formatting and an interface

A model turns audio into words. A usable dictation or transcription tool needs several other parts, and much of the work sits in those parts.

The first is voice activity detection (VAD), which decides when someone is speaking so the model is not run on silence. The whisper.cpp streaming example uses this in its sliding-window mode, which transcribes after voice activity is detected ³⁰.

The second is chunking. A model can only process a file of limited length in one pass. Parakeet v3 handles up to 24 minutes with full attention on an A100 80GB GPU, and up to 3 hours with local attention ¹². Longer recordings need splitting, with the cuts placed in pauses so words are not cut in half. The on-device speech recognition guide covers what changes when the model runs on your own hardware.

The third is formatting. Raw model output may lack the punctuation, capitals or digit form you want, so the inverse text normalization step can matter. Plan for a cleanup pass, and test it on the same recordings you use for accuracy.

The fourth is the interface: getting text into the app where you are typing. Open-source dictation apps are built around exactly this job, and the open-source dictation software guide compares them. If you want to write your own app, the guide to speech recognition in Python covers the code you would build on.

If you want these parts done for you, Yap, which we make, is an on-device dictation app for Mac and Windows. You can download the app. If you would rather build on an open model, start with the license and the language coverage, test on your own recordings, and only then choose the model.