Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual and Long-Form Speech Recognition Evaluation

“Of the 86 models currently listed in the ASR Leaderboard (as of 27 March 2026), 74 are open-source” (Section 2.4, Current models)

“The full and latest table can be found on Hugging Face.” (Table 3 caption)

“NVIDIA Canary 1B v2 Yes 7.15 749 FastConformer” (Table 3, short-form English)

“NVIDIA Canary Qwen 2.5B Yes 5.63 418 FastConformer” (Table 3)

“Cohere Labs Transcribe Yes 5.42 525 FastConformer” (Table 3, short-form English)

“Distil-Whisper Large v3.5 Yes 7.21 202 Whisper” (Table 3, short-form English)

“We report results on two metrics: word error rate (WER) for comparing transcription quality, and inverse real-time factor (RTFx) for comparing inference speed.” (Section 2.3 Metrics)

“For normalized transcriptions, punctuation and casing is removed, as well as disfluencies such as fillers” (Evaluation methodology, normalized transcriptions)

“NVIDIA Parakeet TDT 0.6B v2 Yes 6.05 3390 FastConformer” (Table 3)

“NVIDIA Parakeet TDT 0.6B v3 Yes 6.32 3330 FastConformer” (Table 3, short-form English)

“private evaluation sets could help minimize the risk of test-set contamination” (Conclusion)

“OpenAI Whisper Large v3 Yes 7.44 146 Whisper” (Table 3)

“OpenAI Whisper Large v3 Turbo Yes 7.83 200” (Table 3, short-form English)

“Zoom Scribe v1 No 5.47 - - - 1 IBM Granite Speech 4.0 1B” (Table 3, short-form English)

“We report the inverse real-time factor, rather than real-time factor” (Section 2.3, Metrics)