Other
Announcing AA-WER Streaming (Artificial Analysis)
Artificial Analysis, Artificial Analysis, 2026
Read the originalRead on October 17, 2026
- What it is
- An independent benchmark article, dated June 1, 2026, that measures streaming speech to text models on word error rate and on the latency from end of speech to the first partial and first final transcript.
- How much weight it carries
- Independent benchmark by a third party with a published method. It tests one set of about eight hours of audio, and it measures latency from a voice activity detector's end-of-speech point, so it is not a measure of time from the start of speech. Results can change as vendors update their models.
Facts taken from this source
- In Artificial Analysis's June 2026 streaming benchmark, Deepgram Flux was the fastest service on both the first partial and the first final transcript after end of speech, at 0.020 seconds for the final transcript, with a 7.36 percent word error rate.
“@DeepgramAI Flux leads both Final and Partial at 0.020s and 0.019s respectively (both 7.36% WER)” (Key results, 'Fastest transcription')
- In the same Artificial Analysis benchmark, AssemblyAI U3 Realtime Pro had a first partial transcript 0.47 seconds after end of speech, at 4.46 percent word error rate.
“AssemblyAI U3 Realtime Pro (4.46%, 0.47s)” (Key findings)
- In Artificial Analysis's June 2026 streaming benchmark, Deepgram Flux had the fastest first partial transcript after end of speech, at 0.019 seconds.
“Flux leads both Final and Partial at 0.020s and 0.019s respectively” (Key findings)
- In the same Artificial Analysis benchmark, ElevenLabs Scribe v2 Realtime had the highest accuracy on the first partial after end of speech, at 3.65 percent word error rate and 0.13 seconds of latency.
“ElevenLabs Scribe v2 Realtime at 3.65% WER, 0.13s latency” (Key findings)
- In Artificial Analysis's 2026 streaming benchmark, the first partial transcript from Deepgram Flux had a word error rate of 7.4 percent, while ElevenLabs Scribe v2 Realtime's had 3.6 percent.
“ElevenLabs Scribe v2 Realtime 3.6% WER at both, Deepgram Nova-3 6.7% at both, Deepgram Flux 7.4% at both” (Key results, paragraph on accuracy and first partial)
- Artificial Analysis says streaming speech-to-text performance varies meaningfully across its three test datasets, which differ in audio length, accents, vocabulary and background noise.
“Performance varies meaningfully across the three datasets with different audio lengths, accents, vocabulary, and background noise.” (Key results, paragraph after the first-partial frontier)
- In Artificial Analysis's 2026 streaming benchmark, OpenAI's hosted GPT Realtime Whisper model improved from a 7.5 percent word error rate on the first partial transcript to 5.1 percent on the final transcript.
“led by Voxtral Mini Transcribe Realtime (9.6%→5.3%) and OpenAI GPT Realtime Whisper (7.5%→5.1%)” (Key results, paragraph on which models improve by waiting for the final)
- Artificial Analysis tested its streaming speech-to-text models on about 8 hours of audio, the same audio as its offline benchmark.
“we test these streaming models on the same ~8 hours of audio as our offline benchmark” (Opening paragraph of the article)
Pages that rest on this source
- Real-Time Speech Recognition: Streaming and LatencyGuide
- AI Voice Agents: How They Work and Where They FailGuide
- What is the fastest speech-to-text API?Question
- Which speech to text should you pair with an LLM?Question
- AI Dictation: How AI Voice Typing WorksGuide
- Why is AI dictation slower than built-in dictation?Question
- Which speech-to-text API is the best?Question
- Speech to Text and LLMs: What Language Models ChangeGuide
- Can Whisper transcribe in real time?Question
- Can you transcribe a meeting live, in real time?Question
- How do you connect speech to text to an LLM?Question