Preprint

Voxtral

Mistral AI, arXiv (Audio and Speech Processing), 2025

Read the original

Read on October 17, 2026

What it is: A July 2025 preprint from Mistral AI that introduces its Voxtral speech models and compares them with OpenAI's Whisper large-v3, GPT-4o mini Transcribe, Gemini 2.5 Flash and Scribe. Appendix tables give word error rate for each language on FLEURS and Mozilla Common Voice.
How much weight it carries: Written by a company that sells one of the systems compared, so the ranking is not independent. The same evaluation was applied to every system, which keeps the comparison consistent within the paper. The test sets are read or crowd-sourced speech, not your own recordings.

Facts taken from this source

In the same Voxtral table, Gemini 2.5 Flash had a word error rate of 25.25 percent in Arabic, the highest of the three systems.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column ar)
In the same Voxtral table, Gemini 2.5 Flash had a word error rate of 6.76 percent in Hindi.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column hi)
In the same Voxtral table, GPT-4o mini Transcribe had a word error rate of 14.02 percent in Arabic.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column ar)
In the same Voxtral table, GPT-4o mini Transcribe had a word error rate of 12.36 percent in Hindi.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column hi)
Mistral's Voxtral models are trained to understand both spoken audio and text documents, and the paper calls them multimodal audio chat models.
“Voxtral is trained to comprehend both spoken audio and text documents” (Abstract)
In the Voxtral paper's FLEURS table, Whisper large-v3 had a word error rate of 15.44 percent in Arabic.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column ar)
On the Mozilla Common Voice Hindi test in the Voxtral paper, Whisper large-v3 had a word error rate of 46.75 percent.
“Model ar nl en fr de hi it pt es Whisper large-v3 50.58 5.83 22.91 11.33 6.25 46.75 6.81 7.17 5.66” (Table 5 (Mozilla Common Voice), column hi)
In the Voxtral paper's FLEURS table, Whisper large-v3 had a word error rate of 28.87 percent in Hindi.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column hi)

Pages that rest on this source

Preprint

Voxtral

Mistral AI, arXiv (Audio and Speech Processing), 2025

Read the original

Read on October 17, 2026

What it is: A July 2025 preprint from Mistral AI that introduces its Voxtral speech models and compares them with OpenAI's Whisper large-v3, GPT-4o mini Transcribe, Gemini 2.5 Flash and Scribe. Appendix tables give word error rate for each language on FLEURS and Mozilla Common Voice.
How much weight it carries: Written by a company that sells one of the systems compared, so the ranking is not independent. The same evaluation was applied to every system, which keeps the comparison consistent within the paper. The test sets are read or crowd-sourced speech, not your own recordings.

Facts taken from this source

In the same Voxtral table, Gemini 2.5 Flash had a word error rate of 25.25 percent in Arabic, the highest of the three systems.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column ar)
In the same Voxtral table, Gemini 2.5 Flash had a word error rate of 6.76 percent in Hindi.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column hi)
In the same Voxtral table, GPT-4o mini Transcribe had a word error rate of 14.02 percent in Arabic.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column ar)
In the same Voxtral table, GPT-4o mini Transcribe had a word error rate of 12.36 percent in Hindi.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column hi)
Mistral's Voxtral models are trained to understand both spoken audio and text documents, and the paper calls them multimodal audio chat models.
“Voxtral is trained to comprehend both spoken audio and text documents” (Abstract)
In the Voxtral paper's FLEURS table, Whisper large-v3 had a word error rate of 15.44 percent in Arabic.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column ar)
On the Mozilla Common Voice Hindi test in the Voxtral paper, Whisper large-v3 had a word error rate of 46.75 percent.
“Model ar nl en fr de hi it pt es Whisper large-v3 50.58 5.83 22.91 11.33 6.25 46.75 6.81 7.17 5.66” (Table 5 (Mozilla Common Voice), column hi)
In the Voxtral paper's FLEURS table, Whisper large-v3 had a word error rate of 28.87 percent in Hindi.
“Model ar nl en fr de hi it pt es Whisper large-v3 15.44 5.87 4.00 5.55 5.46 28.87 2.71 3.90 2.81 GPT-4o mini Transcribe 14.02 5.54 3.19 4.51 3.76 12.36 2.02 3.54 2.58 Gemini 2.5 Flash 25.25 6.20 4.64 6.17 4.74 6.76” (Table 4 (FLEURS), column hi)