Fact · Voice and AI
40milliseconds
Qwen2.5-Omni 7B, an end-to-end speech language model, had a 90th percentile time to first token of about 40 milliseconds on one H100 GPU.
Checked against its source on October 17, 2026
In the source's own words
“Qwen2.5-Omni 7B achieves a 90th percentile (p90) TTFT of approximately 40 ms on a single H100 GPU”
VoiceAgentBench: Are Voice Assistants ready for agentic tasks?, Dhruv Jain et al., arXiv, 2025, Appendix F.
Open the original- What it measures
- The benchmark used 100 queries with an average duration of 3.5 seconds of speech. Time to first token (TTFT) is the delay before the first output token appears.
- What it does not show
- This measures the start of the reply, not the whole answer, on one GPU in one benchmark. A commercial product with a different server setup will show different numbers.