Fact · Voice and AI
800milliseconds
A pipeline that runs Whisper large v3 first and then the Qwen3 8B language model had a 90th percentile time to first token of about 800 milliseconds on the same hardware.
Checked against its source on October 17, 2026
In the source's own words
“a pipeline combining Whisper-large-v3 with Qwen3 8B exhibits a p90 TTFT of around 800 ms under the same hardware conditions”
VoiceAgentBench: Are Voice Assistants ready for agentic tasks?, Dhruv Jain et al., arXiv, 2025, Appendix F.
Open the original- What it measures
- The paper contrasts this pipeline, which transcribes speech into text before a language model answers, with end-to-end speech models that skip the transcript. Both were measured on the same single H100 GPU, with 100 queries.
- What it does not show
- This is one pipeline on one GPU in one benchmark. The paper's point is the design gap, and a product with faster servers, or a streaming design, will show a different delay.