Peer-reviewed research

A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency

Tara N. Sainath et al., ICASSP 2020 (arXiv:2003.12710), 2020

Read the original

Read on October 17, 2026

What it is: A Google paper that builds an on-device speech recognizer from a first-pass RNN transducer and a second-pass LAS rescorer. It reports word error rate and endpointer latency, the time between the user stopping speech and the microphone closing, with and without a separate endpointer.
How much weight it carries: Peer-reviewed conference paper from the team that built on-device recognition at Google. Results are on Google's internal test sets, so the absolute numbers apply to that system and not to other engines, but the latency definition and the trade-off between endpointing and word errors are general.

Facts taken from this source

Sainath et al. report that with an acoustic endpointer alone, their on-device model reached a median endpointer latency of 450 ms, and its word error rate rose from 6.2 to 7.4 percent.
“With the acoustic endpointer alone, we degrade the WER from 6.2% (no EP) to 7.4% to achieve a 450ms EP50 latency and 860ms EP90 latency.” (Section 6.2.1, Table 4)
Sainath et al. report that with an acoustic endpointer alone, the 90th-percentile endpointer latency was 860 ms.
“With the acoustic endpointer alone, we degrade the WER from 6.2% (no EP) to 7.4% to achieve a 450ms EP50 latency and 860ms EP90 latency.” (Section 6.2.1, Table 4)
Sainath et al. define endpointer latency as the time it takes for the microphone to close after the user stops speaking.
“We define endpointer (EP) latency as the amount of time it takes for the microphone to close after a user stops speaking.” (Section 4)
Sainath et al. report that a joint RNN-T endpointer model reached a word error rate of 6.8 percent and cut the 90th-percentile endpointer latency by 70 ms.
“obtains a WER of 6.8% and reduces EP50 and EP90 by 20ms and 70ms, respectively” (Section 6.2.1)
Sainath and colleagues report that their streaming on-device model, RNN-T plus LAS, obtained an 8 percent relative improvement in word error rate over a conventional model at the same latency, while being more than 400 times smaller.
“for the same latency, RNN-T+LAS obtains a 8% relative improvement in WER, while being more than 400-times smaller in model size” (Abstract)

Pages that rest on this source

Peer-reviewed research

A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency

Tara N. Sainath et al., ICASSP 2020 (arXiv:2003.12710), 2020

Read the original

Read on October 17, 2026

What it is: A Google paper that builds an on-device speech recognizer from a first-pass RNN transducer and a second-pass LAS rescorer. It reports word error rate and endpointer latency, the time between the user stopping speech and the microphone closing, with and without a separate endpointer.
How much weight it carries: Peer-reviewed conference paper from the team that built on-device recognition at Google. Results are on Google's internal test sets, so the absolute numbers apply to that system and not to other engines, but the latency definition and the trade-off between endpointing and word errors are general.

Facts taken from this source

Sainath et al. report that with an acoustic endpointer alone, their on-device model reached a median endpointer latency of 450 ms, and its word error rate rose from 6.2 to 7.4 percent.
“With the acoustic endpointer alone, we degrade the WER from 6.2% (no EP) to 7.4% to achieve a 450ms EP50 latency and 860ms EP90 latency.” (Section 6.2.1, Table 4)
Sainath et al. report that with an acoustic endpointer alone, the 90th-percentile endpointer latency was 860 ms.
“With the acoustic endpointer alone, we degrade the WER from 6.2% (no EP) to 7.4% to achieve a 450ms EP50 latency and 860ms EP90 latency.” (Section 6.2.1, Table 4)
Sainath et al. define endpointer latency as the time it takes for the microphone to close after the user stops speaking.
“We define endpointer (EP) latency as the amount of time it takes for the microphone to close after a user stops speaking.” (Section 4)
Sainath et al. report that a joint RNN-T endpointer model reached a word error rate of 6.8 percent and cut the 90th-percentile endpointer latency by 70 ms.
“obtains a WER of 6.8% and reduces EP50 and EP90 by 20ms and 70ms, respectively” (Section 6.2.1)
Sainath and colleagues report that their streaming on-device model, RNN-T plus LAS, obtained an 8 percent relative improvement in word error rate over a conventional model at the same latency, while being more than 400 times smaller.
“for the same latency, RNN-T+LAS obtains a 8% relative improvement in WER, while being more than 400-times smaller in model size” (Abstract)