Peer-reviewed research

Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models on GPU

Daniel Galvez et al., Proceedings of Interspeech 2024 (arXiv:2406.03791), 2024 · DOI 10.48550/arXiv.2406.03791

Read on October 19, 2026

What it is: An NVIDIA research paper on GPU decoding for RNN-Transducer speech models. Its Table 1 reports RTFx, the hours of audio transcribed per hour of wall-clock time, for Parakeet models under several decoders, measured on an A100-80GiB PCIe GPU with batch size 32 on LibriSpeech test-other.
How much weight it carries: Peer-reviewed Interspeech paper with an open full text, from the team that implemented the decoders in NVIDIA NeMo. The RTFx figures are offline, batched and in bfloat16, so they are not comparable with batch-size-1 streaming or single-file numbers. The paper measures a decoding optimization; the models themselves are open NVIDIA checkpoints.

Facts taken from this source

In the Galvez et al. 2024 paper, the baseline decoder for Parakeet RNN-T 1.1B ran at an RTFx of 414.63 on an A100-80GiB PCIe GPU at batch size 32, meaning 414.63 times faster than real time.
“Parakeet RNN-T 1.1B Baseline 414.63 67.39 2.7 1 1 ... For all results, we use an A100-80GiB PCIe card, and a batch size of 32” (Section 4 (Results) and Table 1, Parakeet RNN-T 1.1B Baseline row)
In the Galvez et al. 2024 paper, Parakeet RNN-T 1.1B with label looping and CUDA graphs ran at an RTFx of 1120.88 on an A100-80GiB PCIe GPU at batch size 32, meaning about 1,121 times faster than real time.
“Label Looping + CUDA Graphs 1120.88 17.52 ... For all results, we use an A100-80GiB PCIe card, and a batch size of 32” (Section 4 (Results) and Table 1, Parakeet RNN-T 1.1B, Label Looping + CUDA Graphs row)

Peer-reviewed research

Daniel Galvez et al., Proceedings of Interspeech 2024 (arXiv:2406.03791), 2024 · DOI 10.48550/arXiv.2406.03791

Read on October 19, 2026

What it is: An NVIDIA research paper on GPU decoding for RNN-Transducer speech models. Its Table 1 reports RTFx, the hours of audio transcribed per hour of wall-clock time, for Parakeet models under several decoders, measured on an A100-80GiB PCIe GPU with batch size 32 on LibriSpeech test-other.
How much weight it carries: Peer-reviewed Interspeech paper with an open full text, from the team that implemented the decoders in NVIDIA NeMo. The RTFx figures are offline, batched and in bfloat16, so they are not comparable with batch-size-1 streaming or single-file numbers. The paper measures a decoding optimization; the models themselves are open NVIDIA checkpoints.

In the Galvez et al. 2024 paper, the baseline decoder for Parakeet RNN-T 1.1B ran at an RTFx of 414.63 on an A100-80GiB PCIe GPU at batch size 32, meaning 414.63 times faster than real time.
“Parakeet RNN-T 1.1B Baseline 414.63 67.39 2.7 1 1 ... For all results, we use an A100-80GiB PCIe card, and a batch size of 32” (Section 4 (Results) and Table 1, Parakeet RNN-T 1.1B Baseline row)
In the Galvez et al. 2024 paper, Parakeet RNN-T 1.1B with label looping and CUDA graphs ran at an RTFx of 1120.88 on an A100-80GiB PCIe GPU at batch size 32, meaning about 1,121 times faster than real time.
“Label Looping + CUDA Graphs 1120.88 17.52 ... For all results, we use an A100-80GiB PCIe card, and a batch size of 32” (Section 4 (Results) and Table 1, Parakeet RNN-T 1.1B, Label Looping + CUDA Graphs row)