Preprint
LAHAJA: A Robust Multi-accent Benchmark for Evaluating Hindi ASR Systems
Tahir Javed et al., arXiv preprint (AI4Bharat, IIT Madras), 2024
Read the originalRead on October 17, 2026
- What it is
- A Hindi speech benchmark of 12.5 hours from 132 speakers in 83 Indian districts, built to test how well systems handle different accents. The authors report that existing open-source and commercial models perform poorly on it.
- How much weight it carries
- A preprint from the group that built the benchmark. At 12.5 hours it shows trends across accents rather than precise error rates.
Facts taken from this source
- The LAHAJA Hindi benchmark contains 12.5 hours of Hindi audio from 132 speakers across 83 districts of India.
“with a total of 12.5 hours of Hindi audio, sourced from 132 speakers spanning 83 districts of India” (Abstract)
- In the LAHAJA study, recognition performance declined for speakers from North-East and South India, especially on named entities and specialized terms.
“the performance declines for speakers from North-East and South India, especially with content heavy in named entities and specialized terminology” (Abstract)
- The LAHAJA authors evaluated existing open-source and commercial Hindi models and found their performance poor.
“We evaluate existing open-source and commercial models on LAHAJA and find their performance to be poor.” (Abstract)