Fact · Languages and accents
The LAHAJA authors evaluated existing open-source and commercial Hindi models and found their performance poor.
Checked against its source on October 17, 2026
In the source's own words
“We evaluate existing open-source and commercial models on LAHAJA and find their performance to be poor.”
LAHAJA: A Robust Multi-accent Benchmark for Evaluating Hindi ASR Systems, Tahir Javed et al., arXiv preprint (AI4Bharat, IIT Madras), 2024, Abstract.
Open the original- What it measures
- The authors tested models that were available at the time (2024) on Indian-accented Hindi, not on studio Hindi.
- What it does not show
- The abstract does not give a single headline error rate per model. Newer models may do better, so read this as a warning about accents, not a ranking.