Fact · Voice and AI
The Vectara leaderboard asks each model to summarize every document using only the facts in that document, then scores the summary for factual consistency.
Checked against its source on October 17, 2026
In the source's own words
“fed the full set of documents in the dataset to each of the LLMs and asked them to summarize each document, using only the facts presented in the document.”
Vectara Hallucination Leaderboard: LLM hallucination rates when summarizing a document, Vectara, Vectara, GitHub, 2026, Methodology section.
Open the original- What it measures
- The leaderboard runs each model at temperature 0 and reports a hallucination rate, which is 100 minus the factual consistency rate, along with the rate at which the model refuses to answer.
- What it does not show
- The test uses one dataset of documents and one prompt. It measures whether a summary is consistent with its source, not whether it kept the points a speaker cared about.