Fact · Voice and AI
In a 2025 technical report, Chroma found that across all its experiments, model performance consistently degrades as the input gets longer, even on simple tasks.
Checked against its source on October 17, 2026
In the source's own words
“Across all experiments, model performance consistently degrades with increasing input length.”
Context Rot: How Increasing Input Tokens Impacts LLM Performance, Kelly Hong et al., Chroma, 2025, Introduction and Contributions.
Open the original- What it measures
- The report tests 18 models, including Claude, GPT-4.1, Gemini 2.5 and Qwen3 families, on tasks such as repeating words and answering questions from a long text, changing only the input length.
- What it does not show
- A company report on synthetic tasks, not a study of chat prompts. It shows that longer input can cost accuracy, not by how much in a typical prompt. Its authors say real long-context work is more complex.