Other
Context Rot: How Increasing Input Tokens Impacts LLM Performance
Kelly Hong et al., Chroma, 2025
Read the originalRead on October 17, 2026
- What it is
- A technical report from the vector database company Chroma, dated July 14, 2025. It tests 18 language models on simple tasks, such as repeating words and answering questions from a long text, while varying only the input length.
- How much weight it carries
- Company technical report, not peer-reviewed, and published by a firm that sells retrieval software. It isolates input length on purpose and releases its code, but the tasks are synthetic and simpler than most chat prompts, and the models have changed since 2025.
Facts taken from this source
- In a 2025 technical report, Chroma found that across all its experiments, model performance consistently degrades as the input gets longer, even on simple tasks.
“Across all experiments, model performance consistently degrades with increasing input length.” (Introduction and Contributions)