Fact · Voice and AI
In a 2026 ICML study, many large language models kept notable performance even when most words were shuffled letter by letter or many invisible characters were inserted.
Checked against its source on October 17, 2026
In the source's own words
“Surprisingly, even under severe perturbation, such as shuffling nearly all words character-wise to produce text that is almost unreadable to humans, or inserting invisible characters which are several times more than the visible ones as noise, many LLMs still maintain notable performance.”
Understanding the Ability of LLMs to Handle Character-Level Perturbation, Anyuan Zhuo et al., ICML 2026, 2026, Abstract.
Open the original- What it measures
- The study tested three kinds of character noise: many typos inside words, shuffled letters, and invisible characters. The authors link the robustness to how models split text into tokens.
- What it does not show
- The noise was random, not chosen to cause failure, and it is much heavier than a few speech recognition errors. Some models did much worse than others, and the abstract does not give a single average.