Peer-reviewed research
Understanding the Ability of LLMs to Handle Character-Level Perturbation
Anyuan Zhuo et al., ICML 2026, 2026
Read the originalRead on October 17, 2026
- What it is
- A study of how large language models cope with heavy character-level noise: many typos within words, shuffled letters, and invisible characters. It finds that many models keep much of their performance under severe noise, and links this to how they split text into tokens.
- How much weight it carries
- Listed on the ICML 2026 program as a poster, so it has been through the conference review process. It is a counterpoint to the adversarial typo studies, because its noise is random rather than chosen to cause failure. Results depend on the models and tasks tested, which are summarized on the poster page.
Facts taken from this source
- In a 2026 ICML study, many large language models kept notable performance even when most words were shuffled letter by letter or many invisible characters were inserted.
“Surprisingly, even under severe perturbation, such as shuffling nearly all words character-wise to produce text that is almost unreadable to humans, or inserting invisible characters which are several times more than the visible ones as noise, many LLMs still maintain notable performance.” (Abstract)