Peer-reviewed research
Extracting Training Data from Large Language Models
Nicholas Carlini et al., 30th USENIX Security Symposium, 2021
Read the originalRead on October 17, 2026
- What it is
- A peer-reviewed study that queried GPT-2 and recovered hundreds of verbatim sequences from its training data, including names, phone numbers, email addresses and UUIDs. The authors show that rare sequences that appear in a single training document can still be extracted.
- How much weight it carries
- Peer-reviewed at a leading security venue, and widely cited. The experiment used GPT-2, a model from 2019, on public web text. Commercial dictation models are trained on different data and are not tested here, so the paper shows that memorization happens, not how often it happens in a given product.
Facts taken from this source
- The Carlini et al. study recovered hundreds of verbatim text sequences from the training data of GPT-2.
“are able to extract hundreds of verbatim text sequences from the model's training data” (Abstract)
- The Carlini et al. extraction attack worked on sequences that appeared in only one document in the training data.
“Our attack is possible even though each of the above sequences are included in just one document in the training data.” (Abstract)