Peer-reviewed research
Does Writing with Language Models Reduce Content Diversity?
Vishakh Padmakumar and He He, International Conference on Learning Representations (ICLR) 2024; arXiv, 2024
Read the originalRead on October 17, 2026
- What it is
- A controlled experiment in which participants wrote argumentative essays in three setups: with a base language model (GPT3), with a feedback-tuned model (InstructGPT), and without model help. The authors measured how similar and how varied the resulting essays were.
- How much weight it carries
- Peer-reviewed and published at ICLR 2024. It tests one writing task, argumentative essays, and two 2023 models, so the size of the effect may not carry over to today's models or to other kinds of writing.
Facts taken from this source
- In a controlled experiment on argumentative essays, writing with the InstructGPT model produced a statistically significant reduction in diversity, while writing with the base GPT3 model did not.
“writing with InstructGPT (but not the GPT3) results in a statistically significant reduction in diversity” (Abstract)
- The authors found that the loss of diversity came mainly from the InstructGPT model's contributions to co-written essays, while the text the participants wrote themselves was not affected.
“this effect is mainly attributable to InstructGPT contributing less diverse text to co-written essays. In contrast, the user-contributed text remains unaffected by model collaboration” (Abstract)