Peer-reviewed research
Language Models are Few-Shot Learners
Tom B. Brown et al., Advances in Neural Information Processing Systems 33 (NeurIPS 2020); arXiv preprint, 2020
Read the originalRead on October 17, 2026
- What it is
- The paper that introduced GPT-3, an autoregressive language model with 175 billion parameters, and tested it on many language tasks with few or no examples in the prompt, without gradient updates.
- How much weight it carries
- Peer-reviewed at NeurIPS 2020 and widely cited. It describes a 2020 model; later systems are larger and differently trained, so the parameter count is a historical example rather than a current benchmark.
Facts taken from this source
- The 2020 GPT-3 paper describes GPT-3 as an autoregressive language model with 175 billion parameters, 10 times more than any previous non-sparse language model.
“an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model” (Abstract)