Represent and generate
The Transformer supplies the generation backbone, while BERT-style encoders help explain dense representations and semantic retrieval.
RAG paper reading path
Understand retrieval-augmented generation from the original RAG paper
Retrieval-Augmented Generation combines knowledge stored in model parameters with external documents that can be retrieved and updated. The original system retrieved Wikipedia passages with DPR and generated answers from the question plus retrieved evidence. It supported knowledge-intensive tasks, but it did not automatically solve bad retrieval, citation accuracy, access control, or production latency.
From foundation models to RAG agents
Read the original RAG paper
Separate parametric from non-parametric memory, then compare how RAG-Sequence and RAG-Token choose documents.
Extend into products and agents
InstructGPT adds instruction following and ReAct adds tool actions. Modern RAG products often combine all three lines.
Five papers that explain RAG
This path separates retrieval, generation, alignment, and tool use so later product features are not incorrectly attributed to the original RAG paper.
- 01
Attention Is All You Need
The Transformer replaced recurrence with attention and became the shared backbone of modern foundation models.
- 02
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
It established bidirectional pre-training followed by task fine-tuning as a general NLP recipe.
- 03
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
It combined parametric memory with retrievable external knowledge, shaping modern knowledge assistants.
- 04
Training language models to follow instructions with human feedback
It established a practical recipe for aligning language models with human instructions and preferences.
- 05
ReAct: Synergizing Reasoning and Acting in Language Models
It interleaved reasoning with tool actions, providing a direct blueprint for modern agent loops.
FAQ
RAG paper FAQ
Who introduced RAG?
Patrick Lewis and colleagues introduced and evaluated the RAG architecture in the 2020 paper Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.
Does RAG eliminate hallucinations?
No. Retrieval can return wrong, stale, or irrelevant evidence, and the generator can ignore or misread it. RAG creates an evidence path, not a guarantee.
How is RAG different from fine-tuning?
RAG retrieves external knowledge at inference time and is easier to update. Fine-tuning changes model weights and is usually better for behavior, format, or specialized capabilities. They can be combined.