Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Paper: Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou · arXiv:2201.11903 · submitted 28 January 2022, revised 10 January 2023 · primary source · full text

Takeaway

The paper makes one small but powerful change to few-shot prompting: its worked examples show not only a question and final answer, but also a short sequence of intermediate reasoning steps. In the paper's experiments, this often helped very large language models solve multi-step arithmetic, commonsense, and symbolic tasks that standard prompting handled poorly. It did not reliably help smaller models, and the written steps were not guaranteed to be correct or to reveal the model's true internal process. Source: abstract; Sections 2, 3.2, and 6

Three points to remember

1. Show the path, not just the destination. A few worked examples can teach a sufficiently large model to produce intermediate steps before its answer.
2. The benefit depended strongly on scale and difficulty. The clearest gains appeared in very large models and harder, multi-step problems; easy one-step problems gained little.
3. Readable reasoning is useful evidence, not proof. It can expose mistakes, but a fluent chain can still be wrong, and it does not establish what happened inside the network.

Visual 1 — The intervention is in the examples: add intermediate natural-language steps between each question and answer. This is a new teaching diagram, not a reproduction of a paper figure. Source basis: Sections 1–2 and 3.1.

Problem

Scaling alone left a reasoning gap

Larger language models had improved many language tasks, yet standard prompting still struggled on arithmetic word problems, commonsense questions, and symbolic manipulation. A standard few-shot prompt demonstrates only input–output pairs. That format tells the model what answer shape is wanted, but gives no example of how to break a multi-step problem apart. Source: Introduction, paragraphs 1–6

Earlier routes had a cost

Earlier work could train or fine-tune models on natural-language rationales, but assembling a large collection of high-quality, step-by-step annotations is expensive. Ordinary few-shot prompting is cheaper and reuses one model across tasks, but at the time performed poorly on reasoning tasks. The paper asks whether a handful of rationale-bearing examples can combine the advantages of both approaches without training a new model. Source: Introduction and Related Work

Method

The prompt becomes a triple

Each demonstration is changed from input → output to input → chain of thought → output. The chain is a sequence of natural-language intermediate steps. At test time, the model receives a new question and generates its own steps before its final answer. The model weights are not updated. Source: Section 2; Section 6

A small set of worked examples

For most arithmetic benchmarks, the authors manually wrote one set of eight chain-of-thought demonstrations and reused it across datasets; AQuA used four because it is multiple choice. The broader evaluation covered five model families—GPT-3, LaMDA, PaLM, UL2, and Codex—and tasks in arithmetic, commonsense, and symbolic reasoning. Outputs were generated with greedy decoding; LaMDA results were averaged across five shuffled exemplar orders, while other model results used one order. Source: Section 3.1

What the ablations test

The authors compared the full method with three alternatives: produce only an equation, spend extra tokens by emitting dots, or put the reasoning after the answer. On GSM8K, none matched the full chain-of-thought prompt. This supports the narrower claim that ordered natural-language steps were useful in this setup; it does not by itself reveal the model's internal mechanism. Source: Section 3.3

Visual 2 — A qualitative reading of the reported scaling pattern, not a numeric chart or a universal parameter threshold. Source basis: Section 3.2, Figure 4, and Appendix Table 2.

Evidence and limits

The strongest arithmetic result

On GSM8K, PaLM 540B rose from 17.9% with standard prompting to 56.9% with chain-of-thought prompting, a gain of 39.0 percentage points. The same model improved from 69.4% to 79.0% on SVAMP and from 79.2% to 93.3% on MAWPS. But the effect was uneven: on ASDiv the increase was only 72.1% to 73.9%. Source: Appendix Table 2

Breadth, scale, and robustness

The method also improved several commonsense tasks: PaLM 540B reached 75.6% on StrategyQA versus a cited prior best of 69.4%, and 95.4% on the sports task versus 84% for the paper's unaided sports-enthusiast comparison. On two synthetic symbolic tasks, it helped models handle sequences longer than those in the demonstrations. Across arithmetic experiments, different annotators, example sets, orders, and example counts still generally beat the standard baseline. Yet small models often produced fluent but illogical chains; the paper describes useful gains as appearing around the 100B-parameter scale in the tested families, not as a law that fixes an exact threshold for all models. Source: Sections 3.2, 3.4, 4, and 5

What the paper does not establish

The study does not prove that the network is “reasoning” in a human sense, or that its written explanation faithfully reports hidden computation. It offers no guarantee that a chain is correct. In a manual review of 50 wrong LaMDA 137B answers on GSM8K, 46% had nearly correct chains with a small error and 54% had major semantic or coherence errors; among 50 correct answers, two reached the right answer through an incorrect chain. The authors also flag serving cost, because the effect appeared only in very large models, and annotation cost if the idea were expanded from a few examples to fine-tuning data. Source: Sections 3.2 and 6

Visual 3 — The output is inspectable but not guaranteed correct or faithful to hidden computation. Source basis: Sections 2, 3.2, and 6.

Practical meaning

Use it where decomposition matters

For a practitioner, the paper suggests a simple experiment: when a task genuinely needs several linked decisions, place a few correct, concise worked solutions in the prompt and compare against a direct-answer baseline on held-out examples. Treat the result as task- and model-specific. The paper gives little reason to expect a large gain on easy one-step tasks, and its findings come from 2022-era model families rather than every current system. This is an interpretation of the evidence, not an author guarantee.

Inspect answers, not just eloquence

Intermediate steps make failures easier to spot, but fluent prose should not be used as a confidence score. Check the final answer independently when stakes matter; look for arithmetic slips, missing steps, misunderstood premises, and prompt sensitivity. Also account for the extra output tokens and latency. The paper establishes improved benchmark accuracy in its tested settings—not dependable truthfulness, causal faithfulness, or safe deployment.

Verification notes

• Primary record: The official title, author list, submission history, abstract, DOI, and paper links are on the arXiv record.
• Exact results: Arithmetic values above come from Appendix Table 2; commonsense results and comparisons come from Section 4.
• Open questions: How much of the effect comes from better task decomposition versus other prompt cues? How stable is it across modern models and adversarial inputs? When do visible steps faithfully track the computation that produced the answer? The paper leaves these unresolved.