ReActSynergizing Reasoning and Acting in Language Models

Authors: Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao
Published: ICLR 2023 · arXiv v3, 10 March 2023
Primary source: arXiv:2210.03629 · PDF
Reading note: Results below come from the paper's reported experiments. Product implications are explicitly marked as interpretation.

Takeaway

Three things to remember

1. ReAct makes thinking and doing a loop. The language model alternates short reasoning traces with actions, then uses observations from a tool or environment to decide what to do next.
2. The loop is most useful when the task needs fresh evidence or state tracking. In the paper, external Wikipedia lookups reduced one kind of hallucination, while reasoning helped agents keep track of long interactive tasks.
3. Grounding does not guarantee correctness. Bad searches, repeated steps, and rigid reasoning can still derail the model. The best results sometimes came from combining ReAct with chain-of-thought self-consistency rather than using ReAct alone.

The core contribution is a prompting pattern, not a new model architecture: let one language model alternate between a private-looking verbal plan and a concrete action that changes or queries the outside world. The paper tests this pattern on question answering, fact verification, a text-based household environment, and an online-shopping simulator. Source: paper pp. 2–3, Sections 1–2.

Teaching visual: the essential ReAct loop. It is a new explanation, not a reproduction of a paper figure.

Problem

Two incomplete halves

Before ReAct, the paper frames two common approaches as incomplete. Chain-of-thought prompting can produce useful multi-step reasoning, but it relies on what the model already contains and cannot directly check the world; an early mistake may flow into later steps. Action-only agents can query tools or move through an environment, but without an explicit reasoning trace they may lose the goal, fail to split it into subgoals, or repeat unhelpful actions. Source: paper pp. 1–2, Introduction.

Why outside feedback matters

A question may depend on a current fact; a shopping task may require reading the available options; a household task may reveal that a drawer is closed. In each case, the next good move depends on an observation that was not available at the start. The paper's problem statement is therefore broader than “make reasoning longer”: it asks how reasoning can direct information-seeking actions, and how returned information can correct the reasoning. Source: paper pp. 1–3, Introduction and Section 2.

Teaching visual: reasoning alone stays internal; acting alone lacks an explicit planning trace; ReAct connects the two.

Method

Thought, action, observation

ReAct prompts a language model with a small number of example trajectories. During a task, the model emits a thought when planning or updating is useful, then an action in the environment's allowed format. The environment returns an observation, which becomes part of the context for the next model step. This continues until the model emits a finish action or reaches the task's step limit. Thoughts do not directly change the environment; actions do. Source: paper pp. 2–4, Section 2 and Section 3.2.

Sparse reasoning, not constant narration

The pattern changes by task. For knowledge tasks, the trajectory is dense: thought, Wikipedia search or lookup, observation, then another thought. For interactive decision tasks, the paper uses reasoning only at strategically important moments, such as forming a plan, noticing progress, or handling an exception; ordinary environment actions can occur between thoughts. Source: paper p. 3, Section 2; pp. 7–8, Section 4.

What was actually tested

The authors used PaLM-540B for the main prompting experiments, with one to six in-context examples depending on the task; the two interactive tasks used only one or two. The four benchmarks were HotpotQA (multi-hop question answering), FEVER (fact verification), ALFWorld (text-based household tasks), and WebShop (shopping through webpages). HotpotQA and FEVER used a simple Wikipedia interface with search, lookup, and finish actions. Source: paper pp. 1 and 3–4 and pp. 7–8, Abstract and Sections 2–4.

Evidence and limits

What the numbers show

The strongest interactive result is on ALFWorld: the best of six ReAct prompt sets reached 71% success, versus 45% for the best action-only prompt set and 37% for BUTLER. On WebShop, ReAct reached 40.0% success, compared with 30.1% for action-only prompting and 28.7% for the imitation-plus-reinforcement-learning baseline; expert humans reached 59.6%. Source: paper p. 8, Tables 3–4.

The knowledge results are more nuanced. On FEVER, ReAct scored 60.9% accuracy, above chain-of-thought at 56.3%. On HotpotQA, however, ReAct scored 27.4 exact match, below chain-of-thought at 29.4 and self-consistent chain-of-thought at 33.4. Hybrid fallbacks performed best in the table: ReAct followed by self-consistent chain-of-thought reached 35.1 on HotpotQA, while the reverse order reached 64.6% on FEVER. A supervised task-specific baseline was still much higher at 67.5 and 89.5, respectively. Source: paper p. 5, Table 1.

What can go wrong

In a manual analysis of 200 HotpotQA trajectories, hallucinated reasoning or facts appeared in 6% of ReAct's nominally successful examples versus 14% for chain-of-thought. But among ReAct failures, 47% were classified as reasoning errors and 23% as unhelpful search results. The authors also observed repeated thought-action loops and label ambiguity. These percentages describe a small, hand-labeled diagnostic sample, not a universal failure rate. Source: paper p. 6, Table 2 and accompanying analysis.

Teaching visual: external evidence reduces some errors but creates new dependencies on search quality and recovery behavior.

The experiments used older, very large language models, narrow text interfaces, small numbers of prompt examples, greedy decoding in most cases, and benchmark environments rather than open-ended deployment. Prompting smaller PaLM models performed poorly; the authors' initial fine-tuning experiment used 3,000 model-generated correct trajectories and suggests potential, but does not establish broad real-world reliability. Source: paper pp. 5–9, Sections 3.3–5.

Practical meaning

A useful agent design pattern

Interpretation: ReAct is a good fit when a task can be expressed through clear tools, observations are compact enough to feed back to the model, and intermediate choices benefit from explicit state tracking. Examples include research assistants that search before answering, support agents that inspect account state, and workflow agents that verify each step before continuing. The paper supports the mechanism and benchmark behavior; it does not test those products directly.

Questions to ask before shipping

• Can every action be constrained to a small, auditable tool interface?
• What happens when search returns nothing, returns noise, or contradicts earlier evidence?
• Is there a step limit, loop detector, and safe fallback?
• Are thoughts necessary to expose, or would a shorter action-and-evidence log be safer and easier to review?
• Does the evaluation separate task success, factual support, tool cost, latency, and recovery from failure?

The durable lesson is not that models should narrate every step. It is that an agent should alternate between deciding what information or state change it needs and checking what actually happened. ReAct made that loop simple enough to study—and also made its weak points visible.