Chain-of-Thought Prompting: Making AI Show Its Steps

Chain-of-thought prompting is a way of asking an AI language model to write out the intermediate steps of its reasoning before it gives a final answer. In a 2022 research paper, the technique improved results on maths, common-sense and symbolic puzzles, but only in large models. This article explains the idea, what the authors reported and why it is worth treating with some care.

The basic idea

A language model produces text one piece at a time. If you ask a hard question and expect the answer straight away, the model has to jump from question to result in one go. For a multi-step problem, such as a maths word problem, that is difficult. Chain-of-thought prompting changes the prompt so that the model writes the in-between steps first.

The method in the paper is simple. The prompt includes a few worked examples. Each example has a question, then the reasoning written out step by step, then the answer. The model sees the pattern and tends to answer a new question in the same style, with its own steps first.

A small example

Suppose the question is: a shop sells pens at 3 for 90 rupees, and you buy 12 pens. A direct answer is just a number. A chain-of-thought answer reads more like this: 12 pens is 4 groups of 3 pens; each group costs 90 rupees; 4 times 90 is 360; the answer is 360 rupees. Each step is small and easy to check. This example is made up to show the format; it is not taken from the paper.

What the paper reported

The paper is titled Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Its authors are Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le and Denny Zhou. It was first posted on arXiv on 28 January 2022, revised several times (the latest version is dated 10 January 2023), and published in the proceedings of NeurIPS 2022.

According to the abstract, the authors tested the method on arithmetic, common-sense and symbolic reasoning tasks, across three language models, and saw improvements that were sometimes large. Their headline result was on GSM8K, a benchmark of maths word problems. A model with 540 billion parameters, prompted with just eight chain-of-thought examples, reached what the authors call state-of-the-art accuracy, beating a version of GPT-3 that had been fine-tuned and paired with a separate verifier.

It works mainly in large models

The abstract says this reasoning ability appears in sufficiently large models. In other words, the technique is not a trick that helps every system. Smaller models can write steps that look reasonable but do not lead to the right answer, so the gain depends on the size of the model. The paper's wording is about models that are large enough; the exact threshold is a research question, and the abstract does not give one.

Why writing steps can help

The paper presents several reasons why this might help, and they are best read as explanations offered by the authors rather than proven facts about how the model works inside. One is that a problem can be broken into smaller parts, each of which is easier to get right. Another is that the written steps give the model more text to work from as it continues, so later steps can use earlier ones. A third is that people reading the output can see where an answer came from and spot a mistake.

• It needs no extra training, only a change to the prompt.
• The steps make the answer easier for a person to inspect.
• The same style of prompt can be used for different kinds of problems.

Limits worth knowing

A written chain of reasoning is not a guarantee of a correct answer. A model can write steps that read well and still contain an arithmetic slip or a wrong assumption, and the final answer can be wrong. Longer answers also use more tokens, the small units of text a model reads and writes, so they take longer and can cost more where usage is billed per token.

It is also worth being careful about what the steps mean. Whether the written reasoning matches what is actually happening inside the model is a separate question, and this article does not claim to answer it. The paper's claim is narrower: prompting with step-by-step examples improved measured accuracy on the tasks it tested.

Where the idea shows up today

The paper helped popularise asking a model to think step by step, and many later techniques build on the same observation that more written reasoning can improve results on hard problems. Which products use such methods, and how, varies and changes quickly, so check each provider's own documentation for current details.

Sources

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, arXiv:2201.11903
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, NeurIPS 2022 Proceedings