Chain-of-Thought Prompting and the Roots of Reasoning
By Satwik ยท March 19, 2026
Chain-of-thought prompting showed that asking a large model to reason step by step sharply improves its performance on multi-step problems. It exposed reasoning as an elicitable capability and set the stage for the deliberate, compute-at-inference systems that followed.
A one-line change with outsized effect
In 2022, researchers at Google showed that a simple modification to how you prompt a large language model produced large gains on hard, multi-step problems. Instead of asking for the answer directly, you prompt the model to work through intermediate steps, to "think step by step," either by including a few worked examples that show the reasoning or, in the zero-shot variant, by literally appending that instruction. The model then writes out a chain of intermediate reasoning before giving its final answer. This is chain-of-thought prompting.
The effect on arithmetic word problems, symbolic manipulation, and commonsense reasoning was substantial. Tasks where models had been stuck became tractable, without any change to weights or architecture. The capability was already latent; the prompt unlocked it.
Why generating steps helps
A Transformer does a bounded amount of computation per token it produces. When it must answer a multi-step problem in a single step, it has to compress the entire chain of inference into one forward pass, and for genuinely sequential problems that is often not enough computation to get there. Forcing the model to emit intermediate tokens gives it more forward passes, and each generated step becomes part of the context conditioning the next, so the model can decompose the problem and build on its own partial results. The written reasoning is a scratchpad, external working memory that lets the model spend more computation, laid out across tokens, on a hard problem.
Two properties stood out. First, chain-of-thought helped mainly at large scale; for smaller models the generated reasoning was often incoherent and did not improve, and could even hurt, accuracy. Like in-context learning, effective step-by-step reasoning behaved as an emergent property of scale. Second, the reasoning steps could be aggregated: sampling several independent chains and taking a majority vote over their final answers, self-consistency, improved reliability further, because independent reasoning paths that agree are more likely correct.
Why it mattered
Chain-of-thought reframed what large models are for. Before it, they were treated largely as pattern completers and knowledge retrievers. After it, it was clear they could carry out genuine multi-step procedures if prompted to externalize the process, which opened planning, tool use, and complex problem solving as serious application areas.
It also introduced a new axis of improvement: inference-time compute. Rather than only making models bigger or training them longer, you could get better answers by having the model generate more, longer reasoning, or many reasoning paths, at the moment of answering. This idea, that you can trade compute at inference for accuracy, matured into a central theme, culminating in later models explicitly trained to produce long internal reasoning traces and to spend variable amounts of thinking on a problem. Chain-of-thought prompting is the conceptual seed of that entire line of work.
The security angle
Chain-of-thought raises two distinct and somewhat opposed security considerations, and holding both in mind is the mature position.
On the transparency side, an externalized reasoning trace is a partial window into how the model reached its answer. It offers a hook for oversight: you can inspect the steps, check them, and in principle catch errors or detect when the model is pursuing a problematic path. Some safety proposals lean on this, using the visibility of reasoning as a monitoring surface. But the window is unreliable. Research has repeatedly shown that the stated reasoning is not guaranteed to be the model's actual computation. A model can produce a fluent, plausible chain that rationalizes an answer it reached by other means, and its final answer can depend on cues the chain never mentions. Trusting a chain of thought as a faithful account of the model's process is therefore hazardous; it is a plausible narrative, not a verified log, and treating unfaithful reasoning as trustworthy is its own failure mode.
On the capability side, chain-of-thought is an elicitation technique, and elicitation is exactly what capability evaluation and misuse both rely on. The same prompting that unlocks legitimate reasoning can unlock latent dangerous problem-solving that a direct question would not surface, which sharpens the overhang concern: a model's measured capability depends heavily on how hard you prompt it, so a naive safety evaluation that asks questions directly can badly underestimate what a determined user can extract by guiding the model through steps. Reasoning also extends the surface for injected instructions, since a long generated trace gives adversarial content in the context more room to steer the process. Chain-of-thought, in short, made models more capable and made their capability harder to bound, which is precisely the tension that reasoning-era safety work now lives inside.