Self-Consistency Decoding

By Satwik · February 27, 2026

Self-consistency is a decoding strategy that builds directly on chain-of-thought. Instead of greedily generating a single reasoning path, you sample many diverse reasoning paths at nonzero temperature and then take a majority vote over the final answers. The intuition is that correct reasoning can arrive by several routes while errors are more idiosyncratic, so the answer that multiple independent chains converge on is more likely right.

The empirical result was a solid boost on reasoning benchmarks over plain chain-of-thought, with no change to the model or training - purely an inference-time technique that trades compute for accuracy.

Why it belongs in our notes: it is another example of elicitation without retraining. Sampling and aggregation can meaningfully raise a model's effective capability, which again means benchmark scores from single-sample greedy decoding understate what a determined user can extract. When we assess whether a model can do something risky, we should assume access to majority-vote-over-samples, not just one shot.

There is also a cost lens. Self-consistency makes accuracy a knob you buy with test-time compute, foreshadowing a broader trend of spending inference compute to improve outputs. That has downstream implications for how we reason about the marginal cost of a capable, reliable system - a motivated actor can often close a reliability gap simply by sampling more.

It is a small, clean idea with an outsized practical footprint, and it pairs so naturally with chain-of-thought that the two are usually deployed together.