Glossary
What is self-consistency prompting
In short
Self-consistency prompting is a technique in which a language model solves one task several times by different paths. The final answer is the one that most solutions reach.
Prompts · self-consistency
MMOOZZGG.com editorsChecked: 1 min read
For a child
Imagine that you and your friends are counting the candies in a big jar. Each of you counts alone and says a number. One gets forty, three get forty-two, and the last one gets forty-five. Most likely the right answer is forty-two, because most of you got it. A computer sometimes does the same: it solves a problem several times and picks the answer that came up most often.
In plain words
An AI model does not answer by a fixed rule: every answer has some randomness. So on a task with a calculation it can be right once and wrong the next time. This technique turns that weakness into a benefit. The model is asked to solve the task step by step many times, and each time the reasoning takes its own path. Then only the final numbers are compared, and the one that appears most often wins. The mistakes are usually different, while the right answer repeats. It is like several accountants checking an estimate on their own and trusting the figure most of them got.
For business
The method raises accuracy where a task has one right answer: the sum in a contract, the category of a request, a check of an application against the rules. The cost is direct: each extra run is paid for separately, and the answer comes later. So it is used only where a mistake is expensive. It also helps control: if the answers differ a lot, the system is not sure, and the case goes straight to an employee. For open tasks such as emails, voting does not work, because no two texts are the same. Test on real data how many runs pay off, and look at AI errors and who is responsible for them before launch.
For a specialist
Self-consistency was proposed by Wang and co-authors in 2022 (published at ICLR 2023) as a replacement for greedy decoding in chain-of-thought reasoning. Instead of one chain, the model samples a set of diverse reasoning paths and then marginalizes over them. The result is the answer that most paths reach, which is in effect a majority vote on the final answers. The method needs no fine-tuning, no extra model and no labeling. The authors gained over plain chain of thought on arithmetic and common-sense tasks: GSM8K by 17.9%, SVAMP by 11.0%, AQuA by 12.2%, StrategyQA by 6.4% and ARC-challenge by 3.9%. The limits: cost grows linearly with the number of samples, and voting works only when answers can be compared for a match, such as a number or an answer option. For free text the method does not work in its original form. The spread of answers also serves as a measure of the model’s uncertainty. A close idea, with branching and scoring of the middle steps, is the tree of thoughts.
Example
An accounting bot calculates a contract penalty five times. Four runs give one sum and the fifth gives another. The system sends the client the majority sum and logs the difference for a check.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map