Self-improving LLMs through consistency-based self-generated demonstrations
Aspects of the disclosure are directed to automatically selecting examples in a prompt for an LLM to demonstrate how to perform tasks. Aspects of the disclosure can select and build a set of examples from LLM zero-shot outputs via predetermined criteria that can combine consistency, diversity, and repetition. In the zero-shot setting for three different LLMs, using only LLM predictions, aspects of the disclosure can improve performance up to 15% compared to zero-shot baselines and can match or exceed few-shot base-lines for a range of reasoning tasks.
1 . A method for consistency based self-adaptive prompting, comprising:
generating, by one or more processors, a pool of demonstrations using a large language model (LLM) for a plurality of test queries by running chain-of-thought (CoT) over the plurality of test queries;
determining, by the one or more processors, a self-consistency score for respective demonstrations in the pool of demonstrations;
selecting, by the one or more processors, a set of demonstrations from the pool of demonstrations based on the self-consistency scores;
prepending, by the one or more processors, the set of demonstrations to the plurality of test queries; and
generating, by the one or more processors, a plurality of predictions based on the test queries prepended with the set of demonstrations using the LLM.
2 . The method of claim 1 , further comprising receiving, by the one or more processors, the plurality of test queries, each test query being concatenated with a trigger phrase or a labeled demonstration.
3 . The method of claim 1 , wherein the CoT comprises Zero-shot CoT or Few-shot CoT.
4 . The method of claim 1 , wherein the CoT is run multiple times using the LLM to generate multiple reasoning paths and different predictions for each test query.
5 . The method of claim 1 , wherein selecting a set of demonstrations further comprises computing a majority vote prediction and retaining only reasoning paths that result in a majority vote prediction.
6 . The method of claim 5 , wherein the majority vote prediction is computed based on entropy and repetitiveness.
7 . The method of claim 1 , wherein prepending the set of demonstrations further comprises adaptively allocating a number of demonstrations per test query that is proportional to an entropy of the test query.
8 . The method of claim 1 , wherein generating the plurality of predictions further comprises querying the test queries prepended with the set of demonstrations multiple times using the LLM to generate multiple predictions for each test query.
9 . The method of claim 8 , further comprising:
selecting, by the one or more processors, a prediction of the multiple predictions for each test query based on a majority voting; and
outputting, by the one or more processors, the selected prediction for each test query.
10 . A system comprising:
one or more processors; and
one or more storage devices coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations for consistency based self-adaptive prompting, the operations comprising:
generating a pool of demonstrations using a large language model (LLM) for a plurality of test queries by running chain-of-thought (CoT) over the plurality of test queries;
determining a self-consistency score for respective demonstrations in the pool of demonstrations;
selecting a set of demonstrations from the pool of demonstrations based on the self-consistency scores;
prepending the set of demonstrations to the plurality of test queries; and
generating a plurality of predictions based on the test queries prepended with the set of demonstrations using the LLM.
11 . The system of claim 10 , wherein the CoT is run multiple times using the LLM to generate multiple reasoning paths and different predictions for each test query.
12 . The system of claim 10 , wherein selecting a set of demonstrations further comprises computing a majority vote prediction and retaining only reasoning paths that result in a majority vote prediction, the majority vote prediction being computed based on entropy and repetitiveness.
13 . The system of claim 10 , wherein generating the plurality of predictions further comprises querying the test queries prepended with the set of demonstrations multiple times using the LLM to generate multiple predictions for each test query.
14 . The system of claim 13 , wherein the operations further comprise selecting a prediction of the multiple predictions for each test query based on a majority voting.
15 . A non-transitory computer readable medium for storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for consistency based self-adaptive prompting, the operations comprising:
generating a pool of demonstrations using a large language model (LLM) for a plurality of test queries by running chain-of-thought (CoT) over the plurality of test queries;
determining a self-consistency score for respective demonstrations in the pool of demonstrations;
selecting a set of demonstrations from the pool of demonstrations based on the self-consistency scores;
prepending the set of demonstrations to the plurality of test queries; and
generating a plurality of predictions based on the test queries prepended with the set of demonstrations using the LLM.
16 . The non-transitory computer readable medium of claim 15 , wherein the CoT is run multiple times using the LLM to generate multiple reasoning paths and different predictions for each test query.
17 . The non-transitory computer readable medium of claim 15 , wherein selecting a set of demonstrations further comprises computing a majority vote prediction and retaining only reasoning paths that result in a majority vote prediction, the majority vote prediction being computed based on entropy and repetitiveness.
18 . The non-transitory computer readable medium of claim 15 , wherein generating the plurality of predictions further comprises querying the test queries prepended with the set of demonstrations multiple times using the LLM to generate multiple predictions for each test query.
19 . The non-transitory computer readable medium of claim 18 , wherein the operations further comprise selecting a prediction of the multiple predictions for each test query based on a majority voting.