Granted Patent B2

US 12,632,651 · App. 18/477,680 · Granted May 19, 2026

Inventors: Reid Allen Pryzant (Seattle, WA); Jerry Zheng Li (Redmond, WA); Dan Iter (Cedar Park, TX); Yin Tat Lee (Seattle, WA); Chenguang Zhu (Issaquah, WA); Nanshan Zeng (Bellevue, WA); Anup Shirgaonkar (New York, NY)

Assignee: Microsoft Technology Licensing, LLC

G06F40/20G06F40/166G06F16/90

Abstract

Claims (68)

1 . A system for implementing automatic prompt optimization using textual gradients, the system comprising:

at least one processor; and

memory storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations comprising:

providing, as input to a large language model (“LLM”), a first feedback prompt requesting one or more first textual gradients, each first textual gradient including a description of one or more first flaws in an initial prompt resulting in errors in LLM predictions, the first feedback prompt including an initial prompt to be optimized and one or more first predictions that are incorrect compared with corresponding one or more labels associated with a batch of data for which the initial prompt was used to generate the one or more first predictions;

receiving, from output of the LLM in response to the first feedback prompt, the one or more first textual gradients;

providing, as input to the LLM, a first editing prompt requesting a first set of optimized prompts, based on the initial prompt and the one or more first textual gradients;

receiving, from output of the LLM, the first set of optimized prompts;

selecting one or more first optimized prompts from at least the first set of optimized prompts based at least in part on evaluation of prompt performance using a secondary LLM that is finetuned based on a curated dataset for a specific subject area;

instructing the LLM to perform a task focused on the subject area by inputting the selected one or more first optimized prompts into the LLM; and

receiving, from the LLM, results for the instructed task.

2 . The system of claim 1 , wherein the first feedback prompt further includes at least one of the batch of data and the one or more labels corresponding to the one or more first predictions that are incorrect.

3 . The system of claim 1 , wherein the set of operations further comprises:

providing, as input to the secondary LLM, the selected one or more first optimized prompts each requesting a second prediction based on the batch of data;

receiving, from output of the secondary LLM, the second prediction for each of the selected one or more first optimized prompts;

comparing each second prediction with labels contained in the batch of data that is processed by the LLM using each of the selected one or more first optimized prompts; and

based on the comparison, identifying one or more second predictions that are incorrect.

4 . The system of claim 3 , wherein the set of operations further comprises:

providing, as input to the LLM, a second feedback prompt requesting one or more second textual gradients, the second feedback prompt including the selected one or more first optimized prompts and the one or more second predictions that are incorrect compared with labels contained in the batch of data;

receiving, from output of the LLM, the one or more second textual gradients, each second textual gradient including a description of one or more second flaws in one of the selected one or more first optimized prompts;

providing, as input to the LLM, a second editing prompt requesting a second set of optimized prompts, based on the selected one or more first optimized prompts and the one or more second textual gradients;

receiving, from output of the LLM, the second set of optimized prompts; and

selecting one or more second optimized prompts from at least the second set of optimized prompts based at least in part on evaluation of prompt performance using the secondary LLM,

wherein instructing the LLM to perform the task focused on the subject area is performed by inputting the selected one or more second optimized prompts into the LLM.

5 . The system of claim 4 , wherein at least one of the first feedback prompt, the second feedback prompt, the first editing prompt, or the second editing prompt is at least one of generated by the LLM or by a second LLM.

6 . The system of claim 4 , wherein at least one of providing the first feedback prompt, providing the second feedback prompt, providing the first editing prompt, or providing the second editing prompt is performed using an application programming interface (“API”) call to the LLM.

7 . The system of claim 1 , wherein the set of operations further comprises:

providing, as input to the LLM, a first paraphrasing prompt requesting a first set of paraphrased optimized prompts, based on the first set of optimized prompts; and

receiving, from output of the LLM, the first set of paraphrased optimized prompts;

wherein selecting the one or more first optimized prompts comprises selecting from at least one of the first set of optimized prompts or the first set of paraphrased optimized prompts.

8 . The system of claim 7 , wherein selecting the one or more first optimized prompts is performed using one or more selection algorithms comprising a selection algorithm based on a scoring metric, wherein the one or more first optimized prompts are selected based on whether each optimized prompt scores above a set threshold scoring metric values.

9 . The system of claim 7 , further comprising preselecting one or more of gradients, optimized prompts, or paraphrased optimized prompts by performing at least one of:

preselection using a trained classifier;

conversion of each of the one or more of gradients, optimized prompts, or paraphrased optimized prompts into corresponding embeddings, and preselection based on distances between resultant embeddings within corresponding embedding space;

preselection using the LLM itself or another LLM; or

preselection based on a chain of thought-based preselection process.

10 . The system of claim 1 , wherein the batch of data comprises at least one of a random sample of natural language (“NL”) training data or a curated sample of the NL training data that has been labelled as difficult example training data.

11 . A computer-implemented method for implementing automatic prompt optimization using textual gradients, the method comprising:

receiving one or more textual gradients after providing a feedback prompt as input to a large language model (“LLM”), the feedback prompt including an initial prompt to be optimized and one or more predictions that are incorrect compared with corresponding one or more labels associated with a batch of data for which the initial prompt was used to generate the one or more predictions;

receiving, in response to the feedback prompt, a set of optimized prompts after providing an editing prompt as input to the LLM, the editing prompt including the initial prompt and the one or more textual gradients, each textual gradient including a description of one or more flaws in the initial prompt;

receiving a set of paraphrased optimized prompts after providing a paraphrasing prompt as input to the LLM, the paraphrasing prompt including the set of optimized prompts;

selecting one or more optimized prompts from at least one of the set of optimized prompts or the set of paraphrased optimized prompts; and

repeating, until a set condition has been met, the processes of receiving the one or more textual gradients, receiving the set of optimized prompts, receiving the set of paraphrased optimized prompts, and selecting the one or more optimized prompts, wherein, for each successive iteration, the initial prompt or a previously selected group of one or more optimized prompts is replaced with a latest selected group of one or more optimized prompts that is selected during each previous iteration.

12 . The computer-implemented method of claim 11 , wherein the set condition comprises a set number of iterations, and wherein the method further comprises:

sampling a subset of optimized prompts from at least one of the set of optimized prompts or the set of paraphrased optimized prompts; and

receiving an average score after providing the sampled subset of optimized prompts as input to the LLM to output scores corresponding to the sampled subset of optimized prompts and averaging resultant scores, the sampled subset of optimized prompts each including the batch of data;

wherein selecting the one or more optimized prompts comprises selecting based on the average score.

13 . The computer-implemented method of claim 12 , wherein selecting the one or more optimized prompts comprises one of selecting a first number of the one or more optimized prompts that have scores above the average score or selecting a remaining number of the one or more optimized prompts after removing a second number of the one or more optimized prompts that have scores below the average score.

14 . A system for implementing automatic prompt optimization using textual gradients, the system comprising:

a processing system; and

memory coupled to the processing system, the memory comprising computer executable instructions that, when executed by the processing system, causes the system to perform operations comprising:

providing, as input to a large language model (“LLM”), a feedback prompt requesting one or more textual gradients, the feedback prompt including an initial prompt to be optimized and one or more predictions that are incorrect compared with corresponding one or more labels associated with a batch of data for which the initial prompt was used to generate the one or more predictions;

receiving, from output of the LLM in response to the feedback prompt, the one or more textual gradients, each textual gradient including a description of one or more flaws in the initial prompt;

providing, as input to the LLM, an editing prompt requesting a set of optimized prompts, based on the initial prompt and the one or more textual gradients;

receiving, from output of the LLM, the set of optimized prompts;

sampling a subset of optimized prompts from at least the set of optimized prompts;

providing, as input to the LLM, the sampled subset of optimized prompts requesting an average score based on averaging resultant scores corresponding to the sampled subset of optimized prompts, the sampled subset of optimized prompts each including the batch of data;

receiving, from output of the LLM, the average score;

selecting one or more optimized prompts based on the average score; and

repeating, until a set condition has been met, the processes of providing the feedback prompt, receiving the one or more textual gradients, providing the editing prompt, receiving the set of optimized prompts, sampling the subset of optimized prompts, providing the sampled subset of optimized prompts, receiving the average score, and selecting the one or more optimized prompts, wherein, for each successive iteration, the initial prompt or a previously selected group of one or more optimized prompts is replaced with a latest selected group of one or more optimized prompts that is selected during each previous iteration.

15 . The system of claim 14 , wherein the set condition comprises one of a set number of iterations or a determined level of match between comparison of labels contained in the batch of data and one or more subsequent predictions that are generated using the selected one or more optimized prompts.

16 . The system of claim 14 , wherein the set of operations further comprises, for each iteration:

providing, as input to the LLM, a paraphrasing prompt requesting a set of paraphrased optimized prompts, based on the set of optimized prompts; and

receiving, from output of the LLM, the set of paraphrased optimized prompts;

wherein selecting the one or more optimized prompts comprises selecting from at least one of the set of optimized prompts or the set of paraphrased optimized prompts.

17 . The system of claim 16 , wherein at least one of the feedback prompt, the editing prompt, or the paraphrasing prompt is at least one of generated or optimized by the LLM.

18 . The system of claim 16 , wherein at least one of providing the feedback prompt, providing the editing prompt, providing the paraphrasing prompt, or providing the sampled subset of optimized prompts is performed using an application programming interface (“API”) call to the LLM.

19 . The system of claim 16 , wherein selecting the one or more optimized prompts is performed using one or more selection algorithms.

20 . The system of claim 19 , wherein selecting the one or more optimized prompts comprises one of selecting a first number of the one or more optimized prompts that have scores above the average score or selecting a remaining number of the one or more optimized prompts after removing a second number of the one or more optimized prompts that have scores below the average score.

Assignments (1)

Continuity (1)

References Cited (40)