EVALUATING CONFIDENCE IN A CLASSIFICATION PERFORMED BY A GENERATIVE LANGUAGE MACHINE LEARNING MODEL
A large language model (LLM) may be used to classify an input into one of a plurality of categories. However, given the machine-learning operation of the LLM, the output of the LLM does not represent a definitive statement, but is based on probability computations of the machine learning model. Therefore, the classification performed by the LLM might not be correct. Classification into the wrong category by the LLM results in downstream technical problems. In some implementations, when an LLM generates a response that classifies an input, one or more probability values associated with a token that forms the basis of the response may be used to determine a confidence value. The confidence value is indicative of confidence in the classification performed by the LLM. An action may be taken based on the confidence value.
1 . A computer-implemented method comprising:
providing a prompt to a generative language model, the prompt instructing the generative language model to generate a response that classifies an input of the prompt into one category of a plurality of categories;
receiving the response from the generative language model, the response based on one or more tokens selected by the generative language model;
for a selected token that forms a basis of the response: receiving, from the generative language model, a probability value associated with the selected token and a probability value associated with an alternative token that was not selected by the generative language model to form the basis of the response;
determining a confidence value using the probability value associated with the selected token and the probability value associated with the alternative token; and
providing a reply to a query that is dependent upon the confidence value.
2 . The computer-implemented method of claim 1 , wherein the alternative token that was not selected corresponds to another category of the plurality of categories.
3 . The computer-implemented method of claim 1 , wherein the alternative token that was not selected has a next highest associated probability after the selected token.
4 . The computer-implemented method of claim 1 , wherein there are only two categories and the response classifies the input into one of the two categories.
5 . The computer-implemented method of claim 4 , wherein the response has one of two response values, wherein the two response values are a first response value that corresponds to the selected token and a second response value that corresponds to the alternative token.
6 . The computer-implemented method of claim 1 , wherein determining the confidence value comprises performing a comparison between the probability value associated with the selected token and the probability value associated with the alternative token, and wherein the confidence value is based on the comparison.
7 . The computer-implemented method of claim 1 , wherein the category into which the input is classified is a first category, wherein the alternative token corresponds to a different second category, and wherein the method further comprises:
for the selected token that forms the basis of the response, receiving, from the generative language model: (i) an indication of two or more alternative tokens, including the alternative token, that were not selected to form the basis of the response, and (ii) for each of the alternative tokens a respective probability value;
categorizing each of one or more of the alternative tokens into a respective category of the plurality of categories;
determining a first category probability value that is either: (i) obtained by combining the probability value associated with the selected token with at least one other probability value associated with at least one alternative token categorized into the first category, or (ii) equal to the probability value associated with the selected token responsive to none of the alternative tokens being categorized into the first category;
determining a second category probability value that is either: (i) obtained by combining the probability value associated with the alternative token with at least one other probability value associated with at least one other alternative token categorized into the second category, or (ii) equal to the probability value associated with the alternative token responsive to none of the other alternative tokens being categorized into the second category; and
performing a comparison between the first category probability value and the second category probability value, wherein the confidence value is based on the comparison.
8 . The computer-implemented method of claim 1 , wherein providing the reply comprises providing the response along with information generated based on the confidence value.
9 . The computer-implemented method of claim 1 , wherein the prompt is a first prompt, and wherein providing the reply comprises:
responsive to the confidence value not satisfying a threshold, providing a second prompt to the generative language model; and
providing a reply based on output from the generative language model responsive to the second prompt.
10 . The computer-implemented method of claim 9 , wherein the second prompt indicates, to the generative language model, that the input is classified into another category different from the category into which the input was classified by the response.
11 . The computer-implemented method of claim 1 , wherein providing the reply comprises:
responsive to the confidence value not satisfying a threshold, modifying the response to classify the input into another category different from the category into which the input was originally classified by the response;
continuing generation of output from the generative language model with the response modified; and
providing the reply based on the output from the generative language model with the response modified.
12 . The computer-implemented method of claim 1 , wherein:
the input of the prompt is the query;
the prompt instructs the generative language model to classify the query into one of two categories, wherein a first category of the two categories corresponds to the query being unambiguous, and a second category of the two categories corresponds to the query being ambiguous;
the response classifies the query into the first category corresponding to the query being unambiguous; and
the selected token forms the basis of the response classifying the query as unambiguous, and the alternative token corresponds to the query being classified as ambiguous.
13 . The computer-implemented method of claim 1 , wherein:
the input of the prompt is the query and information;
the prompt instructs the generative language model to classify the information into one of two categories, wherein a first category of the two categories corresponds to the information being sufficient to reply to the query, and a second category of the two categories corresponds to the information being insufficient to reply to the query;
the response classifies the information into the first category corresponding to the information being sufficient to reply to the query; and
the selected token forms the basis of the response classifying the information as sufficient to reply to the query, and the alternative token corresponds to the information being insufficient to reply to the query.
14 . A system comprising:
at least one processor; and
a memory storing processor-executable instructions that, when executed by the at least one processor, cause the system to:
provide a prompt to a generative language model, the prompt instructing the generative language model to generate a response that classifies an input of the prompt into one category of a plurality of categories;
receive the response from the generative language model, the response based on one or more tokens selected by the generative language model;
for a selected token that forms a basis of the response: receive, from the generative language model, a probability value associated with the selected token and a probability value associated with an alternative token that was not selected by the generative language model to form the basis of the response;
determine a confidence value using the probability value associated with the selected token and the probability value associated with the alternative token; and
provide a reply to a query that is dependent upon the confidence value.
15 . The system of claim 14 , wherein the alternative token that was not selected corresponds to another category of the plurality of categories.
16 . The system of claim 14 , wherein the alternative token that was not selected has a next highest associated probability after the selected token.
17 . The system of claim 14 , wherein there are only two categories and the response classifies the input into one of the two categories, wherein the response has one of two response values, wherein the two response values are a first response value that corresponds to the selected token and a second response value that corresponds to the alternative token.
18 . The system of claim 14 , wherein determining the confidence value comprises performing a comparison between the probability value associated with the selected token and the probability value associated with the alternative token, and wherein the confidence value is based on the comparison.
19 . The system of claim 14 , wherein the prompt is a first prompt, and wherein providing the reply comprises:
responsive to the confidence value not satisfying a threshold, providing a second prompt to the generative language model; and
providing a reply based on output from the generative language model responsive to the second prompt.
20 . A non-transitory computer readable medium having stored thereon computer-executable instructions that, when executed by a computer, cause the computer to perform operations comprising:
providing a prompt to a generative language model, the prompt instructing the generative language model to generate a response that classifies an input of the prompt into one category of a plurality of categories;
receiving the response from the generative language model, the response based on one or more tokens selected by the generative language model;
for a selected token that forms a basis of the response: receiving, from the generative language model, a probability value associated with the selected token and a probability value associated with an alternative token that was not selected by the generative language model to form the basis of the response;
determining a confidence value using the probability value associated with the selected token and the probability value associated with the alternative token; and
providing a reply to a query that is dependent upon the confidence value.