Query clarification based on confidence in a classification performed by a generative language machine learning model
A large language model (LLM) may be used to classify an input into one of a plurality of categories. However, given the machine-learning operation of the LLM, the output of the LLM does not represent a definitive statement, but is based on probability computations of the machine learning model. Therefore, the classification performed by the LLM might not be correct. Classification into the wrong category by the LLM results in downstream technical problems. In some implementations, when an LLM generates a response that classifies an input, one or more probability values associated with a token that forms the basis of the response may be used to determine a confidence value. The confidence value is indicative of confidence in the classification performed by the LLM. An action may be taken based on the confidence value.
1 . A computer-implemented method comprising:
providing a prompt to a generative language model, the prompt including a query and the prompt instructing the generative language model to generate a response that classifies the query into one of a plurality of categories, wherein the plurality of categories includes a category corresponding to the query being unambiguous;
receiving the response from the generative language model, the response classifying the query into the category corresponding to the query being unambiguous, and the response being based on one or more tokens selected by the generative language model;
for a token that forms a basis of language indicating the query is unambiguous: receiving, from the generative language model, a probability value associated with the token that forms the basis of language indicating the query is unambiguous;
determining a confidence value using at least the probability value associated with the token that forms the basis of language indicating the query is unambiguous; and
responsive to the confidence value not satisfying a threshold, providing a reply to the query that requests clarification relating to the query.
2 . The computer-implemented method of claim 1 , wherein the reply to the query is based on output from the generative language model.
3 . The computer-implemented method of claim 2 , wherein the prompt is a first prompt, and wherein providing the reply comprises:
providing a second prompt to the generative language model; and
providing the reply to the query based on output from the generative language model responsive to the second prompt.
4 . The computer-implemented method of claim 3 , wherein the second prompt indicates, to the generative language model, that the query is classified into another category different from the category into which the query was classified by the response.
5 . The computer-implemented method of claim 3 , wherein the second prompt includes information retrieved based on the query, and the second prompt instructs the generative language model to use the information to generate the reply that requests clarification.
6 . The computer-implemented method of claim 2 , wherein:
responsive to the confidence value not satisfying the threshold, modifying the response to classify the query into another category different from the category into which the query was originally classified by the response;
continuing generation of output from the generative language model with the response modified; and
providing the reply based on the output from the generative language model with the response modified.
7 . The computer-implemented method of claim 1 , wherein the token that forms the basis of language indicating the query is unambiguous is a selected token, wherein the method further comprises receiving, from the generative language model, a probability value associated with an alternative token that was not selected by the generative language model to form the basis of language indicating the query is unambiguous, and wherein the confidence value is determined using the probability value associated with the selected token and the probability value associated with the alternative token.
8 . The computer-implemented method of claim 7 , wherein the alternative token that was not selected corresponds to another category of the plurality of categories.
9 . The computer-implemented method of claim 7 , wherein the alternative token that was not selected has a next highest associated probability after the selected token.
10 . The computer-implemented method of claim 7 , wherein there are only two categories, the two categories being a first category corresponding to the query being unambiguous and a second category corresponding to the query being ambiguous, wherein the response can be one of two response values, wherein a first response value of the two response values corresponds to the query being unambiguous and the second response value of the two response values corresponds to the query being ambiguous, wherein the response is the first response value corresponding to the query being unambiguous, wherein the selected token corresponds to the first response value, and wherein the alternative token corresponds to the second response value.
11 . The computer-implemented method of claim 7 , wherein determining the confidence value comprises performing a comparison between the probability value associated with the selected token and the probability value associated with the alternative token, and wherein the confidence value is based on the comparison.
12 . A system comprising:
at least one processor; and
a memory storing processor-executable instructions that, when executed by the at least one processor, cause the system to:
provide a prompt to a generative language model, the prompt including a query and the prompt instructing the generative language model to generate a response that classifies the query into one of a plurality of categories, wherein the plurality of categories includes a category corresponding to the query being unambiguous;
receive the response from the generative language model, the response classifying the query into the category corresponding to the query being unambiguous, and the response being based on one or more tokens selected by the generative language model;
for a token that forms a basis of language indicating the query is unambiguous: receive, from the generative language model, a probability value associated with the token that forms the basis of language indicating the query is unambiguous;
determine a confidence value using at least the probability value associated with the token that forms the basis of language indicating the query is unambiguous; and
responsive to the confidence value not satisfying a threshold, provide a reply to the query that requests clarification relating to the query.
13 . The system of claim 12 , wherein the reply to the query is based on output from the generative language model.
14 . The system of claim 13 , wherein the prompt is a first prompt, and wherein providing the reply comprises:
providing a second prompt to the generative language model; and
providing the reply to the query based on output from the generative language model responsive to the second prompt.
15 . The system of claim 14 , wherein the second prompt indicates, to the generative language model, that the query is classified into another category different from the category into which the query was classified by the response.
16 . The system of claim 14 , wherein the second prompt includes information retrieved based on the query, and the second prompt instructs the generative language model to use the information to generate the reply that requests clarification.
17 . The system of claim 12 , wherein the token that forms the basis of language indicating the query is unambiguous is a selected token, wherein the instructions, when executed, further cause the system to receive, from the generative language model, a probability value associated with an alternative token that was not selected by the generative language model to form the basis of language indicating the query is unambiguous, and wherein the confidence value is determined using the probability value associated with the selected token and the probability value associated with the alternative token.
18 . The system of claim 17 , wherein there are only two categories, the two categories being a first category corresponding to the query being unambiguous and a second category corresponding to the query being ambiguous, wherein the response can be one of two response values, wherein a first response value of the two response values corresponds to the query being unambiguous and the second response value of the two response values corresponds to the query being ambiguous, wherein the response is the first response value corresponding to the query being unambiguous, wherein the selected token corresponds to the first response value, and wherein the alternative token corresponds to the second response value.
19 . The system of claim 17 , wherein determining the confidence value comprises performing a comparison between the probability value associated with the selected token and the probability value associated with the alternative token, and wherein the confidence value is based on the comparison.
20 . A non-transitory computer readable medium having stored thereon computer-executable instructions that, when executed by a computer, cause the computer to perform operations comprising:
providing a prompt to a generative language model, the prompt including a query and the prompt instructing the generative language model to generate a response that classifies the query into one of a plurality of categories, wherein the plurality of categories includes a category corresponding to the query being unambiguous;
receiving the response from the generative language model, the response classifying the query into the category corresponding to the query being unambiguous, and the response being based on one or more tokens selected by the generative language model;
for a token that forms a basis of language indicating the query is unambiguous: receiving, from the generative language model, a probability value associated with the token that forms the basis of language indicating the query is unambiguous;
determining a confidence value using at least the probability value associated with the token that forms the basis of language indicating the query is unambiguous; and
responsive to the confidence value not satisfying a threshold, providing a reply to the query that requests clarification relating to the query.