Human guided text attention mechanism
View Patent ↗A system includes a processor that executes computer executable components stored in a memory. The computer executable components include a segmentation component that segments a prompt into select groups as a function of saliency estimates. The computer executable components further include a ranking component that assigns a rank to at least one of the groups, and a search component that searches for a likely candidate token, based at least in part on the assigned rank.
1 . A system, comprising:
a processor that executes computer executable components stored in memory, wherein the computer executable components comprise:
a segmentation component that receives an input prompt and segments the input prompt into select groups based on saliency estimates determined from human annotated groupings of the input prompt;
a ranking component that assigns a rank to at least one of the select groups based on the saliency estimates and human annotated groupings; and
a search component configured to:
generate a plurality of candidate tokens by processing the input prompt with a large language model;
determine a saliency estimate for each candidate token by analyzing attention contributions of the candidate token to tokens of the input prompt based on inspection of hidden layers of the large language model;
use the saliency estimate together with the human annotated groupings to calculate grouped saliency values associated with the select groups;
determine a distance between the grouped saliency values and a desired saliency calculation associated with the rank assigned to the at least one of the select groups;
search for a likely candidate token, based at least in part on the assigned rank, wherein the search component performs a first search that is rank-agnostic and a second search that is conditioned on the assigned rank; and
select a most likely candidate token based at least in part on a comparison of results of the first and second searches during decoding of a large language-model output, thereby improving fidelity and relevance of generated text in response to user queries.
2 . The system of claim 1 , further comprising a selection component that determines a distance between the assigned rank and grouped saliency values associated with the select groups, and selects the most likely candidate token, based at least in part on a comparison of the grouped saliency values to a desired saliency calculation associated with the assigned rank.
3 . The system of claim 2 , wherein:
the search component performs a first search that is rank-agnostic based upon the received prompt, and performs a second search that is conditioned on the assigned rank; and
the selection component selects the most likely candidate token based at least in part on a comparison of results of the first and second searches during decoding of a large language-model output.
4 . The system of claim 1 , further comprising an artificial intelligence component that trains an artificial intelligence model to calculate grouped saliency values from saliency estimates derived from attention contributions of candidate tokens.
5 . The system of claim 4 , wherein the artificial intelligence component ranks the saliency values based on a distance to a desired saliency decomposition associated with the human annotated groupings.
6 . The system of claim 5 wherein the artificial intelligence component optimizes similarities between the grouped saliency values and the desired saliency calculation associated with the assigned rank.
7 . The system of claim 1 , wherein the rank assigned by the ranking component determines a desired saliency decomposition used to evaluate grouped saliency values derived from attention contributions of candidate tokens.
8 . The system of claim 1 , wherein the rank assigned by the ranking component determines a priority of the segmented groups for guiding selection of the most likely candidate token during decoding of the large language-model output.
9 . The system of claim 1 , wherein the ranking component adjusts the at least one rank and the search component performs a new search for a likely candidate token by performing updated rank-conditioned searching based on the adjusted rank.
10 . A computer-implemented method that utilizes a processor that executes computer executable components stored in memory to perform the following acts:
segmenting a prompt into select groups based on saliency estimates determined from human annotated groupings of an input prompt;
assigning a rank to at least one of the groups based on the saliency estimates and the human annotated groups; and
searching for a likely candidate token, based at least in part on the assigned rank, wherein the searching includes:
generating a plurality of candidate tokens by processing the input prompt with a large language model;
determining a saliency estimate for each candidate token by analyzing attention contributions of the candidate token to tokens of the input prompt based on inspection of hidden layers of the large language model;
using the saliency estimate together with the human annotated groupings to calculate grouped saliency values;
determining a distance between the grouped saliency values and a desired saliency calculation associated with the assigned rank;
and wherein the searching further includes performing a first search that is rank-agnostic and a second search that is conditioned on the assigned rank and selecting the most likely candidate token based at least in part on a comparison of results of the first and second searches during decoding of a large language-model output, thereby improving fidelity and relevance of generated text in response to user queries.
11 . The method of claim 10 , further comprising determining a distance between grouped saliency values and a desired saliency calculation associated with the assigned rank, and selecting the most likely candidate token based at least in part on minimizing the determined distance.
12 . The method of claim 11 , further comprising:
performing a first search that is rank-agnostic based upon the received prompt, and performing a second search that is conditioned on the assigned rank; and
selecting the most likely candidate token based at least in part on a comparison of results of the first and second searches during decoding of the large language-model output.
13 . The method of claim 10 , further comprising training a large language model to calculate grouped saliency values for candidate tokens based on saliency estimates derived from attention contributions.
14 . The method of claim 13 , wherein the ranking of the saliency values is based on a distance to a desired saliency decomposition associated with human annotated groupings of the input prompt.
15 . The method of claim 14 , wherein further comprising using the distance to the desired saliency decomposition to determine a saliency cost for evaluating candidate tokens during decoding.
16 . The method of claim 10 , further comprising determining a desired saliency decomposition, based at least in part on the assigned rank and the human annotated groupings of the input prompt.
17 . The method of claim 10 , wherein assigning the rank determines a priority of the segmented groups for guiding selection of candidate tokens during decoding of the large language-model output.
18 . The method of claim 10 , further comprising adjusting the at least one rank and performing a new search for a likely candidate token by performing updated rank-conditioned searching based on the adjusted rank.
19 . A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
segment a prompt into select groups based on saliency estimates determined from human annotated groupings of an input prompt;
assign a rank to at least one of the groups based on the saliency estimates and the human annotated groups; and
search for a likely candidate token, based at least in part on the assigned rank, wherein the search includes:
generating a plurality of candidate tokens by processing the input prompt with a large language model;
determining a saliency estimate for each candidate token by analyzing attention contributions of the candidate token to tokens of the input prompt based on inspection of hidden layers of the large language model;
using the saliency estimate together with the human annotated groupings to calculate grouped saliency values;
determining a distance between the grouped saliency values and a desired saliency calculation associated with the assigned rank;
and wherein the search further includes performing a first search that is rank-agnostic and a second search that is conditioned on the assigned rank and selecting the most likely candidate token based at least in part on a comparison of results of the first and second searches during decoding of a large language-model output, thereby improving fidelity and relevance of generated text in response to user queries.
20 . The computer program product of claim 19 , further causing the processor to:
select a most likely candidate token, based at least in part on the assigned rank;
perform a first search that is rank-agnostic;
perform a second search that is conditioned on the assigned rank; and
select the most likely candidate token based at least in part on a comparison of the results of the first and second searches during decoding of the large language-model output.