Generating large language model outputs from stored content items
The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating content-item-specific large language model responses from content items by segmenting a content item and selecting relevant sections of the content item to provide to a large language model to generate a corresponding output. In particular, in one or more embodiments, the disclosed systems can generate a text representation that includes a plurality of text segments each comprising a number of tokens of the text representation. Further, the systems can extract, from the plurality of text segments, segment-specific text embeddings that correspond to respective portions of the text representation of the content item. Additionally, the systems can determine a segment-specific text embedding corresponding to a model output request. Moreover, the systems can generate a model output by passing a text segment corresponding to the segment-specific text embedding to a large language model together with the model output request.
1 . A computer-implemented method comprising:
generating, from a content item utilizing a content conversion model, a text representation that includes a plurality of text segments each comprising a number of tokens of the text representation;
extracting, from the plurality of text segments, segment-specific text embeddings that correspond to respective portions of the text representation of the content item;
in response to receiving a model output request from a client device, determining a segment-specific text embedding corresponding to the model output request;
generating a model output by passing to a large language model one or more text segments corresponding to the segment-specific text embedding together with the model output request and additional text segments comprising a threshold number of tokens extracted from an initial portion of the text representation and a threshold number of tokens following a topic change in the text representation for providing context to guide the large language model;
caching the model output of the large language model for later reuse in response to generating the model output; and
generating, utilizing the large language model to process the model output cached for later reuse, an additional model output for an additional model output request from the client device.
2 . The computer-implemented method of claim 1 , wherein generating the text representation for the content item comprises utilizing the content conversion model to generate a set of text characters that describe the content item according to a compatibility graph.
3 . The computer-implemented method of claim 1 , wherein extracting the segment-specific text embeddings comprises generating a latent vector representation for a segment-specific text embedding from among the segment-specific text embeddings.
4 . The computer-implemented method of claim 1 , further comprising receiving the model output request from the client device by receiving a summary generation request or a response generation request from the client device.
5 . The computer-implemented method of claim 1 , wherein determining the segment-specific text embedding corresponding to the model output request comprises:
extracting an output request embedding from the model output request;
comparing the output request embedding with the segment-specific text embeddings; and
selecting the segment-specific text embedding corresponding to the model output request based on comparing the output request embedding with the segment-specific text embeddings.
6 . The computer-implemented method of claim 1 , wherein generating the model output comprises:
determining source content items that include digital content corresponding to text segments provided to the large language model to generate the model output; and
generating a ranked list of the source content items for display on the client device.
7 . The computer-implemented method of claim 1 , wherein generating the model output comprises:
determining a content item location within the content item corresponding to the one or more text segments provided to the large language model to generate the model output; and
providing an indication of the content item location for display on the client device.
8 . A system comprising:
at least one processor; and
a non-transitory computer readable medium storing instructions that, when executed by the at least one processor, cause the system to:
generate, from a content item utilizing a content conversion model, a text representation that includes a plurality of text segments each comprising a number of tokens of the text representation;
extract, from the plurality of text segments, segment-specific text embeddings corresponding to respective portions of the text representation of the content item;
determine a segment-specific text embedding corresponding to a model output request;
generate a model output by passing to a large language model a text segment corresponding to the segment-specific text embedding together with the model output request and additional text segments comprising a threshold number of tokens extracted from an initial portion of the text representation and a threshold number of tokens following a topic change in the text representation for providing context to guide the large language model;
caching the model output of the large language model for later reuse in response to generating the model output; and
generating, utilizing the large language model to process the model output cached for later reuse, an additional model output for an additional model output request from a client device.
9 . The system of claim 8 , wherein the instructions cause the system to generate the text representation based on a compatibility graph that defines relationships between types of content items, the compatibility graph comprising nodes and edges that indicate conversion paths for converting a first content item of a first type to a second content item of a second type different from the first type.
10 . The system of claim 8 , wherein the instructions cause the system to generate the model output by passing, to the large language model, the text segment corresponding to the segment-specific text embedding along with a threshold number of additional text segments corresponding to the model output request.
11 . The system of claim 8 , further storing instructions that, when executed by the at least one processor, cause the system to generate a one or more suggested model output requests, for generating a range of possible model outputs, specific to the content item to provide for display with the model output by:
determining one or more text segments that are semantically dissimilar to the model output request; and
generating, utilizing the large language model, the one or more suggested model output requests based on the one or more text segments that are semantically dissimilar to the model output request.
12 . The system of claim 8 , further storing instructions that, when executed by the at least one processor, cause the system to:
receive the model output request by receiving a summary generation request; and
generate the model output by generating a content item summary in response to receiving the summary generation request by passing the text segment corresponding to the segment-specific text embedding to the large language model together with the model output request.
13 . The system of claim 8 , further storing instructions that, when executed by the at least one processor, cause the system to:
extract an output request embedding from the model output request; and
compare the segment-specific text embeddings with the output request embedding to determine relevant text embeddings of the text representation.
14 . The system of claim 13 , further storing instructions that, when executed by the at least one processor, cause the system to:
pass at least one of the relevant text embeddings to the large language model; and
omit one or more irrelevant text embeddings from providing to the large language model.
15 . The system of claim 8 , further storing instructions that, when executed by the at least one processor, cause the system to:
determine a location within the content item corresponding to the text segment within the text representation of the content item; and
provide an indication of the location for display on a client device together with the model output.
16 . A non-transitory computer readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to:
generate, from a content item, a text representation that includes a plurality of text segments each comprising a number of tokens of the text representation;
extract, from the plurality of text segments, segment-specific text embeddings that correspond to respective portions of the text representation of the content item;
in response to receiving a model output request, determine a segment-specific text embedding corresponding to the model output request;
generate a model output by passing to a large language model a text segment corresponding to the segment-specific text embedding together with the model output request and additional text segments comprising a threshold number of tokens extracted from an initial portion of the text representation and a threshold number of tokens following a topic change in the text representation for providing context to guide the large language model;
caching the model output of the large language model for later reuse in response to generating the model output; and
generating, utilizing the large language model to process the model output cached for later reuse, an additional model output for an additional model output request from a client device.
17 . The non-transitory computer readable medium of claim 16 , wherein the instructions cause the at least one processor to generate the model output by passing, to the large language model, the text segment corresponding to the segment-specific text embedding along with a first text segment beginning at a start point of the text representation and a second text segment beginning immediately after the first text segment.
18 . The non-transitory computer readable medium of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the at least one processor to generate a suggested model output request specific to the content item to provide for display with the model output by utilizing the large language model to determine the suggested model output request based on the model output and the text segment provided to the large language model.
19 . The non-transitory computer readable medium of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the at least one processor to:
receive the model output request by receiving a response generation request; and
generate the model output by generating a response in response to the response generation request by passing the text segment corresponding to the segment-specific text embedding to the large language model together with the model output request.
20 . The non-transitory computer readable medium of claim 16 , further storing instructions that when executed by the at least one processor, cause the at least one processor to extract the plurality of text segments from the text representation by dividing the text representation into segments having a set number of tokens.