Extractive-abstractive large language model summarization with farthest point sampling
In some systems, a set of sentences of a relatively large document may be vectorized into a set of vectors via an embedding model for summarization. Further, a subset of vectors of the set of vectors may be selected via a farthest point sampling (FPS) procedure based on a vector-space distance between respective vectors of the subset of vectors. Moreover, the subset of vectors that are associated with a subset of sentences may be ordered based on the order of the subset of sentences within the set of sentences of the document. Further, to generate a summary of the document, a query may be transmitted to a large language model (LLM) that includes a summarization prompt and the subset of sentences that correspond with the selected subset of vectors. A summary of the document may then be received from the LLM based on transmitting the query.
1. A method for data processing, comprising:
vectorizing, via an embedding model, a set of sentences of a document into a set of vectors;
selecting, via a farthest point sampling procedure, a subset of vectors of the set of vectors based at least in part on a vector-space distance between respective vectors of the subset of vectors and on a parameter indicative of a level of attention to a global embedding value that corresponds to the document, wherein the farthest point sampling procedure is executed to select the subset of vectors from the set of vectors concurrently with an execution of the embedding model to vectorize the set of sentences into the set of vectors;
ordering the subset of vectors according to a corresponding sentence order within the document;
transmitting, to a large language model, a query comprising a summarization prompt and an input comprising a subset of sentences of the set of sentences that are associated with the subset of vectors; and
receiving, from the large language model, a summary of the document based at least in part on transmitting the query.
2. The method of claim 1 , further comprising:
segmenting a respective sentence from the set of sentences of the document into two or more vectors of the set of vectors based at least in part on a quantity of tokens associated with the respective sentence satisfying a vector token threshold.
3. The method of claim 1 , further comprising:
executing, via one or more central processing units, the embedding model to vectorize the set of sentences into the set of vectors and the farthest point sampling procedure to select the subset of vectors via a parallel processing procedure, wherein the farthest point sampling procedure is executed to select the subset of vectors concurrently with an execution of the embedding model to vectorize the set of sentences in accordance with the parallel processing procedure.
4. The method of claim 1 , wherein selecting the subset of vectors via the farthest point sampling procedure comprises:
selecting a first vector for the subset of vectors via a randomization procedure.
5. The method of claim 1 , wherein selecting the subset of vectors via the farthest point sampling procedure comprises:
selecting a first vector for the subset of vectors based at least in part on the first vector being associated with the global embedding value that corresponds to the document.
6. The method of claim 5 , wherein the subset of vectors are selected based at least in part on both the vector-space distance between the respective vectors of the subset of vectors and the global embedding value.
7. The method of claim 1 , wherein selecting the subset of vectors comprises:
selecting, via the farthest point sampling procedure, the subset of vectors of the set of vectors such that a quantity of vectors within the subset of vectors satisfies a vector quantity threshold.
8. The method of claim 1 , wherein vectorizing the set of sentences into the set of vectors comprises:
extracting, via the embedding model, one or more embeddings for the set of sentences, wherein the set of vectors represents the one or more embeddings of the set of sentences.
9. The method of claim 1 , wherein the subset of vectors of the set of vectors are associated with one or more sentences of the set of sentences that represent the document as a whole for transmitting the query to the large language model.
10. The method of claim 1 , wherein a quantity of tokens associated with the summarization prompt is based at least in part on a prompt token threshold.
11. An apparatus for data processing, comprising:
one or more memories storing processor-executable code; and
one or more processors coupled with the one or more memories and individually or collectively operable to execute the code to cause the apparatus to:
vectorize, via an embedding model, a set of sentences of a document into a set of vectors;
select, via a farthest point sampling procedure, a subset of vectors of the set of vectors based at least in part on a vector-space distance between respective vectors of the subset of vectors and on a parameter indicative of a level of attention to a global embedding value that corresponds to the document, wherein the farthest point sampling procedure is executed to select the subset of vectors from the set of vectors concurrently with an execution of the embedding model to vectorize the set of sentences into the set of vectors;
order the subset of vectors according to a corresponding sentence order within the document;
transmit, to a large language model, a query comprising a summarization prompt and an input comprising a subset of sentences of the set of sentences that are associated with the subset of vectors; and
receive, from the large language model, a summary of the document based at least in part on transmitting the query.
12. The apparatus of claim 11 , wherein the one or more processors are individually or collectively further operable to execute the code to cause the apparatus to:
segment a respective sentence from the set of sentences of the document into two or more vectors of the set of vectors based at least in part on a quantity of tokens associated with the respective sentence satisfying a vector token threshold.
13. The apparatus of claim 11 , wherein, to select the subset of vectors via the farthest point sampling procedure, the one or more processors are individually or collectively operable to execute the code to cause the apparatus to:
select a first vector for the subset of vectors based at least in part on the first vector being associated with the global embedding value that corresponds to the document.
14. The apparatus of claim 11 , wherein, to select the subset of vectors, the one or more processors are individually or collectively operable to execute the code to cause the apparatus to:
select, via the farthest point sampling procedure, the subset of vectors of the set of vectors such that a quantity of vectors within the subset of vectors satisfies a vector quantity threshold.
15. The apparatus of claim 11 , wherein, to vectorize the set of sentences into the set of vectors, the one or more processors are individually or collectively operable to execute the code to cause the apparatus to:
extract, via the embedding model, one or more embeddings for the set of sentences, wherein the set of vectors represents the one or more embeddings of the set of sentences.
16. A non-transitory computer-readable medium storing code for data processing, the code comprising instructions executable by one or more processors to:
vectorize, via an embedding model, a set of sentences of a document into a set of vectors;
select, via a farthest point sampling procedure, a subset of vectors of the set of vectors based at least in part on a vector-space distance between respective vectors of the subset of vectors and on a parameter indicative of a level of attention to a global embedding value that corresponds to the document, wherein the farthest point sampling procedure is executed to select the subset of vectors from the set of vectors concurrently with an execution of the embedding model to vectorize the set of sentences into the set of vectors;
order the subset of vectors according to a corresponding sentence order within the document;
transmit, to a large language model, a query comprising a summarization prompt and an input comprising a subset of sentences of the set of sentences that are associated with the subset of vectors; and
receive, from the large language model, a summary of the document based at least in part on transmitting the query.
17. The non-transitory computer-readable medium of claim 16 , wherein the instructions are further executable by the one or more processors to:
segment a respective sentence from the set of sentences of the document into two or more vectors of the set of vectors based at least in part on a quantity of tokens associated with the respective sentence satisfying a vector token threshold.
18. The non-transitory computer-readable medium of claim 16 , wherein the instructions to select the subset of vectors via the farthest point sampling procedure are executable by the one or more processors to:
select a first vector for the subset of vectors based at least in part on the first vector being associated with the global embedding value that corresponds to the document.
19. The non-transitory computer-readable medium of claim 16 , wherein the instructions to select the subset of vectors are executable by the one or more processors to:
select, via the farthest point sampling procedure, the subset of vectors of the set of vectors such that a quantity of vectors within the subset of vectors satisfies a vector quantity threshold.
20. The non-transitory computer-readable medium of claim 16 , wherein the instructions to vectorize the set of sentences into the set of vectors are executable by the one or more processors to:
extract, via the embedding model, one or more embeddings for the set of sentences, wherein the set of vectors represents the one or more embeddings of the set of sentences.