Look ahead strategy for trie-based beam search in generative retrieval
Systems and methods are provided for generating a keyword sequence from an input query. A first text sequence corresponding to an input query may be received and encoded into a source sequence representation using an encoder of a machine learning model. A keyword sentence may then be generated from the source sequence representation using a decoder of the machine learning model. The decoder may generate a modified generation score for a plurality of prediction tokens, wherein the modified generation score is based on the respective prediction token generation score and a maximum generation score for a suffix of each prediction token. The decoder may then select the prediction token of the plurality of prediction tokens based on the modified generation score, and add the selected prediction token to the previously decoded partial hypothesis provided by the decoder.
1. A method for generating a keyword sequence from an input query, the method comprising:
receiving a first text sequence corresponding to an input query;
encoding the first text sequence into a source sequence representation using an encoder of a machine learning model executed by a server computer system including an encoder and a decoder, the source sequence representation including a first set of tokens corresponding to the input query;
generating a keyword sequence from the source sequence representation using a decoder of the machine learning model, the keyword sequence including a second set of tokens relevant to the input query and not including the first set of tokens corresponding to the input query;
generating, by the decoder, a generation score for a first prediction token of a plurality of prediction tokens, wherein the plurality of prediction tokens are based on the source sequence representation and a previously decoded partial hypothesis provided by the decoder, where the previously decoded partial hypothesis includes a first subset of tokens of the second set of tokens stored in a trie;
generating, by the decoder, a modified generation score for the first prediction token, wherein the modified generation score for the first prediction token is based on the generation score and a maximum generation score for a suffix of the first prediction token, where the suffix includes a second subset of tokens of the second set of tokens stored within a path of the trie corresponding to the first prediction token;
selecting a second prediction token of the second subset of tokens based on the modified generation score;
adding the second prediction token to the previously decoded partial hypothesis generated by the decoder; and
selecting content to be provided to a computing device in response to the input query based on the second subset of tokens of the second set of tokens, wherein at least one token of the second subset of tokens of the second set of tokens matches at least one keyword associated with the content to be provided to the computing device.
2. The method of claim 1 , wherein the content to be provided to the computing device is an advertisement.
3. The method of claim 1 , wherein the generation score based on a second generation score for a token that is a suffix of the suffix of one or more prediction tokens.
4. The method of claim 1 , wherein the machine learning model is a sequence-to-sequence machine learning model trained on data corresponding to keyword sequence and output sequence pairs.
5. The method of claim 1 , further comprising:
receiving a keyword to be added to the trie; and
adding the keyword to the trie, wherein the keyword is a suffix to an existing keyword in the trie.
6. The method of claim 1 , wherein the plurality of prediction tokens and the suffix of each prediction token are constrained to the trie.
7. The method of claim 1 , further comprising selecting content to be provided to the computing device in response to the input query based on the previously decoded partial hypothesis and the second prediction token, wherein the previously decoded partial hypothesis and the second prediction token matches at least one keyword segment associated with the content to be provided to the computing device.
8. The method of claim 1 , further comprising generating a plurality of hypotheses based on the second prediction token.
9. The method of claim 1 , further comprising identifying the trie associated with the first text sequence.
10. The method of claim 9 , further comprising identifying suffixes for tokens stored in the trie associated with the first text sequence.
11. The method of claim 1 , further comprising generating a set of modified generation scores for a set of suffixes of the plurality of prediction tokens.
12. A computer-readable storage medium including instructions, which when executed by a processor, cause the processor to perform the method of claim 1 .
13. A system comprising a processor and memory including instructions which when executed by the processor cause the processor to perform the method of claim 1 .
14. A method for generating a keyword sequence from an input query, the method comprising:
encoding a text sequence into a source sequence representation using an encoder of a machine learning model executed by a server computer system, the machine learning model including an encoder and a decoder, the source sequence representation including a first set of tokens corresponding to an input query;
generating a keyword sequence from the source sequence representation using the decoder of the machine learning model, the keyword sequence including a second set of tokens relevant to the input query and not including the first set of tokens corresponding to the input query;
generating, by the decoder, a generation score for a first prediction token of a plurality of prediction tokens, wherein the plurality of prediction tokens are based on the source sequence representation and a previously decoded partial hypothesis provided by the decoder, where the previously decoded partial hypothesis includes a first subset of tokens of the second set of tokens stored in a trie;
generating, by the decoder, a modified generation score for the first prediction token;
selecting a second prediction token of second set of tokens based on the modified generation score;
adding the second prediction token to the previously decoded partial hypothesis provided by the decoder; and
selecting content to be provided to a computing device based on the second prediction token, wherein the second prediction token matches at least one keyword associated with the content to be provided to the computing device.
15. The method of claim 14 , wherein the modified generation score is based on the generation score and a maximum generation score for a suffix of the first prediction token.
16. The method of claim 15 , wherein the plurality of prediction tokens and the suffix are accessed from a Trie, and wherein the method further comprises:
receiving a keyword to be added to the Trie; and
adding the keyword to the Trie, wherein the keyword is a suffix to an existing keyword in the Trie.
17. A method for generating a keyword sequence from an input query, the method comprising:
receiving a text sequence corresponding to an input query;
encoding the text sequence into a source sequence representation using an encoder of a machine learning model including an encoder and a decoder executed by a computer system, the source sequence representation including a first set of tokens corresponding to the input query;
generating, by the decoder, a generation score for a first prediction token of a plurality of prediction tokens, wherein the plurality of prediction tokens are based on the source sequence representation and a previously decoded partial hypothesis provided by the decoder of the machine learning model, where the previously decoded partial hypothesis includes a first subset of tokens of a second set of tokens stored in a trie;
generating, by the decoder, a modified generation score for the first prediction token of the plurality of prediction tokens, wherein the modified generation score for the first prediction token of the plurality of prediction tokens is based on the respective prediction token generation score for the first prediction token and a maximum generation score for a suffix of the first prediction token of the plurality of prediction tokens, where the suffix includes a second subset of tokens of the second set of tokens stored within a path of the trie corresponding to the first prediction token;
selecting a second prediction token of the second subset of tokens based on the modified generation score;
adding the second prediction token to the previously decoded partial hypothesis provided by the decoder; and
selecting content to be provided to a computing device in response to the input query based on the second prediction token.
18. The method of claim 17 , wherein the modified generation score based on the generation score and a maximum generation score for a suffix of the first prediction token.
19. The method of claim 17 , wherein the suffix is obtained from a trie generated based on the input query.
20. The method of claim 17 , further comprising:
receiving a keyword to be added to the trie; and
adding the keyword to the trie, wherein the keyword is a second suffix to a second keyword stored in the Trie.