Generating author vectors
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating author vectors. One of the methods includes obtaining a set of sequences of words, the set of sequences of words comprising a plurality of first sequences of words and, for each first sequence of words, a respective second sequence of words that follows the first sequence of words, wherein each first sequence of words and each second sequence of words has been classified as being authored by a first author; and training a neural network system on the first sequences and the second sequences to determine an author vector for the first author, wherein the author vector characterizes the first author.
1. A method performed by a system of one or more computers, comprising:
obtaining an input sequence of words;
obtaining an author vector that characterizes at least one attribute of an author of the input sequence of words;
processing, with a first model, the input sequence of words to generate an alternative representation of the input sequence of words; and
processing, with a second model, the author vector and the alternative representation of the input sequence of words to generate an output sequence of words, wherein the output sequence of words reflects the at least one attribute of the author of the input sequence of words.
2. The method of claim 1 , wherein the author vector is generated based on processing a collection of word sequences authored by the author.
3. The method of claim 1 , wherein the at least one attribute of the author comprises a communication style of the author, a personality type of the author, or a content-item selection profile of the author.
4. The method of claim 1 , wherein the input sequence of words comprises a natural language request from the author, and the output sequence of words comprises a natural language response to the request conditioned for the author.
5. The method of claim 1 , wherein the input sequence of words is a first portion of an extended sequence of words, and the output sequence of words is a predicted second portion of the extended sequence of words that follows the first portion.
6. The method of claim 1 , further comprising processing the author vector and the alternative representation of the input sequence of words to generate a combined data set,
wherein processing, with the second model, the author vector and the alternative representation of the input sequence of words to generate the output sequence of words comprises processing the combined data set with the second model to generate the output sequence of words.
7. The method of claim 1 , wherein the input sequence of words has a different length than the output sequence of words.
8. The method of claim 1 , wherein the first model is configured (i) to process input sequences having variable lengths, and (ii) to generate alternative representations of the input sequences having a fixed length.
9. The method of claim 1 , wherein the first model comprises a first neural network, and the second model comprises a second neural network.
10. The method of claim 9 , wherein the first and second neural networks comprise respective recurrent neural networks.
11. The method of claim 9 , wherein the first and second neural networks comprise respective long short-term memory (LSTM) neural networks.
12. One or more non-transitory computer-readable media having instructions stored thereon that, when executed by data processing apparatus, cause the data processing apparatus to perform operations comprising:
obtaining an input sequence of words;
obtaining an author vector that characterizes at least one attribute of an author of the input sequence of words;
processing, with a first model, the input sequence of words to generate an alternative representation of the input sequence of words; and
processing, with a second model, the author vector and the alternative representation of the input sequence of words to generate an output sequence of words, wherein the output sequence of words reflects the at least one attribute of the author of the input sequence of words.
13. The one or more non-transitory computer-readable media of claim 12 , wherein the author vector is generated based on processing a collection of word sequences authored by the author.
14. The one or more non-transitory computer-readable media of claim 12 , wherein the at least one attribute of the author comprises a communication style of the author, a personality type of the author, or a content-item selection profile of the author.
15. The one or more non-transitory computer-readable media of claim 12 , wherein the input sequence of words comprises a natural language request from the author, and the output sequence of words comprises a natural language response to the request conditioned for the author.
16. The one or more non-transitory computer-readable media of claim 12 , wherein the input sequence of words is a first portion of an extended sequence of words, and the output sequence of words is a predicted second portion of the extended sequence of words that follows the first portion.
17. The one or more non-transitory computer-readable media of claim 12 , wherein the operations further comprise processing the author vector and the alternative representation of the input sequence of words to generate a combined data set,
wherein processing, with the second model, the author vector and the alternative representation of the input sequence of words to generate the output sequence of words comprises processing the combined data set with the second model to generate the output sequence of words.
18. The one or more non-transitory computer-readable media of claim 12 , wherein the input sequence of words has a different length than the output sequence of words.
19. The one or more non-transitory computer-readable media of claim 12 , wherein the first model is configured (i) to process input sequences having variable lengths, and (ii) to generate alternative representations of the input sequences having a fixed length.
20. A system, comprising:
a data processing apparatus; and
one or more computer-readable media having instructions stored thereon that, when executed by the data processing apparatus, cause the data processing apparatus to perform operations comprising:
obtaining an input sequence of words;
obtaining an author vector that characterizes at least one attribute of an author of the input sequence of words;
processing, with a first model, the input sequence of words to generate an alternative representation of the input sequence of words; and
processing, with a second model, the author vector and the alternative representation of the input sequence of words to generate an output sequence of words, wherein the output sequence of words reflects the at least one attribute of the author of the input sequence of words.