Author personality trait recognition from short texts with a deep compositional learning approach
A method for trait prediction includes receiving a text sequence and generating character embeddings for characters of the text sequence. Word representations for words of the text sequence are generated with a trained character sequence model, based on the character embeddings. A sequence representation is generated for the text sequence with a trained word sequence model, based on the word representations. At least one trait prediction is generated with a trained trait model, based on the sequence representation.
1. A method for trait prediction comprising:
receiving a text sequence comprising words, each of the words including at least one character, some of the words comprising more than one character;
generating a character embeddings for each of the characters of the text sequence, each character embedding being a multidimensional representation of a respective one of the characters;
generating word representations for words of the text sequence with a trained character sequence model, based on the character embeddings, each word embedding being a multidimensional representation of a respective one of the words of the text sequence, the character sequence model comprising a first recurrent neural network that receives the character embeddings for a word of the text sequence and outputs the word representation for the word;
generating a sequence representation for the text sequence with a trained word sequence model, based on the word representations;
generating at least one trait prediction with a trained trait model, based on the sequence representation; and
outputting the trait prediction or information based on the trait prediction,
wherein at least one of the generating character embeddings, generating word representations, generating the sequence representation, and generating at least one trait prediction is performed with a processor.
2. The method of claim 1 , wherein each character embedding, word representation, and sequence representation is a multidimensional representation of at least ten dimensions.
3. The method of claim 1 , wherein the first recurrent neural network comprises a bi-directional RNN which generates a hidden state for each character of the word in forward and backward directions.
4. The method of claim 1 , wherein the word sequence model comprises a second recurrent neural network that receives the word embeddings and outputs the sequence representation.
5. The method of claim 4 , wherein the second recurrent neural network comprises a bi-directional RNN which generates a hidden state for each word of the text sequence in forward and backward directions.
6. The method of claim 1 , wherein the generating of the at least one trait prediction comprises generating a score for each of a plurality of predefined traits.
7. The method of claim 1 , wherein the trait model comprises at least one neural network.
8. The method of claim 7 , wherein the trait model comprises a neural network for each of a plurality of predefined traits, each of the neural networks being used to predict a respective score for a respective one of the plurality of predefined traits, based on the sequence representation.
9. The method of claim 1 , wherein the character sequence model, word sequence model, and trait model have been trained on a collection of text sequences, each labeled with a respective value for at least one predefined trait.
10. The method of claim 1 , further comprising jointly training the character sequence model, word sequence model, and trait model on a collection of text sequences, each labeled with a respective value for at least one predefined trait.
11. The method of claim 10 , wherein the training comprises, for each of a plurality of predefined traits, jointly training a respective character sequence model, word sequence model, and trait model for a respective one of the plurality of predefined traits.
12. The method of claim 1 , wherein the generating of the at least one trait prediction comprises generating a trait prediction for an author of a group of text sequences, based on respective sequence representations of the text sequences.
13. The method of claim 1 wherein the text sequence comprises a tweet.
14. The method of claim 13 , further comprising preprocessing the text sequence, comprising mapping features to single characters, the features being selected from hashtag-preceded topics, user mentions, and uniform resource locators.
15. A computer program product comprising a non-transitory recording medium storing instructions, which when executed on a computer, causes the computer to perform the method of claim 1 .
16. A system for trait prediction comprising:
a hierarchical prediction model comprising:
a character sequence model, which generates word representations for words of an input text sequence, the text sequence including a sequence of words, each word including a sequence of characters, at least some of the words including more than one character, each word representation being based on the respective character embeddings of each of the characters in the word, the character sequence model comprising a first recurrent neural network that receives the character embeddings for a word of the input text sequence and outputs the word representation for the word;
a word sequence model, which generates a sequence representation for the text sequence, based on the word representations, the word sequence model comprising a second recurrent neural network that receives the word embeddings and outputs the sequence representation; and
a trait model, which generates at least one trait prediction, based on the sequence representation;
a character embedding component, which generates character embeddings for characters of an input text sequence, each character embedding being a multidimensional representation;
a modeling component, which inputs the character embeddings for each word into the hierarchical prediction model; and
a processor which implements the character embedding component and modeling component.
17. The system of claim 16 , further comprising:
a learning component which learns parameters of the hierarchical prediction model on a collection of text sequences, each text sequence being labeled with a respective value for at least one predefined trait.
18. A method for representing text sequences and outputting information regarding an author of the text sequences, the method comprising:
receiving a group of text sequences, each text sequence comprising a sequence of words, each of the words comprising a sequence of at least one character, at least some of the words comprising more than one character;
for each text sequence:
with a character embedding component, generating a character embedding for each of the characters of the input text sequence, each character embedding being a multidimensional representation of at least ten dimensions and being a product of a one hot vector and a learned parameter matrix;
inputting the character embeddings into a trained hierarchical prediction model comprising:
a trained character sequence model, which generates word representations for words of the text sequence, based on respective character embeddings, each word representation being a multidimensional representation of at least ten dimensions, the character sequence model comprising a first recurrent neural network that receives the character embeddings for a word of the text sequence and outputs the word representation for the word; and
a trained word sequence model, which generates a sequence representation for the text sequence, based on the word representations, each sequence representation being a multidimensional representation of at least ten dimensions, the word sequence model comprising a second recurrent neural network that receives the word representations and outputs the sequence representation; and
generating at least one trait prediction with a trained trait model, based on the output sequence representation; and
outputting information regarding an author of the text sequences, based on the at least one trait prediction;
wherein at least one of the generating character embeddings and inputting the character embeddings into a hierarchical prediction model is performed with a processor.