Removing personal information from text using a neural network
A neural network may be used to remove personal information from text (such as names, addresses, credit card numbers, or social security numbers), and replace the personal information with a label indicating the type or class of the removed information. The neural network may comprise multiple layers that compute a context vector for words of the text, compute label scores for words of the text using the context vectors, and select a label for each word using the label scores. Words corresponding to certain labels may be replaced with a label, such as replacing the digits of a credit card number with a label <cc_number>. The redacted text may then be presented to a person or stored for later processing.
1. A computer-implemented method for removing personal information from text using a neural network, the method comprising:
obtaining the neural network, wherein the neural network is configured to process the text and select a label from a plurality of possible labels for each word of the text, wherein each label corresponds to a class of words, and wherein at least one label corresponds to a class of words to be removed from the text;
receiving the text;
obtaining a word embedding for each word of the text, where a word embedding represents a word in a vector space;
computing a context vector for each word of the text by processing the word embeddings with a first layer of the neural network, where a context vector for a given word includes information about words before or after the given word;
computing label scores for each word of the text by processing each of the context vectors with a second layer of the neural network, wherein each label score indicates a match between a word and a class of words;
selecting a label for each word of the text by processing the label scores with a third layer of the neural network; and
generating redacted text by replacing a first word of the text with a first label corresponding to the first word.
2. The computer-implemented method of claim 1 , comprising causing the redacted text to be presented to a person.
3. The computer-implemented method of claim 1 , comprising classifying the redacted text by processing the redacted text with a text classifier.
4. The computer-implemented method of claim 1 , wherein the first layer is a recurrent neural network layer or a bidirectional recurrent neural network layer.
5. The computer-implemented method of claim 1 , wherein the second layer comprises a classifier.
6. The computer-implemented method of claim 1 , wherein the third layer comprises a conditional random field.
7. The computer-implemented method of claim 1 , wherein the text is from a message received from a customer of a company and relates to obtaining support from the company.
8. The computer-implemented method of claim 1 , comprising:
storing the redacted text in a first data store;
generating second redacted text, wherein the second redacted text has a higher redaction level then the redacted text; and
storing the second redacted text in a second data store.
9. A system for removing personal information from text using a neural network, the system comprising at least one computer configured to:
obtain the neural network, wherein the neural network is configured to process the text and select a label from a plurality of possible labels for each word of the text, wherein each label corresponds to a class of words, and wherein at least one label corresponds to a class of words to be removed from the text;
receive the text;
obtain a word embedding for each word of the text;
compute a context vector for each word of the text by processing the word embeddings with a first layer of the neural network;
compute label scores for each word of the text by processing each of the context vectors with a second layer of the neural network, wherein each label score indicates a match between a word and a class of words;
select a label for each word of the text by processing the label scores with a third layer of the neural network; and
generate redacted text by replacing a first word of the text with a first label corresponding to the first word.
10. The system of claim 9 , wherein the at least one computer is configured to obtain a word embedding by:
obtaining a first embedding corresponding to words of a vocabulary;
obtaining character embeddings for characters of the word, wherein each character embedding corresponds to a character of a set of characters;
computing a second embedding using the character embeddings; and
obtaining the word embedding by combining the first embedding and the second embedding.
11. The system of claim 9 , wherein the at least one computer is configured to:
generate linguistic features for each word of the text; and
computing the label scores for each word of the text comprises processing the linguistic features with the second layer.
12. The system of claim 9 , wherein the second layer comprises a multi-layer perceptron.
13. The system of claim 9 , wherein the at least one computer is configured to generate the redacted text by replacing sequences of a label with a single label.
14. The system of claim 9 , wherein the at least one computer is configured to, before obtaining the word embedding for each word of the text, replacing digits in the text with a token that represents digits.
15. The system of claim 9 , wherein the system is implemented by a third party providing services to a plurality of companies.
16. One or more non-transitory computer-readable media comprising computer executable instructions that, when executed, cause at least one processor to perform actions comprising:
obtaining a neural network, wherein the neural network is configured to process text and select a label from a plurality of possible labels for each word of the text, wherein each label corresponds to a class of words, and wherein at least one label corresponds to a class of words to be removed from the text;
receiving the text;
obtaining a word embedding for each word of the text;
computing a context vector for each word of the text by processing the word embeddings with a first layer of the neural network;
computing label scores for each word of the text by processing each of the context vectors with a second layer of the neural network, wherein each label score indicates a match between a word and a class of words;
selecting a label for each word of the text by processing the label scores with a third layer of the neural network; and
generating redacted text by replacing a first word of the text with a first label corresponding to the first word.
17. The one or more non-transitory computer-readable media of claim 16 , wherein a context vector of a word corresponds to a hidden state vector of the first layer of the neural network.
18. The one or more non-transitory computer-readable media of claim 16 , wherein the first layer of the neural network is a convolutional layer.
19. The one or more non-transitory computer-readable media of claim 16 , wherein the third layer comprises a sequence model.
20. The one or more non-transitory computer-readable media of claim 16 , wherein generating the redacted text comprises removing personally identifiable information from the text.