Removing undesirable signals from language models using negative data
A method for training a language model using negative data may include accessing a first training corpus comprising positive training data and accessing a second training corpus comprising negative training data. The method may further include training a first language model using at least the first training corpus, the second training corpus, and a maximum likelihood function. The maximum likelihood function may maximize the likelihood of the first language model predicting the positive training data while minimizing the likelihood of the first language model predicting the negative training data.
1. A method for training a language model using negative data, the method comprising:
accessing a first training corpus comprising positive training data for training a first language model;
accessing a second language model, wherein the second language model comprises an n-gram model or a transformer model that is inhibited by removing position information from the transformer model and configured to generate outputs that are less grammatically correct than outputs generated by the first language model;
generating output text from the second language model to use as a second training corpus of negative training data; and
training the first language model using at least the first training corpus, the second training corpus, and a maximum likelihood function, wherein the maximum likelihood function maximizes a likelihood of the first language model predicting the positive training data while minimizing a likelihood of the first language model predicting the negative training data.
2. The method of claim 1 , wherein minimizing the likelihood of the first language model predicting the negative training data comprises:
maximizing 1 minus the likelihood of the first language model predicting the negative training data.
3. The method of claim 2 , wherein the maximum likelihood function maximizes the likelihood of 1 minus the likelihood of the first language model predicting the negative training data by:
maximizing a lower bound on the likelihood of 1 minus the likelihood of the first language model predicting the negative training data.
4. The method of claim 3 , wherein the lower bound comprises a product of 1 minus a probability of the first language model predicting each word in the second training corpus.
5. The method of claim 1 , wherein the likelihood of the first language model predicting the positive training data is calculated using a likelihood function that accepts the positive training data and a plurality of weights for the first language model as inputs.
6. The method of claim 1 , wherein the likelihood of the first language model predicting the negative training data is calculated using a likelihood function that accepts the negative training data and a plurality of weights for the first language model as inputs.
7. The method of claim 6 , wherein the likelihood function optimizes values for the plurality of weights.
8. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
accessing a first training corpus comprising positive training data for training a first language model;
accessing a second language model, wherein the second language model comprises an n-gram model or a transformer model that is inhibited by removing position information from the transformer model and configured to generate outputs that are less grammatically correct than outputs generated by the first language model;
generating output text from the second language model to use as a second training corpus of negative training data; and
training the first language model using at least the first training corpus, the second training corpus, and a maximum likelihood function, wherein the maximum likelihood function maximizes a likelihood of the first language model predicting the positive training data while minimizing a likelihood of the first language model predicting the negative training data.
9. The non-transitory computer-readable medium of claim 8 wherein training the first language model using at least the first training corpus, the second training corpus, and the maximum likelihood function removes negative n-gram statistics from the first language model.
10. The non-transitory computer-readable medium of claim 8 , wherein training the first language model using at least the first training corpus, the second training corpus, and the maximum likelihood function decreases an error rate for subject-verb agreement.
11. The non-transitory computer-readable medium of claim 8 , wherein the second language model is inhibited such that the second language model does not consider word position.
12. The non-transitory computer-readable medium of claim 8 , wherein the second language model comprises a transformer-based model with word-location identifiers removed.
13. The non-transitory computer-readable medium of claim 8 , wherein the outputs from the n-gram model or a transformer model that is inhibited comprise text strings that may be provided as training data to the first language model.
14. The non-transitory computer-readable medium of claim 8 , wherein the transformer model is also inhibited by implementing a statistical assumption that is not true about a language of the second training corpus in general, but which is true about the data in the second training corpus.
15. A system comprising:
one or more processors; and
one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
accessing a first training corpus comprising positive training data for training a first language model;
accessing a second language model, wherein the second language model comprises an n-gram model or a transformer model that is inhibited by removing position information from the transformer model and configured to generate outputs that are less grammatically correct than outputs generated by the first language model;
generating output text from the second language model to use as a second training corpus of negative training data; and
training the first language model using at least the first training corpus, the second training corpus, and a maximum likelihood function, wherein the maximum likelihood function maximizes a likelihood of the first language model predicting the positive training data while minimizing a likelihood of the first language model predicting the negative training data.
16. The system of claim 15 , wherein the first language model comprises a neural language model.
17. The system of claim 15 , wherein the first language model comprises a transformer-based language model.
18. The system of claim 15 , wherein the first training corpus does not include the second training corpus.
19. The system of claim 15 , wherein the first training corpus and the second training corpus are both subsets of a larger training corpus.