Contrastive sequence-to-sequence data selector
A method includes generating a base model by training with a first dataset of data pairs and generating an adapted model by training the base model on a second dataset of data pairs. The method also includes determining a contrastive score for each data pair of a third dataset of data pairs using the base model and the adapted model. The contrastive score is indicative of a probability of quality of the respective data pair. The method also includes training a target model using the data pairs of the third dataset and the contrastive scores.
1 . A computer-implemented method executed by data processing hardware that causes the data processing hardware to perform operations comprising:
receiving a dataset comprising a plurality of data pairs;
for each respective data pair of the plurality of data pairs:
determining a first probability score for the respective data pair using a first trained machine learning model;
determining a second probability score for the respective data pair using a second trained machine learning model, the second trained machine learning model different from the first trained machine learning model; and
determining, based on the first probability score and the second probability score, a respective contrastive score for the respective data pair, the respective contrastive score indicative of a probability of quality of the respective data pair;
selecting, based on the respective contrastive scores, a batch of data pairs from the dataset, wherein selecting the batch of data pairs comprises:
determining a selection ratio based on a current training step of a plurality of training steps, the selection ratio decreasing after each training step;
determining a batch size based on the selection ratio and a size of the dataset;
selecting a number of data pairs from the dataset that corresponds with the determined batch size;
sorting, based on the respective contrastive scores, the selected data pairs; and
removing, from the batch, a quantity of the selected data pairs with the lowest contrastive scores based on a removal ratio, the removal ratio an inverse of the selection ratio and increasing after each training step; and
at each training step of the plurality of training steps, training, using the sorted batch of data pairs of the batch of data pairs with the removed data pairs with the lowest contrastive scores, a target machine learning model, the target machine learning model different from the first trained machine learning model and the second trained machine learning model.
2 . The method of claim 1 , wherein a first model size of the first trained machine learning model is larger than a second model size of the second trained machine learning model.
3 . The method of claim 1 , wherein the first trained machine learning model is trained on a first dataset and the second trained machine learning model is trained on a second dataset different from the first dataset.
4 . The method of claim 1 , wherein the batch size is equal to a fixed batch size divided by the selection ratio.
5 . A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
receiving a dataset comprising a plurality of data pairs;
for each respective data pair of the plurality of data pairs:
determining a first probability score for the respective data pair using a first trained machine learning model;
determining a second probability score for the respective data pair using a second trained machine learning model, the second trained machine learning model different from the first trained machine learning model; and
determining, based on the first probability score and the second probability score, a respective contrastive score for the respective data pair, the respective contrastive score indicative of a probability of quality of the respective data pair;
selecting, based on the respective contrastive scores, a batch of data pairs from the dataset, wherein selecting the batch of data pairs comprises:
determining a selection ratio based on a current training step of a plurality of training steps, the selection ratio decreasing after each training step;
determining a batch size based on the selection ratio and a size of the dataset;
selecting a number of data pairs from the dataset that corresponds with the determined batch size;
sorting, based on the respective contrastive scores, the selected data pairs; and
removing, from the batch, a quantity of the selected data pairs with the lowest contrastive scores based on a removal ratio, the removal ratio an inverse of the selection ratio and increasing after each training step; and
at each training step of the plurality of training steps, training, using the sorted batch of data pairs of the batch of data pairs with the removed data pairs with the lowest contrastive scores, a target machine learning model, the target machine learning model different from the first trained machine learning model and the second trained machine learning second model.
6 . The system of claim 5 , wherein a first model size of the first trained machine learning model is larger than a second model size of the second trained machine learning model.
7 . The system of claim 5 , wherein the first trained machine learning model is trained on a first dataset and the second trained machine learning model is trained on a second dataset different from the first dataset.
8 . The system of claim 5 , wherein the batch size is equal to a fixed batch size divided by the selection ratio.