Using multiple trained models to reduce data labeling efforts
A method of labeling training data includes inputting a plurality of unlabeled input data samples into each of a plurality of pre-trained neural networks and extracting a set of feature embeddings from multiple layer depths of each of the plurality of pre-trained neural networks. The method also includes generating a plurality of clusterings from the set of feature embeddings. The method also includes analyzing, by a processing device, the plurality of clusterings to identify a subset of the plurality of unlabeled input data samples that belong to a same unknown class. The method also includes assigning pseudo-labels to the subset of the plurality of unlabeled input data samples.
1. A method, comprising:
inputting a plurality of unlabeled input data samples into each of a plurality of pre-trained neural networks and extracting a set of feature embeddings from multiple layer depths of each of the plurality of pre-trained neural networks;
generating a plurality of clusterings from the set of feature embeddings;
analyzing, by a processing device, the plurality of clusterings to identify a subset of the plurality of unlabeled input data samples that belong to a same unknown class; and
assigning pseudo-labels to the subset of the plurality of unlabeled input data samples.
2. The method of claim 1 , wherein analyzing the plurality of clusterings comprises computing a cluster quality for each of the plurality of clusterings and identifying a feature embedding of the set of feature embeddings based on the cluster quality.
3. The method of claim 2 , wherein the cluster quality comprises a Davies Bouldin score or a Silhouette score.
4. The method of claim 1 , wherein analyzing the plurality of clusterings comprises determining a degree of cluster member overlap between two or more of the set of feature embeddings.
5. The method of claim 1 , wherein the plurality of pre-trained neural networks is selected from a collection of neural networks that have been trained for a variety of tasks, the method comprising:
analyzing metadata associated with the collection of neural networks to identify at least one of a modality or an objective for each of the collection of neural networks; and
selecting the plurality of pre-trained neural networks from the collection of neural networks based on at least one of the modality or the objective.
6. The method of claim 1 , wherein each of the plurality of pre-trained neural networks is trained to perform a task comprising at least one of: an image classification task, a video classification task, an audio classification task, a text categorization task, a face recognition task, or a machine translation task.
7. The method of claim 1 , further comprising:
receiving a label for a first sample of the plurality of unlabeled input data samples; and
assigning the label to a second sample of the plurality of unlabeled input data samples based on the second sample having a same pseudo-label as the first sample.
8. A system comprising:
a memory; and
a processing device operatively coupled to the memory, the processing device to:
input a plurality of unlabeled input data samples into each of a plurality of pre-trained neural networks and extract a set of feature embeddings from multiple layer depths of each of the plurality of pre-trained neural networks;
generate a plurality of clusterings from the set of feature embeddings;
analyze the plurality of clusterings to identify a subset of the plurality of unlabeled input data samples that belong to a same unknown class; and
assign pseudo-labels to the subset of the plurality of unlabeled input data samples.
9. The system of claim 8 , wherein to analyze the plurality of clusterings, the processing device is to compute a cluster quality for each of the plurality of clusterings and identify a feature embedding of the set of feature embeddings based on the cluster quality.
10. The system of claim 9 , wherein the cluster quality comprises a Davies Bouldin score or a Silhouette score.
11. The system of claim 8 , wherein to analyze the plurality of clusterings, the processing device is to determine a degree of cluster member overlap between two or more of the set of feature embeddings.
12. The system of claim 8 , wherein the plurality of pre-trained neural networks is selected from a collection of neural networks that have been trained for a variety of tasks, the processing device:
analyze metadata associated with the collection of neural networks to identify at least one of a modality or an objective for each of the collection of neural networks; and
select the plurality of pre-trained neural networks from the collection of neural networks based on at least one of the modality or the objective.
13. The system of claim 8 , wherein each of the plurality of pre-trained neural networks is trained to perform a task comprising at least one of: an image classification task, a video classification task, an audio classification task, a text categorization task, a face recognition task, or a machine translation task.
14. The system of claim 8 , wherein the processing device is further to:
receive a label for a first sample of the plurality of unlabeled input data samples; and
assign the label to a second sample of the plurality of unlabeled input data samples based on the second sample having a same pseudo-label as the first sample.
15. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by a processing device, cause the processing device to:
input a plurality of unlabeled input data samples into each of a plurality of pre-trained neural networks and extract a set of feature embeddings from multiple layer depths of each of the plurality of pre-trained neural networks;
generate a plurality of clusterings from the set of feature embeddings;
analyze, by the processing device, the plurality of clusterings to identify a subset of the plurality of unlabeled input data samples that belong to a same unknown class; and
assign pseudo-labels to the subset of the plurality of unlabeled input data samples.
16. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions to cause the processing device to analyze the plurality of clusterings cause the processing device to compute a cluster quality for each of the plurality of clusterings and identify a feature embedding of the set of feature embeddings based on the cluster quality.
17. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions to cause the processing device to analyze the plurality of clusterings cause the processing device to determine a degree of cluster member overlap between two or more of the set of feature embeddings.
18. The non-transitory computer-readable storage medium of claim 15 , wherein the plurality of pre-trained neural networks is selected from a collection of neural networks that have been trained for a variety of tasks, and wherein the instructions cause the processing device to:
analyze metadata associated with the collection of neural networks to identify at least one of a modality or an objective for each of the collection of neural networks; and
select the plurality of pre-trained neural networks from the collection of neural networks based on at least one of the modality or the objective.
19. The non-transitory computer-readable storage medium of claim 15 , wherein each of the plurality of pre-trained neural networks is trained to perform a task comprising at least one of: an image classification task, a video classification task, an audio classification task, a text categorization task, a face recognition task, or a machine translation task.
20. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions further cause the processing device to:
receive a label for a first sample of the plurality of unlabeled input data samples; and
assign the label to a second sample of the plurality of unlabeled input data samples based on the second sample having a same pseudo-label as the first sample.