Method and apparatus for generating training data to train student model using teacher model
Disclosed is a training data generating apparatus and method to change output data inappropriate for training a student model to an ignore value such that the inappropriate output data is not used to train the student model, change output data appropriate for training the student model such that the student model outputs an improved result in comparison to output data of a teacher model, and change a label value in a form of probability to an identifier corresponding to intervals divided based on threshold label values using the teacher model based on input data.
1. A method of generating training data, the method comprising:
inputting input data to a teacher model and acquiring from the teacher model a label value of the input data, the label value comprising a probability of the input data predicted by the teacher model;
classifying the label value into one of a plurality of intervals based on comparing the label value to one or more threshold probabilities, wherein the intervals comprise an interval less than a second threshold probability, an interval greater than or equal to the second threshold probability and less than a first threshold probability, and an interval greater than or equal to the first threshold probability, wherein the first threshold probability is greater than the second threshold probability; and
generating training data for training a student model corresponding to the teacher model based on combining the input data and an identifier of the corresponding interval of the plurality of intervals.
2. The method of claim 1 , wherein the classifying comprises classifying the label value into a positive interval among the intervals, based on the label value being greater than or equal to the first threshold probability, and the classification of the label value into the positive interval indicating that the input data belongs to a class associated with the label value.
3. The method of claim 1 , wherein the classifying comprises classifying the label value into a negative interval among the intervals, based on the label value being less than the first threshold probability, and the classification of the label value into the negative interval indicating that the input data does not belong to a class associated with the label value.
4. The method of claim 1 , wherein the one of the intervals comprises an ignore interval, and wherein the classifying comprises classifying the label value into the ignore interval, the classification into the ignore interval causing the label value to be not applied to a training of the student model when the combined input data is used to train the student model.
5. The method of claim 1 , wherein a number of hidden layers in the student model is less than a number of hidden layers in the teacher model, or a number of filters in the student model is less than a number of filters in the teacher model.
6. The method of claim 1 , wherein the plurality of intervals comprise one or more rational numbers indicating whether the input data corresponds to a class associated with the label value.
7. The data recognition method of claim 1 , further comprising generating additional training data and repeating the training of the student model, in response to an accuracy of the student model being lesser than a threshold.
8. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the user authentication method of claim 1 .
9. A data recognition method comprising:
receiving a target data item to be recognized; and
recognizing the target data item using a student model that is trained,
wherein the student model has been trained with training data generated by:
predicting, by a teacher model corresponding to the student model, for the target data item, label values for respective classification labels for the target data item, each classification label having a respectively corresponding predicted label value,
classifying the label values according to intervals by determining which of the intervals each label value is in, wherein the intervals comprise an interval less than a second threshold probability, an interval greater than or equal to the second threshold probability and less than a first threshold probability, and an interval greater than or equal to the first threshold probability.
10. The data recognition method of claim 9 , wherein the intervals comprise a positive interval indicating that a classification label corresponds to a class associated with the label value, a negative interval indicating that a classification label does not correspond to the class associated with the label value, and an ignore interval indicating that the classification label is not applied to a training of the student model when the target data item is used to train the student model.
11. The data recognition method of claim 9 , wherein the teacher model comprises a number of hidden layers less than a number of hidden layers in the student model or a number of filters less than a number of filters included in the student model.
12. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the user authentication method of claim 9 .
13. An apparatus for generating training data, the apparatus comprising:
a memory configured to store a single input data item; and
a processor configured to:
input the single input data item to at least one teacher model to acquire probabilities that the input data item matches respective classes;
compare the probabilities to threshold probabilities comprising a first threshold probability greater than a second threshold probability, and based thereon change the probabilities to respective values corresponding to results of the comparing; and
generate a training data item for training a student model corresponding to the at least one teacher model by combining the input data item and the values.
14. The apparatus of claim 13 , wherein the processor is further configured to change a probability, among the probabilities, to a positive value indicating that the input data item matches a class corresponding to the probability, based on the probability being greater than or equal to a threshold probability among the threshold probabilities.
15. The apparatus of claim 13 , wherein the processor is further configured to change a probability, among the probabilities, to a negative value indicating that the input data does not match a class corresponding to the probability, in response to the probability being less than or equal to a threshold probability among the threshold probabilities.
16. The apparatus of claim 13 , wherein the processor is further configured to change a probability, among the probabilities, to an ignore-value indicating that a feature of the data item corresponding to the probability is not applied to a training of the student model when the input data item is used to train the student model.
17. The apparatus of claim 13 , wherein the at least one teacher model comprises a plurality of teacher models and the processor is further configured to acquire the probabilities based on the plurality of teacher models.
18. The apparatus of claim 16 , wherein, the probability is changed to the ignore-value based on the probability being greater than a first of the probability thresholds and less than a second of the probability thresholds, wherein the first probability threshold is greater than zero and the second probability threshold is less than one.