Method and device for training neural network
The present disclosure relates to neural network training. The neural network training relates to a training method, a training device, and a system including the neural network. The neural network training includes extracting annotation data and first reliability values for first data using a neural network trained based on training data, selecting second data from among the first data based on the second data having second reliability values greater than or equal to a threshold value, expanding the training data based on the second data, and retraining the neural network based on the expanded training data.
1 . A neural network training method comprising:
providing a first training set comprising annotated training data;
training a neural network in a first stage using the first training set;
applying the trained neural network to unannotated data to obtain annotated data and reliability values for labels corresponding to the annotated data;
selecting, as reference data, a piece of the annotated data having a first reliability value for its label that is greater than or equal to a threshold value;
generating a plurality of transformed versions of the reference data by transforming the reference data;
obtaining a reliability value for a corresponding label of each transformed version of the reference data;
identifying two or more transformed versions of the reference data each having a second reliability value that is greater than or equal to a predefined reference value;
selecting, at least two transformed versions of the reference data from the two or more transformed versions of the reference data;
creating a second training set including the first training set and the at least two transformed versions of the reference data; and
retraining the neural network in a second stage based on the second training set, wherein the at least two transformed versions of the reference data include a first number of transformed versions at a first reliability range and a second number of transformed versions at a second reliability range, wherein the first number is different from the second number.
2 . The neural network training method of claim 1 , wherein selecting at least two transformed versions of the reference data comprises selecting an amount of the at least two transformed versions of the reference data having the reliability values within the first reliability range corresponding to a ratio set with respect to an amount of the first training set.
3 . The neural network training method of claim 1 , wherein selecting at least two transformed versions of the reference data comprises selecting a first amount of the at least two transformed versions of the reference data having the reliability values within the first reliability range and a second amount of the at least two transformed versions of the reference data having the reliability values within the second reliability range from the transformed versions based on a reliability distribution for data in the second training set.
4 . The neural network training method of claim 1 , further comprising:
measuring a performance of the neural network retrained in the second stage based on the second training set;
determining whether the performance of the neural network is below a threshold level;
adjusting the threshold value in response to determining that the performance of the neural network is below the threshold level; and
performing the selection of the reference data, the generating of the plurality of transformed versions of the reference data, and the creating of the second training set from among the annotated data based on the adjusted threshold value.
5 . The neural network training method of claim 1 , further comprising:
determining whether a condition for finishing training is satisfied;
in response to determining that the condition is satisfied, outputting the neural network retrained in the second stage based on the second training set; and
in response to determining that the condition is not satisfied, extracting additional annotation data and additional reliability values.
6 . The neural network training method of claim 5 , wherein the determining of whether the condition for finishing the training is satisfied comprises, in response to determining that an amount of an expanded training set is greater than or equal to a set amount, determining that the condition is satisfied.
7 . The neural network training method of claim 5 , wherein the determining of whether the condition for finishing the training is satisfied comprises, in response to determining that a performance of the retrained neural network is greater than or equal to a reference level, determining that the condition is satisfied.
8 . A neural network training device comprising:
a memory storing one or more instructions; and
at least one processor configured to execute the one or more instructions stored in the memory to:
provide a first training set comprising annotated training data;
train a neural network in a first stage using the first training set;
apply the trained neural network to unannotated data to obtain annotated data and reliability values for labels corresponding to the annotated data;
select, as reference data, a piece of the annotated data having a first reliability value for its label that is greater than or equal to a threshold value;
generate a plurality of transformed versions of the reference data by transforming the reference data;
obtain a reliability value for a corresponding label of each transformed version of the reference data;
select, when two or more transformed versions of the reference data each have a second reliability value that is greater than or equal to a predefined reference value, at least two transformed versions of the reference data from the two or more transformed versions of the reference data;
create a second training set including the first training set and the at least two transformed versions of the reference data; and
retrain the neural network in a second stage based on the second training set,
wherein the at least two transformed versions of the reference data include a first number of transformed versions at a first reliability range and a second number of transformed versions at a second reliability range, wherein the first number is different from the second number.
9 . The neural network training device of claim 8 ,
wherein the at least one processor is further configured to execute the one or more instructions to:
process the reference data using a plurality of methods to generate the plurality of transformed versions of the reference data.
10 . The neural network training device of claim 9 , wherein the at least one processor is further configured to execute the one or more instructions to select the at least two transformed versions of the reference data to be added to the first training set based on the reliability values such that an amount of the at least two transformed versions of the reference data to be added satisfies a reference amount set for each of a plurality of reliability value ranges that are greater than or equal to the reference value.
11 . The neural network training device of claim 8 , wherein the at least one processor is further configured to execute the one or more instructions to select an amount of the transformed versions of the reference data to be included in the first training set corresponding to a preset ratio set with respect to an amount of the first training set.
12 . The neural network training device of claim 11 , wherein the preset ratio is greater than 0 but less than or equal to 1.
13 . The neural network training device of claim 11 , wherein the at least one processor is further configured to execute the one or more instructions to repeat the application of the trained neural network to the unannotated data, the selecting of the reference data, the generating of the transformed versions of the reference data, the creating of the second training set, and the retraining until the amount of added data to the first training set reaches a set amount or a performance of the trained neural network reaches a set reference level.
14 . The neural network training device of claim 8 , wherein the at least one processor includes at least one accelerator configured to perform computations of the neural network during inference and the retraining.
15 . The neural network training device of claim 8 , provided in at least one of an application processor, a data server, and an advanced driver assistance system.
16 . A neural network system comprising:
a memory storing one or more instructions; and
at least one processor configured to execute the one or more instructions stored in the memory to:
receive a first training set including annotated training data:
train a neural network in a first stage using the first training set;
apply the trained neural network to unannotated data to obtain annotated data and reliability values for labels corresponding to each piece of the annotated data;
select, as reference data, a piece of the annotated data having a first reliability value for its label that satisfies a threshold value;
generate a plurality of transformed versions of the reference data by transforming the reference data and obtaining a reliability value for a corresponding label of each transformed version of the reference data;
select, when two or more transformed versions of the reference data each have a second reliability value that is greater than or equal to a predefined reference value, at least two transformed versions of the reference data from the two or more transformed versions of the reference data, wherein the at least two transformed versions of the reference data comprise a first amount of some of the transformed versions of the reference data having the reliability values within a first reliability range and a second amount of other some of the transformed versions of the reference data having the reliability values within a second reliability range from among the plurality of the transformed versions of the reference data based on a reliability distribution for data in a training set, wherein the first reliability range and the second reliability range are greater than or equal to the predefined reference value, wherein the first amount of the some of the transformed versions of the reference data is different from the second amount of the other some of the transformed versions of the reference data;
create a second training set including the first training set, the first amount of the some of the transformed versions of the reference data and the second amount of the other some of the transformed versions of the reference data;
retrain a neural network in a second stage based on the second training set; and
extract recognition information for input data by performing inference on the input data based on the retrained neural network.