Dynamic class weighting for training one or more neural networks
Apparatuses, systems, and techniques are presented to train neural networks and use those neural networks for inferencing tasks. In at least one embodiment, one or more neural networks are caused to be trained using weight parameters based, at least in part, on an amount of training data used to train the one or more neural networks.
1 . A processor, comprising:
one or more circuits to cause one or more neural networks to be trained to classify data according to a plurality of classes, wherein to cause the one or more neural networks to be trained the one or more circuits are further to:
calculate respective losses according to a loss function for different classes of the plurality of classes based on inferences of the different classes by the one or more neural networks, wherein calculation of the respective losses comprises adjusting a respective loss for each of at least two of the different classes using a different class-specific weight value parameter for each of the at least two different classes during training;
cause network parameters of the one or more neural networks to be updated during the training of the one or more neural networks based on the respective losses as adjusted using the different class-specific weight values; and
update one or more of the class-specific weight values for subsequent training based, at least in part, on an amount of loss according to the loss function for a corresponding class of the at least two different classes during the training.
2 . The processor of claim 1 , wherein the class-specific weight values correspond to different classes of objects to be detected by the one or more neural networks.
3 . The processor of claim 2 , wherein the one or more circuits are further to adjust the class-specific weight values during training based, at least in part, upon the relative amount of training data in each of the different classes of objects.
4 . The processor of claim 3 , wherein the one or more circuits are further to determine offsets from a set of default anchors to one or more objects detected in the individual training images.
5 . The processor of claim 4 , wherein the one or more circuits are further to determine a loss per class per anchor for the different classes of objects, and to adjust the class-specific weight values based at least in part upon the determined losses per class per anchor.
6 . The processor of claim 2 , wherein the one or more circuits are further to adjust the class-specific weight values using a momentum vector for each of a number of training epochs.
7 . A system, comprising:
one or more processors to cause one or more neural networks to be trained to classify data according to a plurality of classes, wherein to cause the one or more neural networks to be trained the one or more processors are further to:
calculate respective losses according to a loss function for different classes of the plurality of classes based on inferences of the different classes by the one or more neural networks, wherein calculation of the respective losses comprises adjusting a respective loss for each of at least two of the different classes using a different class-specific weight value for each of the at least two different classes during training;
cause network parameters of the one or more neural networks to be updated during the training of the one or more neural networks based on the respective losses as adjusted using the different class-specific weight values; and
update one or more of the class-specific weight values for subsequent training based, at least in part, on an amount of loss according to the loss function for a corresponding class of the at least two different classes during the training.
8 . The system of claim 7 , wherein the class-specific weight values correspond to different classes of objects to be detected by the one or more neural networks.
9 . The system of claim 8 , wherein the one or more processors are further to adjust the class-specific weight values during training based, at least in part, upon the relative amount of training data in each of the different classes of objects.
10 . The system of claim 9 , wherein the one or more processors are further to determine offsets from a set of default anchors to one or more objects detected in the individual training images.
11 . The system of claim 10 , wherein the one or more processors are further to determine a loss per class per anchor for the different classes of objects, and to adjust the class-specific weight values based at least in part upon the determined losses per class per anchor.
12 . The system of claim 8 , wherein the one or more processors are further to adjust the class-specific weight values using a momentum vector for each of a number of training epochs.
13 . A method, comprising:
causing one or more neural networks to be trained to classify data according to a plurality of classes, wherein said causing comprises:
calculating respective losses according to a loss function for different classes of the plurality of classes based on inferences of the different classes by the one or more neural networks, wherein calculation of the respective losses comprises adjusting a respective loss for each of at least two of the different classes using a different class-specific weight value for each of the at least two different classes during training;
causing network parameters of the one or more neural networks to be updated during the training of the one or more neural networks based on the respective losses as adjusted using the different class-specific weight values; and
updating one or more of the class-specific weight values for subsequent training based, at least in part, on an amount of loss according to the loss function for a corresponding class of the at least two different classes during the training.
14 . The method of claim 13 , wherein the class-specific weight values correspond to different classes of objects to be detected by the one or more neural networks.
15 . The method of claim 14 , further comprising:
adjusting the class-specific weight values during training based, at least in part, upon the relative amount of training data in each of the different classes of objects.
16 . The method of claim 15 , further comprising:
determining offsets from a set of default anchors to one or more objects detected in the individual training images.
17 . The method of claim 16 , further comprising:
determining a loss per class per anchor for the different classes of objects, and to adjust the class-specific weight values based at least in part upon the determined losses per class per anchor.
18 . The method of claim 14 , further comprising:
adjusting the class-specific weight values using a momentum vector for each of a number of training epochs.
19 . A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:
cause one or more neural networks to be trained to classify data according to a plurality of classes, wherein to cause the one or more neural networks to be trained the instructions if performed further cause the one or more processors to:
calculate respective losses according to a loss function for different classes of the plurality of classes based on inferences of the different classes by the one or more neural networks, wherein calculation of the respective losses comprises adjusting a respective loss for each of at least two of the different classes using a different class-specific weight value for each of the at least two different classes during training;
cause network parameters of the one or more neural networks to be updated during the training of the one or more neural networks based on the respective losses as adjusted using the different class-specific weight values; and
update one or more of the class-specific weight values for subsequent training based, at least in part, on an amount of loss according to the loss function for a corresponding class of the at least two different classes during the training.
20 . The non-transitory machine-readable medium of claim 19 , wherein the class-specific weight values correspond to different classes of objects to be detected by the one or more neural networks.
21 . The non-transitory machine-readable medium of claim 20 , wherein the instructions if performed further cause the one or more processors to:
adjust the class-specific weight values during training based, at least in part, upon the relative amount of training data in each of the different classes of objects.
22 . The non-transitory machine-readable medium of claim 21 , wherein the instructions if performed further cause the one or more processors to:
determine offsets from a set of default anchors to one or more objects detected in the individual training images.
23 . The non-transitory machine-readable medium of claim 22 , wherein the instructions if performed further cause the one or more processors to:
determine a loss per class per anchor for the different classes of objects, and to adjust the class-specific weight values based at least in part upon the determined losses per class per anchor.
24 . The non-transitory machine-readable medium of claim 20 , wherein the instructions if performed further cause the one or more processors to:
adjust the class-specific weight values using a momentum vector for each of a number of training epochs.
25 . A network training system, comprising:
one or more processors to cause one or more neural networks to be trained to classify data according to a plurality of classes, wherein to cause the one or more neural networks to be trained the one or more processors are further to:
calculate respective losses according to a loss function for different classes of the plurality of classes based on inferences of the different classes by the one or more neural networks, wherein calculation of the respective losses comprises adjusting a respective loss for each of at least two of the different classes using a different class-specific weight value for each of the at least two different classes during training;
cause network parameters of the one or more neural networks to be updated during the training of the one or more neural networks based on the respective losses as adjusted using the different class-specific weight values; and
update one or more of the class-specific weight values for subsequent training based, at least in part, on an amount of loss according to the loss function for a corresponding class of the at least two different classes during the training; and
memory for storing the network parameters for the one or more neural networks.
26 . The network training system of claim 25 , wherein the class-specific weight values correspond to different classes of objects to be detected by the one or more neural networks.
27 . The network training system of claim 26 , wherein the one or more processors are further to adjust the class-specific weight values during training based, at least in part, upon the relative amount of training data in each of the different classes of objects.
28 . The network training system of claim 27 , wherein the one or more processors are further to determine offsets from a set of default anchors to one or more objects detected in the individual training images.
29 . The network training system of claim 28 , wherein the one or more processors are further to determine a loss per class per anchor for the different classes of objects, and to adjust the class-specific weight values based at least in part upon the determined losses per class per anchor.
30 . The network training system of claim 26 , wherein the one or more processors are further to adjust the class-specific weight values using a momentum vector for each of a number of training epochs.