Systems and methods for training multi-class object classification models with partially labeled training data
Systems and methods of the present disclosure are directed to a computer-implemented method for training a machine-learned multi-class object classification model with partially labeled training data. The method can include obtaining image data depicting objects and ground truth data comprising a subset of object class annotations respectively associated with a subset of object classes of a plurality of object classes. The method can include processing the image data with the machine-learned multi-class object classification model to obtain object classification data. The method can include evaluating a loss function that evaluates a multi-class classification loss and adjusting one or more parameters of the multi-class object classification model based on the loss function.
1 . A computing system for training a multi-class object classification model with partially labeled training data, comprising:
one or more processors;
a machine-learned multi-class object classification model configured to classify a plurality of object classes; and
one or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising:
obtaining image data depicting one or more objects and ground truth data comprising a subset of object class annotations respectively associated with a subset of object classes of the plurality of object classes;
processing the image data with the machine-learned multi-class object classification model to obtain object classification data;
evaluating a loss function that evaluates a multi-class classification loss comprising a difference between the object classification data and the subset of object class annotations, wherein the loss function comprises a plurality of weighted loss signals respectively associated with the plurality of object classes, wherein the weight of each of the weighted loss signals has a magnitude greater than zero and is based at least in part on the inclusion of the object class associated with the respective loss signal within the subset of object classes; and
adjusting one or more parameters of the machine-learned multi-class object classification model based at least in part on the loss function.
2 . The computing system of claim 1 , wherein:
the weight of each of the weighted loss signals is a normalized value;
a first weighted loss signal of the plurality of weighted loss signals is associated with a first object class that is included within the subset of object classes;
a second weighted loss signal of the plurality of weighted loss signals is associated with a second object class that is excluded from the subset of object classes; and
the weight of the first weighted loss signal is greater than the weight of the second weighted loss signal.
3 . The computing system of claim 1 , wherein the one or more parameters of the machine-learned multi-class object classification model are adjusted proportionally to the weights of each of the weighted loss signals of the loss function.
4 . The computing system of claim 1 , wherein:
a first annotation of the subset of object class annotations comprises a bounding box and a label for a corresponding object of a first object class depicted in the image data; and
a second annotation of the subset of object class annotations comprises a label indicating that an object of a second object class is not depicted in the image data.
5 . The computing system of claim 1 , wherein the object classification data comprises one or more predicted object class annotations for one or more objects predicted to be depicted in the image data.
6 . The computing system of claim 1 , wherein evaluating the loss function comprises evaluating a subset of weighted loss signals of the loss function respectively associated with the subset of object classes.
7 . The computing system of claim 1 , wherein the operations further comprise:
obtaining additional image data depicting one or more additional objects; and
processing the additional image data with the machine-learned multi-class classification model to obtain an image classification output, the image classification output comprising one or more labels descriptive of the additional image data.
8 . A computer-implemented method for training a machine-learned multi-class object classification model with partially labeled training data, comprising:
obtaining, by a computing system comprising one or more computing devices, image data depicting one or more objects and ground truth data comprising a subset of object class annotations respectively associated with a subset of object classes of a plurality of object classes;
processing, by the computing system, the image data with the machine-learned multi-class object classification model to obtain object classification data;
evaluating, by the computing system, a loss function that evaluates a multi-class classification loss comprising a difference between the object classification data and the subset of object class annotations, wherein the loss function comprises a plurality of weighted loss signals respectively associated with the plurality of object classes, wherein the weight of each of the weighted loss signals has a magnitude greater than zero and is based at least in part on the inclusion of the object class associated with the respective loss signal within the subset of object classes; and
adjusting, by the computing system, one or more parameters of the machine-learned multi-class object classification model based at least in part on the loss function.
9 . The computer-implemented method of claim 8 , wherein:
the weight of each of the weighted loss signals is a normalized value;
a first weighted loss signal of the plurality of weighted loss signals is associated with a first object class that is included within the subset of object classes;
a second weighted loss signal of the plurality of weighted loss signals is associated with a second object class that is excluded from the subset of object classes; and
the weight of the first weighted loss signal is greater than the weight of the second weighted loss signal.
10 . The computer-implemented method of claim 8 , wherein the one or more parameters of the machine-learned multi-class object classification model are adjusted proportionally to the weights of each of the weighted loss signals of the loss function.
11 . The computer-implemented method of claim 8 , wherein:
a first annotation of the subset of object class annotations comprises a bounding box and a label for a corresponding object of a first object class depicted in the image data; and
a second annotation of the subset of object class annotations comprises a label indicating that an object of a second object class is not depicted in the image data.
12 . The computer-implemented method of claim 8 , wherein the object classification data comprises one or more predicted object class annotations for one or more objects predicted to be depicted in the image data.
13 . The computer-implemented method of claim 8 , wherein evaluating the loss function comprises evaluating, by the computing system, a subset of weighted loss signals of the loss function respectively associated with the subset of object classes.
14 . The computer-implemented method of claim 8 , wherein the method further comprises:
obtaining, by the computing system, additional image data depicting one or more additional objects; and
processing, by the computing system, the additional image data with the machine-learned multi-class object classification model to obtain an image classification output, the image classification output comprising one or more labels descriptive of the additional image data.