Hierarchy-preserving learning for multi-label classification

1. A method comprising, by one or more computing systems:

accessing a set of content objects, wherein each content object of the set of content objects is pre-labeled with one or more concepts of a plurality of concepts, and wherein the plurality of concepts are organized according to a hierarchical relationship; and

training, by a machine-learning model, a classification model for classifying content objects within the set of content objects, wherein training the classification model comprises:

determining, for each object in the set of content objects, a plurality of classification values corresponding to the plurality of concepts, each classification value corresponding to a different layer in the hierarchical relationship;

calculating, for each object and based on a loss function, a loss for each of the plurality of classification values based on the pre-labeled concepts associated with the object;

training the classification model to predict, for any particular content object, a probability of a class higher in the hierarchical relationship that is less than or equal to a probability of a class lower in the hierarchical relationship by:

generating a hierarchical constraint loss function for the set of content objects by setting the loss for each classification value, for each object, to be the maximum of (1) the calculated loss for the classification value and corresponding pre-labeled concept and (2) a loss corresponding to a classification value in a higher, parent layer of the hierarchical relationship and a corresponding pre-labeled concept for that classification value; and

training the classification model using the generated hierarchical constraint loss function as the loss function for the classification model; and

updating the classification model based on the hierarchical constraint loss function until the maximum loss satisfies a predetermined criterion.

2. The method of claim 1 , wherein the calculated loss for each of the plurality of classification values comprises a first calculated loss for a parent concept of the plurality of concepts and a second calculated loss for a child concept of the plurality of concepts.

3. The method of claim 2 , wherein the hierarchical constraint loss function comprises one or more constraints configured to limit the second calculated loss to a value less than or equal to that of the first calculated loss.

4. The method of claim 2 , wherein the hierarchical constraint loss function is bounded around a 0-1 loss function.

5. The method of claim 2 , wherein training, by the machine-learning model, the classification model comprises training the classification model utilizing content objects of the set of content objects corresponding to the parent concept in accordance with the predetermined criteria prior to utilizing content objects of the set of content objects corresponding to the child concept in accordance with the predetermined criteria.

6. The method of claim 1 , further comprising:

prior to utilizing the hierarchical constraint loss function to calculate the maximum loss, deriving the hierarchical constraint loss function based on one or more base loss functions.

7. The method of claim 6 , wherein, subsequent to deriving the hierarchical constraint loss function, the method further comprising:

sorting the respective losses for each of the plurality of classification values in order of increasing loss value; and

selecting an initial K concepts corresponding to an initial K losses of the sorted respective losses, such that a cumulative sum of the initial K losses is greater than a sum of a predetermined threshold loss value and 1-K, wherein K is a hyperparameter.

8. One or more computing devices, comprising:

one or more non-transitory computer-readable storage media including instructions; and

one or more processors coupled to the one or more storage media, the one or more processors configured to execute the instructions to:

access a set of content objects, wherein each content object is pre-labeled with one or more concepts of a plurality of concepts, and wherein the plurality of concepts are organized according to a hierarchical relationship; and

train, by a machine-learning model, a classification model for classifying content objects within the set of content objects, wherein training the classification model comprises:

determine, for each object in the set of content objects, a plurality of classification values corresponding to the plurality of concepts, each classification value corresponding to a different layer in the hierarchical relationship;

calculate for each object and based on a loss function, a loss for each of the plurality of classification values based on the pre-labeled concepts associated with the object;

train the classification model to predict, for any particular content object, a probability of a class higher in the hierarchical relationship that is less than or equal to a probability of a class lower in the hierarchical relationship by:

generating a hierarchical constraint loss function for the set of content objects by setting the loss for each classification value, for each object, to be the maximum of (1) the calculated loss for the classification value and corresponding pre-labeled concept and (2) a loss corresponding to a classification value in a higher, parent layer of the hierarchical relationship and a corresponding pre-labeled concept for that classification value; and

training the classification model using the generated hierarchical constraint loss function as the loss function for the classification model; and

update the classification model based on the hierarchical constraint loss function until the maximum loss satisfies a predetermined criterion.

9. The one or more computing devices of claim 8 , wherein the calculated loss for each of the plurality of classification values comprises a first calculated loss for a parent concept of the plurality of concepts and a second calculated loss for a child concept of the plurality of concepts.

10. The one or more computing devices of claim 9 , wherein the hierarchical constraint loss function comprises one or more constraints configured to limit the second calculated loss to a value less than or equal to that of the first calculated loss.

11. The one or more computing devices of claim 9 , wherein the hierarchical constraint loss function is bounded around a 0-1 loss function.

12. The one or more computing devices of claim 9 , wherein the instructions to train, by the machine-learning model, the classification model further comprises instructions to train the classification model utilizing content objects of the set of content objects corresponding to the parent concept in accordance with the predetermined criteria prior to utilizing content objects of the set of content objects corresponding to the child concept in accordance with the predetermined criteria.

13. The one or more computing devices of claim 8 , wherein, prior to utilizing the hierarchical constraint loss function to calculate the maximum loss, the instructions further comprises instructions to:

derive the hierarchical constraint loss function based on one or more base loss functions.

14. The one or more computing devices of claim 13 , wherein, subsequent to deriving the hierarchical constraint loss function, the instructions further comprises instructions to:

sort the respective losses for each of the plurality of classification values in order of increasing loss value; and

select an initial K concepts corresponding to an initial K losses of the sorted respective losses, such that a cumulative sum of the initial K losses is greater than a sum of a predetermined threshold loss value and 1−K, wherein K is a hyperparameter.

15. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of one or more computing devices, cause the one or more processors to:

access a set of content objects, wherein each content object is pre-labeled with one or more concepts of a plurality of concepts, and wherein the plurality of concepts are organized according to a hierarchical relationship; and

train, by a machine-learning model, a classification model for classifying content objects within the set of content objects, wherein training the classification model comprises:

determine, for each object in the set of content objects, a plurality of classification values corresponding to the plurality of concepts, each classification value corresponding to a different layer in the hierarchical relationship;

calculate for each object and based on a loss function, a loss for each of the plurality of classification values based on the pre-labeled concepts associated with the object;

train the classification model to predict, for any particular content object, a probability of a class higher in the hierarchical relationship that is less than or equal to a probability of a class lower in the hierarchical relationship by:

generating a hierarchical constraint loss function for the set of content objects by setting the loss for each classification value, for each object, to be the maximum of (1) the calculated loss for the classification value and corresponding pre-labeled concept and (2) a loss corresponding to a classification value in a higher, parent layer of the hierarchical relationship and a corresponding pre-labeled concept for that classification value; and

training the classification model using the generated hierarchical constraint loss function as the loss function for the classification model; and

update the classification model based on the hierarchical constraint loss function until the maximum loss satisfies a predetermined criterion.

16. The non-transitory computer-readable medium of claim 15 , wherein the calculated loss for each of the plurality of classification values comprises a first calculated loss for a parent concept of the plurality of concepts and a second calculated loss for a child concept of the plurality of concepts.

17. The non-transitory computer-readable medium of claim 16 , wherein the hierarchical constraint loss function comprises one or more constraints configured to limit the second calculated loss to a value less than or equal to that of the first calculated loss.

18. The non-transitory computer-readable medium of claim 16 , wherein the hierarchical constraint loss function is bounded around a 0-1 loss function.

19. The non-transitory computer-readable medium of claim 15 , wherein, prior to utilizing the hierarchical constraint loss function to calculate the maximum loss, the instructions further comprises instructions to:

derive the hierarchical constraint loss function based on one or more base loss functions.

20. The non-transitory computer-readable medium of claim 19 , wherein, subsequent to deriving the hierarchical constraint loss function, the instructions further comprises instructions to:

sort the respective losses for each of the plurality of classification values in order of increasing loss value; and

select an initial K concepts corresponding to an initial K losses of the sorted respective losses, such that a cumulative sum of the initial K losses is greater than a sum of a predetermined threshold loss value and 1−K, wherein K is a hyperparameter.