Information processing apparatus, information processing method, and storage medium

1. An apparatus comprising:

one or more processors; and

one or more memories that store a computer-readable instruction configured to be executed by the one or more processors, thereby the computer-readable instruction causing the apparatus to:

acquire position information indicating positions of joints of a plurality of objects in image data;

acquire a score map in which a feature for identifying each of the plurality of objects is converted into a numerical value, the score map being output by a pre-trained model in response to input of the image data;

acquire, using the score map, a first evaluation score based on scores of a first joint at a first position, a second joint at a second position and a plurality of pixels between the first and second joints in the image data, and acquire, using the score map, a second evaluation score based on scores of the first joint at the first position, a third joint at a third position and a plurality of pixels between the first and third joints in the image data; and

determine which of the second joint and third joint belongs to an object to which the first joint belongs first and second evaluation scores.

2. The apparatus according to claim 1 , wherein, for each of the plurality of objects, positions of joints are identified based on scores output to the positions of the joints indicated by the position information in the score map.

3. The apparatus according to claim 2 , wherein, in the score map, in a case where dispersion of scores output to respective pixels on a line segment connecting a pair of joints determined based on types of the joints is less than a threshold, the pair of joints are identified as joints belonging to a same object, and in a case where the dispersion is more than or equal to the threshold, the pair of joints are identified as joints each belonging to a different object.

4. The apparatus according to claim 2 , wherein, in the score map, in a case where a difference between scores output to positions of a pair of joints determined based on types of the joints is less than a threshold, the pair of joints are identified as joints belonging to a same object, and in a case where a difference between scores output to positions of a pair of joints is more than or equal to a predetermined threshold, the pair of joints are identified as joints each belonging to a different object.

5. The apparatus according to claim 1 , wherein the pre-trained model is a model trained by updating an interlayer connection weighting coefficient of the model to decrease a loss value, by using a loss function that outputs the loss value which increases in a case where a difference between a score corresponding to a joint belonging to a first object and a score corresponding to a joint belonging to an object different from the first object is smaller than a predetermined threshold, the scores being output by the pre-trained model based on the position information.

6. The apparatus according to claim 1 , wherein the pre-trained model is a model trained by updating an interlayer connection weighting coefficient of the model to decrease a loss value, by using a loss function that outputs the loss value which increases in a case where dispersion of scores corresponding to a joint group belonging to a same object is greater than a threshold, for the scores in the same object.

7. The apparatus according to claim 1 , wherein the pre-trained model is a model trained by updating an interlayer connection weighting coefficient of the model to decrease a loss value, by using a loss function that outputs the loss value which increases with decrease in a distance between a first object and an object different from the first object, based on the distance.

8. The apparatus according to claim 1 ,

wherein, for each type of joints in the image data, the position information indicates positions of joints belonging to each of the plurality of object and a positional relationship between connectable joints of different types, and

wherein, for each of the plurality of objects, positions of joints are identified based on scores acquired from the score map for a pair of joints in a positional relationship of connectable joints that is determined based on the position information.

9. The apparatus according to claim 1 , wherein a joint map indicating a position for each type of the joints are acquired.

10. The apparatus according to claim 1 , further configured to recognize a posture of each object, based on the identified positions of joints belonging to each of the plurality of objects.

11. The apparatus according to claim 1 , further configured to update a weighting parameter of the pre-trained model.

12. The apparatus according to claim 11 , wherein an interlayer connection weighting coefficient of the model is updated to decrease a loss value, by using a loss function that outputs the loss value which increases in a case where a difference between a score corresponding to a joint belonging to a first object and a score corresponding to a joint belonging to an object different from the first object is smaller than a predetermined threshold, the scores being output by the pre-trained model based on the position information.

13. The apparatus according to claim 11 , wherein an interlayer connection weighting coefficient of the model is updated to decrease a loss value, by using a loss function that outputs the loss value which increases in a case where dispersion of scores corresponding to a joint group belonging to a same object is greater than a threshold, for the scores in the same object.

14. The apparatus according to claim 11 , wherein an interlayer connection weighting coefficient of the model is updated to decrease a loss value, by using a loss function that outputs the loss value which increases with decrease in a distance between a first object and an object different from the first object, based on the distance.

15. The apparatus according to claim 1 , wherein positions of joints belonging to each of the plurality of objects in the image data are acquired, for each type of joint based on the pre-trained model.

16. A non-transitory computer-readable storage medium storing a program that causes a computer to execute a method comprising:

acquiring position information positions of joints of a plurality of objects in image data;

acquiring a score map in which a feature for identifying each of the plurality of objects is converted into a numerical value, the score map being output by a pre-trained model in response to input of the image data;

acquiring, using the score map, a first evaluation score based on scores of a first joint at a first position, a second joint at a second position and a plurality of pixels between the first and second joints in the image data, and acquire, using the score map, a second evaluation score based on scores of the first joint at the first position, a third joint at a third position and a plurality of pixels between the first and third joints in the image data; and

determining which of the second joint and third joint belongs to an object to which the first joint belongs, based on the first and second evaluation scores.

17. The non-transitory computer-readable storage medium according to claim 16 , wherein identifying identifies positions of the joints belonging to each of the plurality of objects, based on a score output to the position of the joint indicated by the position information in the score map.

18. The non-transitory computer-readable storage medium according to claim 16 , further comprising recognizing a posture of each object, based on the identified positions of joints belonging to each of the plurality of objects.

19. The non-transitory computer-readable storage medium according to claim 16 , further comprising updating a weighting parameter of the pre-trained model.

20. A method comprising:

acquiring position information indicating positions of joints of a plurality of objects in image data;

acquiring a score map in which a feature for identifying each of the plurality of objects is converted into a numerical value, the score map being output by a pre-trained model in response to input of the image data;

acquiring, using the score map, a first evaluation score based on scores of a first joint at a first position, a second joint at a second position and a plurality of pixels between the first and second joints in the image data, and acquire, using the score map, a second evaluation score based on scores of the first joint at the first position, a third joint at a third position and a plurality of pixels between the first and third joints in the image data; and

determining which of the second joint and third joint belongs to an object to which the first joint belongs, based on the first and second evaluation scores.