Optimizations for dynamic object instance detection, segmentation, and structure mapping

1. A method comprising, by a computing system:

accessing a plurality of pose probability models for a plurality of predetermined parts of a body that is depicted in an image, respectively, wherein each of the plurality of pose probability models is configured for determining a probability of the associated predetermined part of the body being at a location in the image, wherein the plurality of pose probability models is generated by a machine-learning model;

determining a candidate pose that is defined by a set of coordinates representing candidate locations of the predetermined parts of the body in the image;

determining a first probability score for the candidate pose based on the plurality of pose probability models and the set of coordinates of the candidate pose;

generating a pose representation for the candidate pose using a transformation model and the candidate pose;

determining a second probability score for the pose representation based on a pose-representation probability model; and

selecting the candidate pose to represent a pose of the body depicted in the image based on at least the first probability score and the second probability score.

2. The method of claim 1 ,

wherein each coordinate in the set of coordinates of the candidate pose is defined in a first coordinate system of the image;

wherein the pose representation is defined in a first spatial dimension and is generated by applying the transformation model to a set of normalized coordinates that correspond to the set of coordinates of the candidate pose, respectively; and

wherein each coordinate in the set of normalized coordinates is defined in a second coordinate system that is different from the first coordinate system.

3. The method of claim 2 , further comprising:

reprojecting the pose representation from the first spatial dimension into a second spatial dimension associated with the second coordinate system; and

computing a reprojection error based on the reprojected pose representation and the normalized coordinates;

wherein the selection of the candidate pose is further based on the reprojection error.

4. The method of claim 2 , wherein the pose representation is generated based on differences between the set of normalized coordinates and an aggregate representation of a plurality of sets of normalized coordinates that are associated with a plurality of poses, respectively.

5. The method of claim 2 , wherein the second coordinate system is defined relative to one or more of the predetermined parts of the body.

6. The method of claim 1 , wherein each of the plurality of probability models is a probability heat map.

7. The method of claim 1 , wherein the transformation model is generated using principal component analysis.

8. The method of claim 1 , wherein at least one of the plurality of predetermined parts of the body corresponds to a joint of the body.

9. A system comprising: one or more processors and one or more computer-readable non-transitory storage media coupled to one or more of the processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by one or more of the processors to cause the system to perform operations comprising:

accessing a plurality of pose probability models for a plurality of predetermined parts of a body that is depicted in an image, respectively, wherein each of the plurality of pose probability models is configured for determining a probability of the associated predetermined part of the body being at a location in the image, wherein the plurality of pose probability models is generated by a machine-learning model;

determining a candidate pose that is defined by a set of coordinates representing candidate locations of the predetermined parts of the body in the image;

determining a first probability score for the candidate pose based on the plurality of pose probability models and the set of coordinates of the candidate pose;

generating a pose representation for the candidate pose using a transformation model and the candidate pose;

determining a second probability score for the pose representation based on a pose-representation probability model; and

selecting the candidate pose to represent a pose of the body depicted in the image based on at least the first probability score and the second probability score.

10. The system of claim 9 ,

wherein each coordinate in the set of coordinates of the candidate pose is defined in a first coordinate system of the image;

wherein the pose representation is defined in a first spatial dimension and is generated by applying the transformation model to a set of normalized coordinates that correspond to the set of coordinates of the candidate pose, respectively; and

wherein each coordinate in the set of normalized coordinates is defined in a second coordinate system that is different from the first coordinate system.

11. The system of claim 10 , wherein the processors are further operable when executing the instructions to perform operations comprising:

reprojecting the pose representation from the first spatial dimension into a second spatial dimension associated with the second coordinate system; and

computing a reprojection error based on the reprojected pose representation and the normalized coordinates;

wherein the selection of the candidate pose is further based on the reprojection error.

12. The system of claim 10 , wherein the pose representation is generated based on differences between the set of normalized coordinates and an aggregate representation of a plurality of sets of normalized coordinates that are associated with a plurality of poses, respectively.

13. The system of claim 10 , wherein the second coordinate system is defined relative to one or more of the predetermined parts of the body.

14. The system of claim 9 , wherein each of the plurality of probability models is a probability heat map.

15. One or more computer-readable non-transitory storage media embodying software that is operable when executed to cause one or more processors to perform operations comprising:

accessing a plurality of pose probability models for a plurality of predetermined parts of a body that is depicted in an image, respectively, wherein each of the plurality of pose probability models is configured for determining a probability of the associated predetermined part of the body being at a location in the image, wherein the plurality of pose probability models is generated by a machine-learning model;

determining a candidate pose that is defined by a set of coordinates representing candidate locations of the predetermined parts of the body in the image;

determining a first probability score for the candidate pose based on the plurality of pose probability models and the set of coordinates of the candidate pose;

generating a pose representation for the candidate pose using a transformation model and the candidate pose;

determining a second probability score for the pose representation based on a pose-representation probability model; and

selecting the candidate pose to represent a pose of the body depicted in the image based on at least the first probability score and the second probability score.

16. The media of claim 15 ,

wherein each coordinate in the set of coordinates of the candidate pose is defined in a first coordinate system of the image;

wherein the pose representation is defined in a first spatial dimension and is generated by applying the transformation model to a set of normalized coordinates that correspond to the set of coordinates of the candidate pose, respectively; and

wherein each coordinate in the set of normalized coordinates is defined in a second coordinate system that is different from the first coordinate system.

17. The media of claim 16 , wherein the software is further operable when executed to cause the one or more processors to perform operations comprising:

reprojecting the pose representation from the first spatial dimension into a second spatial dimension associated with the second coordinate system; and

computing a reprojection error based on the reprojected pose representation and the normalized coordinates;

wherein the selection of the candidate pose is further based on the reprojection error.

18. The media of claim 16 , wherein the pose representation is generated based on differences between the set of normalized coordinates and an aggregate representation of a plurality of sets of normalized coordinates that are associated with a plurality of poses, respectively.

19. The media of claim 16 , wherein the second coordinate system is defined relative to one or more of the predetermined parts of the body.

20. The media of claim 15 , wherein each of the plurality of probability models is a probability heat map.