IP Library Granted Patent US 10,733,431
Granted Patent B2
US 10,733,431 · App. 16/236,974 · Granted Aug 4, 2020

Systems and methods for optimizing pose estimation

Inventors: Peizhao Zhang (Fremont, CA); Peter Vajda (Palo Alto, CA); Kevin Matzen (Seattle, WA); Ross Girshick (Seattle, WA)
Assignee: Facebook, Inc.
G06K9/00369G06F17/18G06N3/0454G06N3/084G06N7/00G06N20/00G06T7/75G06N5/022G06T2210/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,733,431
App. No.
16/236,974
Granted
Aug 4, 2020
Kind
B2
Abstract

In one embodiment, a system may access first, second, and third probability models that are respectively associated with predetermined first and second body parts and a predetermined segment connecting the first and second body parts. Each model includes probability values associated with regions in an image, with each value representing the probability of the associated region containing the associated body part or segment. The system may select a first and second region based on the first probability model and a third region based on the second probability model. Based on the third probability model, the system may compute a first probability score for regions connecting the first and third regions and a second probability score for regions connecting the second and third regions. Based on the first and second probability scores, the system may select the first region to indicate where the predetermined first body part appears in the image.

Claims (68)

1. A method comprising, by a computing system:

accessing a first probability model associated with an image depicting a body, wherein the first probability model comprises first probability values associated with regions of the image, wherein each of the first probability values represents a probability of the associated region of the image containing a predetermined first body part;

accessing a second probability model associated with the image, wherein the second probability model comprises second probability values associated with the regions of the image, wherein each of the second probability values represents a probability of the associated region of the image containing a predetermined second body part;

accessing a third probability model associated with the image, wherein the third probability model comprises third probability values associated with the regions of the image, wherein each of the third probability values represents a probability of the associated region of the image containing a predetermined segment connecting the predetermined first body part and the predetermined second body part;

selecting a first region and a second region from the regions of the image based on the first probability model;

selecting a third region from the regions of the image based on the second probability model;

computing, based on the third probability model, a first probability score for regions connecting the first region and the third region;

computing, based on the third probability model, a second probability score for regions connecting the second region and the third region; and

selecting, based at least one the first probability score and the second probability score, the first region to indicate where the predetermined first body part of the body appears in the image.

2. The method of claim 1 , further comprising:

generating a regional definition encompassing the depicted body in the image; and

generating the first probability model, the second probability model, and the third probability model based on the regional definition encompassing the depicted body.

3. The method of claim 1 , further comprising:

selecting a fourth region from the regions of the image based on the second probability model;

computing, based on the third probability model, a third probability score for regions connecting the first region and the fourth region; and

computing, based on the third probability model, a fourth probability score for regions connecting the second region and the fourth region;

wherein the selection of the first region to indicate where the predetermined first body part of the body appears in the image is further based one the third probability score and the fourth probability score.

4. The method of claim 1 , further comprising:

computing, based on at least one additional probability model associated with at least one additional predetermined segment connecting different predetermined body parts, at least one additional probability score;

wherein the selection of the first region to indicate where the predetermined first body part of the body appears in the image is further based one the at least one additional probability score.

5. The method of claim 4 , wherein the different predetermined body parts connected by the at least one additional predetermined segment are different from the predetermined first body part and the predetermined second body part.

6. The method of claim 1 , wherein each of the first region and the second region has an associated first probability value that is a local maximum in the first probability model.

7. The method of claim 1 , wherein the first probability score for the regions connecting the first region and the third region is computed based on the third probability values that are associated with those regions connecting the first region and the third region.

8. The method of claim 1 , wherein the predetermined first body part and the predetermined second body part are directly connected in a body-pose model.

9. The method of claim 1 , wherein the predetermined first body part and the predetermined second body part represent joints.

10. The method of claim 9 , wherein the predetermined segment represents one or more bones connecting the joints.

11. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

access a first probability model associated with an image depicting a body, wherein the first probability model comprises first probability values associated with regions of the image, wherein each of the first probability values represents a probability of the associated region of the image containing a predetermined first body part;

access a second probability model associated with the image, wherein the second probability model comprises second probability values associated with the regions of the image, wherein each of the second probability values represents a probability of the associated region of the image containing a predetermined second body part;

access a third probability model associated with the image, wherein the third probability model comprises third probability values associated with the regions of the image, wherein each of the third probability values represents a probability of the associated region of the image containing a predetermined segment connecting the predetermined first body part and the predetermined second body part;

select a first region and a second region from the regions of the image based on the first probability model;

select a third region from the regions of the image based on the second probability model;

compute, based on the third probability model, a first probability score for regions connecting the first region and the third region;

compute, based on the third probability model, a second probability score for regions connecting the second region and the third region; and

select, based at least one the first probability score and the second probability score, the first region to indicate where the predetermined first body part of the body appears in the image.

12. The media of claim 11 , wherein the software is further operable when executed to:

generate a regional definition encompassing the depicted body in the image; and

generate the first probability model, the second probability model, and the third probability model based on the regional definition encompassing the depicted body.

13. The media of claim 11 , wherein the software is further operable when executed to:

select a fourth region from the regions of the image based on the second probability model;

compute, based on the third probability model, a third probability score for regions connecting the first region and the fourth region; and

compute, based on the third probability model, a fourth probability score for regions connecting the second region and the fourth region;

wherein the selection of the first region to indicate where the predetermined first body part of the body appears in the image is further based one the third probability score and the fourth probability score.

14. The media of claim 11 , wherein the software is further operable when executed to:

compute, based on at least one additional probability model associated with at least one additional predetermined segment connecting different predetermined body parts, at least one additional probability score;

wherein the selection of the first region to indicate where the predetermined first body part of the body appears in the image is further based one the at least one additional probability score.

15. The media of claim 14 , wherein the different predetermined body parts connected by the at least one additional predetermined segment are different from the predetermined first body part and the predetermined second body part.

16. A system comprising: one or more processors; and one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:

access a first probability model associated with an image depicting a body, wherein the first probability model comprises first probability values associated with regions of the image, wherein each of the first probability values represents a probability of the associated region of the image containing a predetermined first body part;

access a second probability model associated with the image, wherein the second probability model comprises second probability values associated with the regions of the image, wherein each of the second probability values represents a probability of the associated region of the image containing a predetermined second body part;

access a third probability model associated with the image, wherein the third probability model comprises third probability values associated with the regions of the image, wherein each of the third probability values represents a probability of the associated region of the image containing a predetermined segment connecting the predetermined first body part and the predetermined second body part;

select a first region and a second region from the regions of the image based on the first probability model;

select a third region from the regions of the image based on the second probability model;

compute, based on the third probability model, a first probability score for regions connecting the first region and the third region;

compute, based on the third probability model, a second probability score for regions connecting the second region and the third region; and

select, based at least one the first probability score and the second probability score, the first region to indicate where the predetermined first body part of the body appears in the image.

17. The system of claim 16 , wherein the processors are further operable when executing the instructions to:

generate a regional definition encompassing the depicted body in the image; and

generate the first probability model, the second probability model, and the third probability model based on the regional definition encompassing the depicted body.

18. The system of claim 16 , wherein the processors are further operable when executing the instructions to:

select a fourth region from the regions of the image based on the second probability model;

compute, based on the third probability model, a third probability score for regions connecting the first region and the fourth region; and

compute, based on the third probability model, a fourth probability score for regions connecting the second region and the fourth region;

wherein the selection of the first region to indicate where the predetermined first body part of the body appears in the image is further based one the third probability score and the fourth probability score.

19. The system of claim 16 , wherein the processors are further operable when executing the instructions to:

compute, based on at least one additional probability model associated with at least one additional predetermined segment connecting different predetermined body parts, at least one additional probability score;

wherein the selection of the first region to indicate where the predetermined first body part of the body appears in the image is further based one the at least one additional probability score.

20. The system of claim 19 , wherein the different predetermined body parts connected by the at least one additional predetermined segment are different from the predetermined first body part and the predetermined second body part.

Assignments (2)
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058553/0802 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 7, 2019
From: ZHANG, PEIZHAO; VAJDA, PETER; MATZEN, KEVIN; GIRSHICK, ROSS
To: FACEBOOK, INC.
Reel/Frame 047921/0907 →
Cited By (2)
US 12,326,971 US 12,333,778