System and method to detect articulate body pose
A system for detecting an articulate body pose from an imagery content includes an imaging module for capturing the imagery content, and a processor that is operable to obtain a top-down view of the imagery content, and process the top-down view to detect the articulate body pose using a machine learning algorithm, wherein the articulate body pose includes a plurality of joints. The processing includes creating a part confidence map corresponding to each joint of the articulate body pose, generating a heatmap by projecting the part confidence map on the top-down view of the imagery content, creating a part affinity map corresponding to each body part, generating a vector map by projecting the part affinity map on the top-down view of the imagery content, and generating a body-framework corresponding to the articulate body pose, using the heatmap and the vector map.
1. A system to detect an articulate body pose that includes a plurality of joints from an imagery content, the system comprising:
an imaging device disposed at a height to focus on a vertical line of sight for capturing a top-down view of the imagery content; and
a processor communicatively coupled to the imaging device and a memory communicatively coupled to the imaging device and the processor, wherein the processor is operable to:
obtain the top down view of the imagery content from the imaging device; and
process the top-down view of the imagery content co-operatively with a machine learning algorithm, trained in tandem with the process, by:
creating a part confidence map, for a plurality of sigma values, corresponding to each joint of the articulate body pose;
creating a part affinity map, for a plurality of threshold values, corresponding to each body part associated with the each joint of the articulate body pose;
receiving a defined part confidence map and a defined part affinity map from the memory;
comparing the part confidence map for the plurality of sigma values with the defined part confidence map to select a true sigma value from the plurality of sigma values;
comparing the part affinity map for the plurality of threshold values with the defined part affinity map to select a true threshold value from the plurality of threshold values;
generating a heatmap by projecting the part confidence map of the true sigma value on the top-down view of the imagery content;
generating a vector map by projecting the part affinity map of the true threshold value on the top-down view of the imagery content; and
generating a body-framework comprising skeleton structures corresponding to the articulate body pose, using the heatmap and the vector map, to detect the articulate body pose.
2. The system of claim 1 , wherein the imaging device forms part of an imaging module comprising a processor and a memory.
3. The system of claim 1 , wherein the imagery content comprises at least one of: image, video and graphics interchange format (GIF) based content.
4. The system of claim 1 , wherein the training of the machine learning algorithm includes:
generating ground truth skeletons using a plurality of datasets including image content similar to the imagery content;
comparing inferred skeletons of the body-framework with the ground truth skeletons; and
determining a number of matched joint points and a number of matched skeletons based on the comparison of the inferred skeletons with the ground truth skeletons.
5. The system of claim 1 , wherein the system comprises lowering a resolution of the top-down view of the imagery content prior to processing thereof.
6. The system of claim 1 , wherein obtaining the top-down view comprise one of:
receiving the imagery content in the form of the top-down view; or
processing a plurality of views of the imagery content to obtain the top-down view therefrom.
7. The system of claim 6 , wherein the plurality of views of the imagery content comprises: rear view, front view, top view, bottom view, left-hand side view, right-hand side view, and perspective view.
8. The system of claim 1 , wherein the system further comprises a display device for viewing the detected articulate body pose.
9. A method for detecting an articulate body pose that includes a plurality of joints from imagery content, the method comprising:
capturing a top-down view of the imagery content using an imaging device, the imaging device disposed at a height to focus on a vertical line of sight when capturing the top-down view of the imagery content;
obtaining the top-down view of the imagery content; and
processing the top-down view of the imagery content co-operatively with a machine learning algorithm, trained in tandem with the process, by:
creating a part confidence map, for a plurality of sigma values, corresponding to each joint of the articulate body pose;
creating a part affinity map, for a plurality of threshold values, corresponding to each body part associated with the each joint of the articulate body pose;
receiving a defined part confidence map and a defined part affinity map from a memory communicatively coupled with the imaging device;
comparing the part confidence map for the plurality of sigma values with the defined part confidence map to select a true sigma value from the plurality of sigma values;
comparing the part affinity map for the plurality of threshold values with the defined part affinity map to select a true threshold value from the plurality of threshold values;
generating a heatmap by projecting the part confidence map of the true sigma value on the top-down view of the imagery content;
generating a vector map of the true threshold value by projecting the part affinity map on the top-down view of the imagery content; and
generating a body-framework comprising skeleton structures corresponding to the articulate body pose, using the heatmap and the vector map, to detect the articulate body pose.
10. The method of claim 9 , wherein the method comprises employing a convolutional neural network to create the part confidence map and the part affinity map.
11. The method of claim 9 , wherein the method further comprises training of the machine learning algorithm by:
generating ground truth skeletons using a plurality of datasets including image content similar to the imagery content;
comparing inferred skeletons of the body-framework with the ground truth skeletons; and
determining a number of matched joint points and a number of matched skeletons based on the comparison of the inferred skeletons with the ground truth skeletons.
12. The method of claim 9 , wherein the method comprises lowering a resolution of the top-down view of the imagery content prior to processing thereof.
13. The method of claim 9 , wherein obtaining the top-down view comprise one of:
receiving the imagery content in the form of the top-down view; or
processing a plurality of views of the imagery content to obtain the top-down view therefrom.
14. The method of claim 13 , wherein the plurality of views of the imagery content comprises: rear view, front view, top view, bottom view, left-hand side view, right-hand side view, and perspective view.
15. A processing circuit for detecting an articulate body pose that includes a plurality of joints from an imagery content, the processing circuit configured to:
capture a top-down view of the imagery content using an imaging device, the imaging device disposed at a height to focus on a vertical line of sight when capturing the top-down view of the imagery content;
obtain the top-down view of the imagery content; and
process the top-down view of the imagery content co-operatively with a machine learning algorithm, trained in tandem with the process, by:
creating a part confidence map, for a plurality of sigma values, corresponding to each joint of the articulate body pose;
creating a part affinity map, for a plurality of threshold values, corresponding to each body part associated with the each joint of the articulate body pose;
receiving a defined part confidence map and a defined part affinity map from a memory communicatively coupled with the imaging device;
comparing the part confidence map for the plurality of sigma values with the defined part confidence map to select a true sigma value from the plurality of sigma values;
comparing the part affinity map for the plurality of threshold values with the defined part affinity map to select a true threshold value from the plurality of threshold values;
generating a heatmap by projecting the part confidence map of the true sigma value on the top-down view of the imagery content;
generating a vector map of the true threshold value by projecting the part affinity map on the top-down view of the imagery content; and
generating a body-framework comprising skeleton structures corresponding to the articulate body pose, using the heatmap and the vector map, to detect the articulate body pose.
16. The processing circuit of claim 15 , wherein a convolutional neural network is employed to create the part confidence map and the part affinity map.
17. The processing circuit of claim 15 , further configured to:
generate ground truth skeletons using a plurality of datasets including image content similar to the imagery content;
compare inferred skeletons of the body-framework with the ground truth skeletons; and
determine a number of matched joint points and a number of matched skeletons based on the comparison of the inferred skeletons with the ground truth skeletons.
18. The processing circuit of claim 15 , further configured to:
receive the imagery content in the form of the top-down view; or
process a plurality of views of the imagery content to obtain the top-down view therefrom.
19. The processing circuit of claim 18 , wherein the plurality of views of the imagery content comprises: rear view, front view, top view, bottom view, left-hand side view, right-hand side view, and perspective view.