Body pose tracking system and method

1. A computer-implemented method, comprising:

acquiring an image of a body including a plurality of limb segments, where the body is configured in a pose and a plurality of marker bands are positioned around the limb segments;

acquiring depth information corresponding to distances to regions in the image;

determining two-dimensional (2D) locations in the image corresponding to visible portions of the marker bands;

determining, from the 2D locations and the depth information, three-dimensional (3D) locations of the visible portions of the marker bands;

fitting virtual bands to the 3D locations of the visible portions of the marker bands to generate 3D positions of the virtual bands;

determining the pose by fitting a skeletal model of the body to the 3D positions of the virtual bands; and

outputting the pose.

2. The computer-implemented method of claim 1 , where:

each marker band includes a plurality of light emitters;

the visible portions of the marker bands include one or more visible light emitters; and

said determining 2D locations includes determining regions of higher light intensity associated with the visible light emitters.

3. The computer-implemented method of claim 2 , where, for each marker band, the light emitters are light emitting diodes (LEDs) having a same color.

4. The computer-implemented method of claim 1 , where:

each marker band includes light reflective material;

the visible portions of the marker bands include a visible portion of the light reflective material; and

said determining 2D locations includes determining regions of higher light intensity associated with the visible portions of the reflective light material.

5. The computer-implemented method of claim 1 , where:

the skeletal model of the body is an inverse kinematic model of the body including at least a plurality of limb segments and a plurality of joints;

the 3D positions of the virtual bands define a center position of each virtual band;

each limb segment of the inverse kinematic model includes a central axis; and

said determining the pose includes iteratively adjusting the inverse kinematic model to align the central axes of the limb segments with the center positions of the virtual bands.

6. The computer-implemented method of claim 1 , where:

the skeletal model of the body is an inverse kinematic model of the body including at least a plurality of limb segments and a plurality of joints;

the 3D positions of the virtual bands define a perimeter of each virtual band;

each limb segment of the inverse kinematic model includes a surface mesh; and

said determining the pose includes iteratively adjusting the inverse kinematic model to align the surface mesh of the limb segments with the perimeters of the virtual bands.

7. The computer-implemented method of claim 1 , where the skeletal model of the body is an inverse kinematic model of the body including at least a plurality of limb segments and a plurality of joints, and the method further comprises:

when said determining the pose generates a partial solution of the pose:

inputting the partial solution of the pose, the image, the depth information, camera information associated with the image, and limb segment data to an artificial neural network (ANN) model;

executing the ANN model to generate a final solution of the pose; and

outputting the final solution of the pose,

where a virtual body includes at least a plurality of virtual limb segments and a plurality of virtual joints,

where a plurality of virtual marker bands are positioned around the virtual limb segments,

where the ANN model is trained under supervised learning using a plurality of virtual poses, each virtual pose including:

3D positions and angles of each virtual joint,

3D positions of each virtual limb segment,

3D positions of visible portions of each virtual marker band, and

a red green blue depth (RGB-D) image of the virtual body for each virtual camera position, the RGB-D image including red pixels, green pixels, blue pixels, and depth pixels.

8. The computer-implemented method of claim 1 , where the skeletal model of the body is an inverse kinematic model of the body including at least a plurality of limb segments and a plurality of joints, and the method further comprises:

when said determining the pose generates a partial solution of the pose:

transmitting, to a server via a wireless network, the partial solution of the pose, the image, the depth information, camera information associated with the image, and limb segment data;

inputting, at the server, the partial solution of the pose, the image, the depth information, the camera information associated with the image, and the limb segment data to an artificial neural network (ANN) model, and

executing, at the server, the ANN model to generate a final solution of the pose;

receiving, from the server via the wireless network, the final solution of the pose; and

outputting the final solution of the pose.

9. A system, comprising:

a plurality of marker bands; and

an apparatus, including:

one or more cameras configured to:

acquire an image of a body including a plurality of limb segments, where the body is configured in a pose and the plurality of marker bands are positioned around the limb segments, and

acquire depth information corresponding to distances to regions in the image;

a memory configured to store the image, the depth information and a skeletal model of the body; and

a processor, coupled to the memory, configured to:

determine two-dimensional (2D) locations in the image corresponding to visible portions of the marker bands,

determine, from the 2D locations and the depth information, three-dimensional (3D) locations of the visible portions of the marker bands,

fit virtual bands to the 3D locations of the visible portions of the marker bands to generate 3D positions of the virtual bands,

determine the pose by fitting the skeletal model of the body to the 3D positions of the virtual bands, and

output the pose.

10. The system of claim 9 , where:

each marker band includes a plurality of light emitters;

the visible portions of the marker bands include one or more visible light emitters; and

said determining 2D locations includes determining regions of higher light intensity associated with the visible light emitters.

11. The system of claim 10 , where:

the light emitters are powered by one or more batteries;

the light emitters are light emitting diodes (LEDs); and

each marker band includes LEDs having a same color.

12. The system of claim 9 , where:

each marker band includes light reflective material;

the visible portions of the marker bands include a visible portion of the light reflective material; and

said determining 2D locations includes determining regions of higher light intensity associated with the visible portions of the reflective light material.

13. The system of claim 9 , where:

the apparatus is a handheld electronic device; and

one of the cameras is a depth camera configured to acquire the depth information, or

the depth information is acquired from respective concurrent images from two or more cameras.

14. The system of claim 9 , where:

the skeletal model of the body is an inverse kinematic model of the body including at least a plurality of limb segments and a plurality of joints;

the 3D positions of the virtual bands define a center position of each virtual band;

each limb segment of the inverse kinematic model includes a central axis; and

said determine the pose includes iteratively adjusting the inverse kinematic model to align the central axes of the limb segments with the center positions of the virtual bands.

15. The system of claim 9 , where:

the skeletal model of the body is an inverse kinematic model of the body including at least a plurality of limb segments and a plurality of joints;

the 3D positions of the virtual bands define a perimeter of each virtual band;

each limb segment of the inverse kinematic model includes a surface mesh; and

said determine the pose includes iteratively adjusting the inverse kinematic model to align the surface mesh of the limb segments with the perimeters of the virtual bands.

16. The system of claim 9 , where:

the skeletal model of the body is an inverse kinematic model of the body including at least a plurality of limb segments and a plurality of joints;

when said determine the pose generates a partial solution of the pose, the processor is further configured to:

input a data package to an artificial neural network (ANN) model, the data package including the partial solution of the pose, the image, the depth information, camera information associated with the image, and limb segment data,

execute the ANN model to generate a final solution of the pose, and

output the final solution of the pose;

a virtual body includes at least a plurality of virtual limb segments and a plurality of virtual joints;

a plurality of virtual marker bands are positioned around the virtual limb segments; and

the ANN model is trained under supervised learning using a plurality of virtual poses, each virtual pose including:

3D positions and angles of each virtual joint,

3D positions of each virtual limb segment,

3D positions of visible portions of each virtual marker band,

a red green blue depth (RGB-D) image of the virtual body for each virtual camera position, the RGB-D image including red pixels, green pixels, blue pixels, and depth pixels.

17. The system of claim 16 , where the processor is further configured to input at least one of a previous data package or a previous final solution of the pose to the ANN model.

18. The system of claim 9 , further comprising:

a server coupled to the apparatus via a wireless network,

where, when said determine the pose generates a partial solution of the pose, the processor is further configured to:

transmit, to the server, the partial solution of the pose, the image, the depth information, camera information associated with the image, and limb segment data,

receive, from the server, a final solution of the pose, and

output the final solution of the pose, and

where the server includes a processor configured to:

input the partial solution of the pose, the image, the depth information, the camera information associated with the image, and the limb segment data to an artificial neural network (ANN) model, and

execute the ANN model to generate the final solution of the pose.

19. The system of claim 9 , where the apparatus includes:

one camera to acquire the image; and

a depth sensor, co-located with the camera, to acquire the depth information,

where the image is a red green blue (RGB) image including red pixels, green pixels and blue pixels, and

where the depth information corresponds to the red pixels, the green pixels and the blue pixels.

20. The system of claim 9 , where each marker band is adjustable to match a diameter of a respective limb.