User center-of-mass and mass distribution extraction using depth images

1. A method for using a depth image to extract user behavior, comprising:

receiving a depth image that specifies that a plurality of pixels correspond to a user,

wherein the depth image is obtained using a capture device located a distance from the user,

wherein the depth image also specifies, for each of the pixels corresponding to the user, a pixel location and a pixel depth, and

wherein the pixel depth, specified for each of the pixels corresponding to the user, is indicative of a distance between the capture device and a portion of the user represented by the pixel;

determining, for each of the pixels corresponding to the user, a pixel mass that accounts for a distance between the portion of the user represented by the pixel and the capture device used to obtain the depth image; and

determining, based on the pixel mass determined for each of the pixels corresponding to the user, a depth-based center-of-mass position for the plurality of pixels corresponding to the user that accounts for distances between the portions of the user represented by the pixels and the capture device used to obtain the depth image.

2. The method of claim 1 , further comprising:

using the determined depth-based center-of-mass position to update an application.

3. The method of claim 1 , wherein:

the pixel depth for each pixel is specified by a depth value; and

the pixel mass, determined for each pixel corresponding to the user, is determined by squaring the depth value that specifies the pixel depth for the pixel.

4. The method of claim 1 , further comprising:

using the determined depth-based center-of-mass position to divide the plurality of pixels corresponding to the user into quadrants; and

determining a separate depth-based quadrant center-of-mass position for each of the quadrants.

5. The method of claim 4 , further comprising:

using one or more of the determined depth-based quadrant center-of-mass positions to update an application.

6. The method of claim 1 , further comprising:

determining a depth-based inertia tensor for the plurality of pixels corresponding to the user based on the determined depth-based center-of-mass position for the plurality of pixels corresponding to the user.

7. The method of claim 6 , further comprising:

scaling the determined depth-based inertia tensor using an assumption that the plurality of pixels corresponding to the user has a predetermined mass; and

using the scaled depth-based inertia tensor to update an application.

8. The method of claim 6 , further comprising:

using the determined depth-based center-of-mass position and the determined depth-based inertia tensor to divide the plurality of pixels corresponding to the user into quadrants;

determining a separate depth-based quadrant center-of-mass position for each of the quadrants; and

using one or more of the determined depth-based quadrant center-of-mass positions to update an application.

9. The method of claim 8 , wherein the using the determined depth-based center-of-mass position and the determined depth-based inertia tensor to divide the plurality of pixels corresponding to the user into quadrants, comprises:

determining principal axes for the plurality of pixels corresponding to the user based on the determined depth-based inertia tensor, or a scaled version thereof; and

using one of the determined principal axes, and a line perpendicular to the one of the determined principal axes that intersects the determined depth-based center-of-mass position, to divide the plurality of pixels corresponding to the user into the quadrants.

10. The method of claim 6 , further comprising:

using the determined depth-based center-of-mass position and the determined depth-based inertia tensor to divide the plurality of pixels corresponding to the user into quadrants;

determining a separate depth-based quadrant center-of-mass position for each of the quadrants;

determining a separate depth-based quadrant inertia tensor for each of the quadrants; and

using one or more of the determined depth-based quadrant inertia tensors to update an application.

11. A system that uses depth images to extract user behavior, comprising:

a capture device that obtains depth images;

a communication interface that receives depth images from the capture device;

one or more storage devices that store depth images;

a display interface; and

one or more processors in communication with the one or more storage devices and the display interface, wherein the one or more processors are configured to determine, for each of a plurality of depth images

a depth-based center-of-mass position for a plurality of pixels of the depth image that correspond to a user, and

a depth-based inertia tensor for the plurality of pixels of the depth image that correspond to the user.

12. The system of 11 , wherein the depth-based center-of mass position and the depth-based inertia tensor are determined in a manner that accounts for distances between portions of the user represented by the pixels and the capture device used to obtain the depth image.

13. The system of claim 11 , wherein the one or more processors use one or more of the determined depth-based center-of mass position or the determined depth-based inertia tensor to update an application running on the system and provide signals on the display interface that indicate an update to the application.

14. The system of claim 11 , wherein the one or more processors are also configured to:

use a determined depth-based center-of-mass position and a determined depth-based inertia tensor to divide a plurality of pixels corresponding to the user into quadrants;

determine a separate depth-based quadrant center-of-mass position for each of the quadrants; and

use one or more of the determined depth-based quadrant center-of-mass positions to update an application running on the system and provide signals on the display interface that indicate an update to the application.

15. The system of claim 11 , wherein the one or more processors are also configured to:

use a determined depth-based center-of-mass position and a determined depth-based inertia tensor to divide a plurality of pixels corresponding to the user into quadrants;

determine a separate depth-based quadrant center-of-mass position for each of the quadrants; and

determine a separate depth-based quadrant inertia tensor for each of the quadrants; and

use one or more of the determined depth-based quadrant inertia tensors, or scaled versions thereof, to update an application running on the system and provide signals on the display interface that indicate an update to the application.

16. One or more processor readable storage devices having instructions encoded thereon which when executed cause one or more processors to perform a method for using depth images to extract user behavior, the method comprising:

receiving a depth image that specifies that a plurality of pixels correspond to a user;

determining pixel masses for the pixels corresponding to the user;

determining, based on the determined pixel masses for the pixels corresponding to the user, a depth-based center-of-mass position for the plurality of pixels corresponding to the user; and

determining a depth-based inertia tensor for the plurality of pixels corresponding to the user based on the determined depth-based center-of-mass position for the plurality of pixels corresponding to the user.

17. The one or more processor readable storage devices of claim 16 , wherein:

the determined pixel masses account for distances between the portions of the user represented by the pixels and a capture device used to obtain the depth image; and

the depth-based center-of-mass position accounts for distances between portions of the user represented by the pixels and the capture device used to obtain the depth image.

18. The one or more processor readable storage devices of claim 16 , wherein the method further comprises:

using at least one of the depth-based center-of-mass position, the depth-based inertia tensor, or a scaled version of the depth-based inertia tensor to update an application.

19. The one or more processor readable storage device of claim 16 , wherein the method further comprises:

using the determined depth-based center-of-mass position and the determined depth-based inertia tensor to divide the plurality of pixels corresponding to the user into quadrants;

determining a separate depth-based quadrant center-of-mass position for each of the quadrants; and

using one or more of the determined depth-based quadrant center-of-mass positions to update an application.