Systems and methods for estimating a non-visible or occluded body part
A depth image of a scene may be received, observed, or captured by a device. The depth image may include a human target that may have, for example, a portion thereof non-visible or occluded. For example, a user may be turned such that a body part may not be visible to the device, may have one or more body parts partially outside a field of view of the device, may have a body part or a portion of a body part behind another body part or object, or the like such that the human target associated with the user may also have a portion body part or a body part non-visible or occluded in the depth image. A position or location of the non-visible or occluded portion or body part of the human target associated with the user may then be estimated.
1. A method comprising:
receiving a depth image of a scene;
determining that a portion of a human target in the depth image is non-visible;
estimating a position of the non-visible portion of the human target, wherein the estimating comprises:
generating an initial estimation of the position of the non-visible portion of the human target based at least in part on force vectors associated with a visible portion of the human target; and
modifying the initial estimation based at least in part on historical data associated with the non-visible portion, wherein the historical data comprises data associated with a previously received depth image where the non-visible portion is visible, wherein the historical data comprises at least one of an X value, a Y value, or a Z value of the non-visible portion in the previously received depth image where the non-visible portion is visible;
determining a new position of the non-visible portion when a pixel associated with the non-visible portion becomes visible in a subsequent depth image.
2. The method of claim 1 , wherein estimating the position of the non-visible portion of the human target further comprises dragging the non-visible portion of the human target behind the visible portion of the human target, wherein the non-visible portion drifts behind the visible portion of the human target.
3. The method of claim 1 , wherein estimating the position of the non-visible portion of the human target further comprises generating at least one of the following: an estimated X value, an estimated Y value, and an estimated Z value for the non-visible portion.
4. The method of claim 3 , wherein the estimated X value, the estimated Y value, and the estimated Z value are generated based on the visible portion of the human target.
5. The method of claim 4 , wherein the estimated X value, the estimated Y value, and the estimated Z value of the non-visible portion is generated based on at least one of the following: an X value of the visible portion, a Y value of the visible portion, a Z value of the visible portion, momentum of the visible portion, a velocity of the visible portion, a rotation of the visible portion, or a trajectory of the visible portion.
6. The method of claim 3 , further comprising:
adjusting a model of the human target based on the estimated X value, the estimated Y value, and the estimated Z value; and processing the model of the human target with the estimated X value, the estimated Y value, and the estimated Z value.
7. A computer-readable storage device having stored thereon computer-executable instructions, the computer-executable instructions comprising instructions for:
receiving a depth image of a scene;
determining that a portion of a human target in the depth image is non-visible;
estimating a position of the non-visible portion of the human target, wherein the estimating comprises:
generating an initial estimation of the position of the non-visible portion of the human target based at least in part on force vectors associated with a visible portion of the human target; and
modifying the initial estimation based at least in part on historical data associated with the non-visible portion, wherein the historical data comprises data associated with a previously received depth image where the non-visible portion is visible, wherein the historical data comprises at least one of an X value, a Y value, or a Z value of the non-visible portion in the previously received depth image where the non-visible portion is visible; and
determining a new position of the non-visible portion when a pixel associated with the non-visible portion becomes visible in a subsequent depth image.
8. The computer-readable storage device of claim 7 , wherein estimating the position of the non-visible portion of the human target further comprises dragging the non-visible portion of the human target behind the visible portion of the human target, wherein the non-visible portion drifts behind the visible portion of the human target.
9. The computer-readable storage device of claim 7 , wherein estimating the position of the non-visible portion of the human target further comprises generating at least one of the following: an estimated X value, an estimated Y value, and an estimated Z value for the non-visible portion.
10. The computer-readable storage device of claim 9 , wherein the estimated X value, the estimated Y value, and the estimated Z value are generated based on the visible portion of the human target.
11. The computer-readable storage device of claim 10 , wherein the estimated X value, the estimated Y value, and the estimated Z value of the non-visible portion is generated based on at least one of the following: an X value of the visible portion, a Y value of the visible portion, a Z value of the visible portion, momentum of the visible portion, a velocity of the visible portion, a rotation of the visible portion, or a trajectory of the visible portion.
12. The computer-readable storage device of claim 9 , wherein the computer executable instructions further comprise instructions for:
adjusting a model of the human target based on the estimated X value, the estimated Y value, and the estimated Z value; and processing the model of the human target with the estimated X value, the estimated Y value, and the estimated Z value.
13. A system comprising:
a capture device, wherein the capture device comprises a camera component operable to receive a depth image of a scene, wherein the depth image comprises a human target associated with a user; and
a computing device operable to communicate with the capture device, wherein the computing device is operable to perform operations comprising:
determining that a portion of the human target in the depth image is non-visible;
estimating a position of the non-visible portion of the human target, wherein the estimating comprises:
generating an initial estimation of the position of the non-visible portion of the human target based at least in part on force vectors associated with a visible portion of the human target; and
modifying the initial estimation based at least in part on historical data associated with the non-visible portion, wherein the historical data comprises data associated with a previously received depth image where the non-visible portion is visible, wherein the historical data comprises at least one of an X value, a Y value, or a Z value of the non-visible portion in the previously received depth image where the non-visible portion is visible; and
determining a new position of the non-visible portion when a pixel associated with the non-visible portion becomes visible in a subsequent depth image.
14. The system of claim 13 , wherein estimating the position of the non-visible portion of the human target further comprises dragging the non-visible portion of the human target behind the visible portion of the human target, wherein the non-visible portion drifts behind the visible portion of the human target.
15. The system of claim 13 , wherein estimating the position of the non-visible portion of the human target further comprises generating at least one of the following: an estimated X value, an estimated Y value, and an estimated Z value for the non-visible portion.
16. The system of claim 15 , wherein the estimated X value, the estimated Y value, and the estimated Z value are generated based on the visible portion of the human target.
17. The system of claim 16 , wherein the estimated X value, the estimated Y value, and the estimated Z value of the non-visible portion is generated based on at least one of the following: an X value of the visible portion, a Y value of the visible portion, a Z value of the visible portion, momentum of the visible portion, a velocity of the visible portion, a rotation of the visible portion, or a trajectory of the visible portion.
18. The system of claim 15 , wherein the operations further comprise:
adjusting a model of the human target based on the estimated X value, the estimated Y value, and the estimated Z value; and processing the model of the human target with the estimated X value, the estimated Y value, and the estimated Z value.