IP Library › Granted Patent US 8,577,085
Granted Patent B2
US 8,577,085 · App. 12/632,652 · Granted Nov 5, 2013

Visual target tracking

Inventor: Ryan M. Geiss (San Jose, CA)
Assignee: Microsoft Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,577,085
App. No.
12/632,652
Granted
Nov 5, 2013
Kind
B2
Abstract

A target tracking method includes modeling the target in a first frame with a first frame iteration of a machine-readable model and receiving an observed depth image of a second frame of a scene including the target. The first frame iteration of the machine-readable model is then adjusted into a second frame iteration of the machine-readable model based on the observed depth image of the second frame.

Claims (32)

1. A method of tracking a target in a plurality of successive frames, the method comprising:

receiving a previous frame iteration of a machine-readable model representing a pose of the target in the previous frame;

receiving an observed depth image of a scene including the target, the observed depth image imaging the scene in a current frame; and

adjusting the previous frame iteration of the machine-readable model into a current frame iteration of the machine-readable model based on a comparison of the observed depth image of the current frame and a depth image representing the previous frame iteration of the machine-readable model.

2. The method of claim 1 , where finding the current frame iteration of the machine-readable model includes moving one or more portions of the previous frame iteration of the machine-readable model based on the observed depth image.

3. The method of claim 2 , where a portion of the previous frame iteration of the machine-readable model is moved so that a depth value at a pixel of the machine-readable model more closely matches a corresponding depth value of a pixel of the observed depth image.

4. The method of claim 1 , where the machine-readable model includes a skeletal model including a plurality of skeletal points, each skeletal point defining a three-dimensional world space position for that skeletal point.

5. The method of claim 4 , where finding the current frame iteration of the machine-readable model includes calculating a world space direction and a world space distance to move one or more of the plurality of skeletal points from the previous frame to the current frame.

6. The method of claim 5 , where the world space direction and the world space distance are calculated for a skeletal point so that a depth value at a pixel of the machine-readable model proximate to the skeletal point more closely matches a corresponding depth value of a pixel of the observed depth image.

7. The method of claim 5 , where finding the current frame iteration of the machine-readable model includes adjusting the previous frame iteration of the machine-readable model into an adjusted model based on momentum derived from two or more previous frame iterations of the machine-readable model and rasterizing the adjusted model into a synthesized depth image, the synthesized depth image including a synthesized depth value at each pixel rasterized from the adjusted model.

8. The method of claim 7 , where rasterizing the adjusted model into the synthesized depth image includes labeling a first pixel of the synthesized depth image with an original body-part index indicating a forward-most part of the adjusted model at the first pixel.

9. The method of claim 8 , where rasterizing the adjusted model into the synthesized depth image includes labeling the first pixel of the synthesized depth image with a snap body-part index indicating a point on a surface of the machine-readable model that is closest to an observed depth value at the first pixel.

10. The method of claim 9 , where rasterizing the adjusted model into the synthesized depth image includes labeling one or more neighboring pixels neighboring the first pixel with a grow body-part index equal to the snap body-part index of the first pixel if the neighboring pixels do not have a snap body-part index corresponding to a body part and depth values of the neighboring pixels are within a tolerance of a depth value of the first pixel.

11. The method of claim 1 , where the current frame iteration is found without looking up the current frame iteration in a database.

12. A method of tracking a target in a plurality of successive frames, the method comprising:

modeling the target in a first frame with a first frame iteration of a machine-readable model;

receiving an observed depth image of a second frame of a scene including the target; and

adjusting the first frame iteration of the machine-readable model into a second frame iteration of the machine-readable model based on a comparison of the observed depth image of the second frame and a depth image representing the first frame iteration of the machine-readable model.

13. The method of claim 12 , where adjusting the first frame iteration of the machine-readable model into the second frame iteration of the machine-readable model includes moving one or more portions of the first frame iteration of the machine-readable model based on the observed depth image.

14. The method of claim 13 , where a portion of the first frame iteration of the machine-readable model is moved so that a depth value at a pixel of the second frame iteration of the machine-readable model more closely matches a corresponding depth value of a pixel of the observed depth image than a depth value at a pixel of the first frame iteration of the machine-readable model.

15. The method of claim 12 , where the machine-readable model includes a skeletal model including a plurality of skeletal points, each skeletal point defining a three-dimensional world space position for that skeletal point.

16. The method of claim 15 , where adjusting the first frame iteration of the machine-readable model into the second frame iteration of the machine-readable model includes calculating a world space direction and a world space distance to move one or more of the plurality of skeletal points from the first frame to the second frame.

17. A computing system for tracking a target in a plurality of successive frames, the computing system comprising:

an input to receive depth information from a source configured to capture the depth information;

a logic subsystem operatively connected to the input; and

a data-holding subsystem holding instructions executable by the logic subsystem to:

model the target in a first frame with a first frame iteration of a machine-readable model;

receive from the source via the input an observed depth image of a second frame of a scene including the target; and

adjust the first frame iteration of the machine-readable model into a second frame iteration of the machine-readable model based on a comparison of the observed depth image of the second frame and a depth image representing the first frame iteration of the machine-readable model.

18. The computing system of claim 17 , where the data-holding subsystem holds instructions executable by the logic subsystem so that adjusting the first frame iteration of the machine-readable model into the second frame iteration of the machine-readable model includes moving one or more portions of the first frame iteration of the machine-readable model based on the observed depth image.

19. The computing system of claim 17 , where the data-holding subsystem holds instructions executable by the logic subsystem so that a portion of the first frame iteration of the machine-readable model is moved so that a depth value at a pixel of the second frame iteration of the machine-readable model more closely matches a corresponding depth value of a pixel of the observed depth image than a depth value at a pixel of the first frame iteration of the machine-readable model.

20. The computing system of claim 17 , where the source includes a depth camera.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034564/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2010
From: GEISS, RYAN M.
To: MICROSOFT CORPORATION
Reel/Frame 025458/0769 →
Continuity (2)
Continuation In Part 12363604 · Jan 30, 2009
Related Publication 20100197392A1 · Aug 5, 2010