IP Library › Granted Patent US 8,994,718
Granted Patent B2
US 8,994,718 · App. 12/975,086 · Granted Mar 31, 2015

Skeletal control of three-dimensional virtual world

Inventors: Stephen Latta (Seattle, WA); Darren Bennett (Mercer Island, WA); Kevin Geisner (Seattle, WA); Relja Markovic (Seattle, WA)
Assignee: Microsoft Technology Licensing, LLC
G06F3/005G06T13/40G06F3/011G06F3/017G06F3/0304G06F3/0486G06F2203/04806
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,994,718
App. No.
12/975,086
Filed
Dec 21, 2010
Granted
Mar 31, 2015
Kind
B2
Examiner
DU, HAIXIA
Art Unit
2611
USPC
345/419
Abstract

A virtual skeleton includes a plurality of joints and provides a machine readable representation of a human target observed with a three-dimensional depth camera. A relative position of a hand joint of the virtual skeleton is translated as a gestured control, and a three-dimensional virtual world is controlled responsive to the gestured control.

Claims (49)

1. A data holding device holding instructions executable by a logic device, the instructions comprising:

instructions to render a three-dimensional virtual gaming world for display on a display device;

instructions to receive a machine-readable virtual skeleton structurally representing a human being sighted by a depth camera in a volume large enough to envelop the human being, the virtual skeleton including a hand joint position in three dimensions corresponding to a position of a hand of the human being;

instructions to render a control cursor for display on the display device, a screen position of the control cursor tracking the hand joint position;

instructions to lock the control cursor to an object in the three-dimensional virtual gaming world if a grab threshold of the object is overcome;

instructions to, when the control cursor is locked to the object, move the object in all three dimensions to track the hand joint position anywhere in the volume, such that movement of the hand effects a corresponding movement of the object in the three-dimensional virtual gaming world; and

instructions to unlock the control cursor from the object at a release position of the object within the three-dimensional virtual gaming world if a release threshold of the object is overcome.

2. The data holding device of claim 1 , where the grab threshold of the object is overcome if the hand is closed.

3. The data holding device of claim 1 , where the grab threshold of the object is overcome if the hand is within a threshold distance of the object for a threshold duration.

4. The data holding device of claim 1 , where the grab threshold of the object is overcome if the hand is within a threshold distance of the object and a speed of the hand is below a threshold speed for a threshold duration.

5. The data holding device of claim 1 , where the release threshold is overcome if the hand is open.

6. The data holding device of claim 1 , wherein the instructions further comprise:

instructions to, when the control cursor is locked to the object, rotate the object in the three-dimensional virtual gaming world responsive to a rotation gesture of the virtual skeleton.

7. The data holding device of claim 1 , wherein the instructions further comprise:

instructions to, when the control cursor is locked to the object, scale the object in the three-dimensional virtual gaming world responsive to a scale gesture of the virtual skeleton.

8. The data holding device of claim 1 , wherein the object is one of a plurality of objects in the three-dimensional virtual gaming world selected responsive to a selection gesture of the virtual skeleton, and wherein the instructions further comprise:

instructions to lock the control cursor to the plurality of objects if a grab threshold of the plurality of objects is overcome;

instructions to, when the control cursor is locked to the plurality of objects, move the plurality of objects in all three dimensions to track the hand joint position, such that movement of the hand effects a corresponding movement of the plurality of objects in the three-dimensional virtual gaming world; and

instructions to unlock the control cursor from the plurality of objects if a release threshold of the plurality of objects is overcome.

9. The data holding device of claim 8 , wherein the instructions further comprise:

instructions to, when the control cursor is locked to the plurality of objects, rotate the plurality of objects in the three-dimensional virtual gaming world responsive to a rotation gesture of the virtual skeleton.

10. The data holding device of claim 8 , wherein the instructions further comprise:

instructions to, when the control cursor is locked to the plurality of objects, scale the plurality of objects in the three-dimensional virtual gaming world responsive to a scale gesture of the virtual skeleton.

11. The data holding device of claim 1 , wherein the instructions further comprise:

instructions to recognize a zoom-in gesture in which right and left hands of the human being are separated; and

instructions to magnify a view of the three-dimensional virtual gaming world on the display responsive to the zoom-in gesture.

12. A data holding device holding instructions executable by a logic device, the instructions comprising:

instructions to render a three-dimensional virtual gaming world for display on a display device;

instructions to receive a machine-readable virtual skeleton structurally representing a human being sighted by a depth camera, the virtual skeleton including a hand joint position in three dimensions corresponding to a position of a hand of the human being;

instructions to render a control cursor as a human-hand image for display on the display device, a screen position of the control cursor tracking the hand joint position;

instructions to lock the control cursor to an object in the three-dimensional virtual gaming world if a grab threshold of the object is overcome, where the control cursor when locked to the object is rendered closed on the object;

instructions to, when the control cursor is locked to the object, move the object in all three dimensions to track the hand joint position, such that movement of the hand effects a corresponding movement of the object in the three-dimensional virtual gaming world;

instructions to unlock the control cursor from the object at a release position of the object within the three-dimensional virtual gaming world if a release threshold of the object is overcome, where the control cursor when unlocked from the object is rendered reopened; and

instructions to, responsive to the screen position of the control cursor reaching a scrolling threshold at a visible edge of the three-dimensional virtual gaming world, scrolling the three-dimensional virtual gaming world such that a previously hidden portion of the three-dimensional virtual gaming world adjacent to the visible edge of the three-dimensional virtual gaming world becomes unhidden.

13. The data holding device of claim 12 wherein the instructions further comprise:

instructions to recognize a zoom-out gesture in which right and left hands of the human being are brought closer; and

instructions to demagnify a view of the three-dimensional virtual gaming world on the display responsive to the zoom-out gesture.

14. A data holding device holding instructions executable by a logic device, the instructions comprising:

instructions to render a three-dimensional virtual gaming world for display on a display device;

instructions to receive a machine-readable virtual skeleton structurally representing a human being sighted by a depth camera, the virtual skeleton including a hand joint position in three dimensions corresponding to a position of a hand of the human being;

instructions to render a visually closable control cursor for display on the display device, a screen position of the control cursor tracking the hand joint position;

instructions to lock the control cursor to an object in the three-dimensional virtual gaming world if a grab threshold of the object is overcome, where the control cursor when locked to the object is rendered closed on the object;

instructions to, when the control cursor is locked to the object, move the object in all three dimensions to track the hand joint position, such that movement of the hand effects a corresponding movement of the object in the three-dimensional virtual gaming world; and

instructions to unlock the control cursor from the object at a release position of the object within the three-dimensional virtual gaming world if a release threshold of the object is overcome, where the control cursor when unlocked from the object is rendered reopened.

15. The data holding device of claim 14 wherein the instructions further comprise:

instructions to, responsive to recognizing a first mode initiation gesture of the virtual skeleton, set a first input modality;

instructions to, while the first input modality is set, interpret a specific action gesture of the virtual skeleton as a first gesture control;

instructions to, responsive to recognizing a second mode initiation gesture of the virtual skeleton, set a second input modality; and

instructions to, while the second input modality is set, interpret the specific action gesture of the virtual skeleton as a second gesture control, the first gesture control and the second gesture control producing different controls of the three-dimensional virtual world.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034544/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 6, 2011
From: LATTA, STEPHEN; BENNETT, DARREN; GEISNER, KEVIN; MARKOVIC, RELJA
To: MICROSOFT CORPORATION
Reel/Frame 025591/0201 →
Continuity (1)
Related Publication 20120157203A1 · Jun 21, 2012