IP Library Granted Patent US 8,659,548
Granted Patent B2
US 8,659,548 · App. 12/124,375 · Granted Feb 25, 2014

Enhanced camera-based input

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,659,548
App. No.
12/124,375
Granted
Feb 25, 2014
Kind
B2
Abstract

Enhanced camera-based input, in which a detection region surrounding a user is defined in an image of the user within a scene, and a position of an object (such as a hand) within the detection region is detected. Additionally, a control (such as a key of a virtual keyboard) in a user interface is interacted with based on the detected position of the object.

Claims (85)

1. A non-transitory computer readable medium encoded with a computer program product, the computer program product comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

generating an image of a user within a scene;

defining, in the image, a detection region surrounding the user, further comprising:

determining a position of a torso and a reach of an arm of the user,

defining the detection region to exclude the torso and at least a portion of a region of the image unreachable by the arm;

determining a portion of the detection region in which a second user could be detected; and

defining the detection region to exclude the portion of the detection region in which the second user could be detected;

detecting a position of a hand of the user within the detection region; and

interacting with a control in a user interface based on the detected position of the hand, the control comprising items aligned with a guide line defined relative to an avatar representation of the user on the user interface.

2. A computer-implemented method comprising:

defining, in an image of a user within a scene, a detection region surrounding the user, the defining comprising:

determining an unreachable region of the image not reachable by an object associated with the user,

defining the detection region to exclude at least a portion of the unreachable region;

determining a portion of the detection region in which a second user could be detected; and

defining the detection region to exclude the portion of the detection region in which the second user could be detected;

detecting a position of the object within the detection region; and

interacting with a control in a user interface based on the detected position of the object.

3. The method of claim 2 , further comprising detecting an engagement gesture of the user, wherein the position of the object is detected based on detecting the engagement gesture.

4. The method of claim 2 , wherein the object is a hand of the user.

5. The method of claim 2 , wherein interacting with the control further comprises selecting a character.

6. The method of claim 2 , wherein the user interface further comprises a representation of the user.

7. The method of claim 6 , further comprising displaying a segmented image as the representation of the user.

8. The method of claim 6 , wherein the control comprises items aligned with a guide line defined relative to the representation, wherein the items are displayed without obscuring the representation.

9. The method of claim 6 , wherein the representation of the user further comprises an avatar or a cursor.

10. The method of claim 9 , further comprising animating the avatar to mimic motion of the user, based on the detected position of the object.

11. The method of claim 6 , further displaying the control in the user interface above or aside the representation.

12. The method of claim 2 , wherein defining the detection region further comprises:

determining a position of a torso of the user; and

defining the detection region to exclude the torso, based on the determined position of the torso.

13. The method of claim 12 , wherein the position of the torso of the user is determined using image segmentation.

14. The method of claim 12 , wherein defining the detection region comprises:

determining the unreachable region of the image not reachable by the object, based on the determined position of the torso; and

defining the detection region to exclude the determined unreachable region of the image.

15. The method of claim 2 , wherein the object comprises at least a portion of an arm of the user, and wherein defining the detection region further comprises:

determining a reach of the arm of the user; and

defining the detection region to exclude at least a portion of a region of the image unreachable by the arm, based on the determined reach of the arm.

16. The method of claim 15 , wherein the reach of the arm of the user is based on a determined position of a torso, head or shoulder of the user.

17. The method of claim 2 , wherein defining the detection region further comprises:

determining a position of a head of the user; and

defining the detection region to exclude the head, based on the determined position of the head.

18. The method of claim 17 , wherein the position of the head of the user is determined using face detection.

19. The method of claim 2 , further comprising:

cropping the image; and

displaying the image within the user interface.

20. The method of claim 19 , further comprising centering the image on the user, wherein the image is cropped based on centering the image.

21. The method of claim 20 , further comprising determining a magnification factor that makes the control reachable to the user within the centered image, wherein the image is cropped based on the magnification factor.

22. The method of claim 21 , wherein the magnification factor is determined using an anatomical model.

23. The method of claim 2 , further comprising:

detecting a face of the user;

determining an identity of the user based on the detected face; and

adjusting the control based on the identity of the user.

24. The method of claim 2 , further comprising:

detecting a position of a second object within the detection region; and

adjusting the control based on the position of the second object.

25. The method of claim 2 , further comprising capturing the image using a camera.

26. The method of claim 2 , wherein the detection region is shaped as an arc-shaped detection sub-region above the user, contiguous with two linear detection sub-regions, one on each side of the user.

27. The method of claim 2 , wherein detecting the position of the object within the detection region further comprises:

detecting a position of the object in relation to a guideline mapped within the detection region.

28. A non-transitory computer readable medium encoded with a computer program product, the computer program product comprising instructions that, when executed, operate to cause a computer to perform operations comprising:

defining, in an image of a user within a scene, a detection region surrounding the user, the defining comprising:

determining an unreachable region of the image not reachable by an object associated with the user,

defining the detection region to exclude at least a portion of the unreachable region;

determining a portion of the detection region in which a second user could be detected; and

defining the detection region to exclude the portion of the detection region in which the second user could be detected;

detecting a position of the object within the detection region; and

interacting with a control in a user interface based on the detected position of the object.

29. A device comprising a processor configured to:

define, in an image of a user within a scene, a detection region surrounding the user, at least in part by:

determining an unreachable region of the image not reachable by an object associated with the user,

defining the detection region to exclude at least a portion of the unreachable region;

determining a portion of the detection region in which a second user could be detected; and

defining the detection region to exclude the portion of the detection region in which the second user could be detected;

detect a position of the object within the detection region; and

interact with a control in a user interface based on the detected position of the object.

30. The method of claim 2 , wherein the detection region is defined to overlap at least one portion of the user.

31. The method of claim 2 , wherein the detection region is defined not to overlap the user.

32. The method of claim 2 , wherein the detection region is defined in response to an engagement gesture being performed.

33. An apparatus comprising:

means for defining, in an image of a user within a scene, a detection region surrounding the user, the means for defining comprising:

means for determining an unreachable region of the image not reachable by an object associated with the user,

means for defining the detection region to exclude at least a portion of the unreachable region;

means for determining a portion of the detection region in which a second user could be detected; and

means for defining the detection region to exclude the portion of the detection region in which the second user could be detected;

means for detecting a position of the object within the detection region; and

means for interacting with a control in a user interface based on the detected position of the object.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 2, 2011
From: GESTURETEK, INC.
To: QUALCOMM INCORPORATED
Reel/Frame 026690/0421 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 22, 2008
From: HILDRETH, EVAN
To: GESTURETEK, INC.
Reel/Frame 020987/0976 →