IP Library › Granted Patent US 8,818,027
Granted Patent B2
US 8,818,027 · App. 13/013,657 · Granted Aug 26, 2014

Computing device interface

Inventors: Babak Forutanpour (San Diego, CA); Jianfeng Ren (San Diego, CA)
Assignee: Qualcomm Incorporated
G06F3/017G06T2207/30196G06T2207/10016G06K9/00355G06T7/204G06F3/0418G06F3/0346G06K9/2036G06F3/0425
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,818,027
App. No.
13/013,657
Granted
Aug 26, 2014
Kind
B2
Abstract

A computing device configured for providing an interface is described. The computing device includes a processor and instructions stored in memory. The computing device projects a projected image from a projector. The computing device also captures an image including the projected image using a camera. The camera operates in a visible spectrum. The computing device calibrates itself, detects a hand and tracks the hand based on a tracking pattern in a search space. The computing device also performs an operation.

Claims (91)

1. A computing device configured for providing an interface, comprising:

a processor;

memory in electronic communication with the processor; and

instructions stored in the memory, the instructions being executable to:

project a projected image from a projector;

capture an image using a camera, wherein the camera operates in a visible spectrum;

calibrate the computing device;

detect a hand;

track the hand by scanning a search space within the captured image for a tracking pattern representative of the hand, the search space size being based on the hand size, the hand size being determined by a change in intensity of a portion of the captured image; and

perform an operation.

2. The computing device of claim 1 , wherein calibrating the computing device comprises:

finding corners of the projected image; and

mapping coordinates of the projected image to a desktop space.

3. The computing device of claim 1 , wherein calibrating the computing device comprises determining the hand size.

4. The computing device of claim 3 , wherein the hand size is determined based on the projected image, wherein the projected image comprises a plurality of ellipses.

5. The computing device of claim 3 , wherein the hand size is determined based on facial recognition.

6. The computing device of claim 1 , wherein the hand is detected in an initial interaction area.

7. The computing device of claim 6 , wherein the instructions are further executable to:

track a body; and

move the initial interaction area based on tracking the body.

8. The computing device of claim 1 , wherein the instructions are further executable to determine that the tracking pattern is matched to a hand.

9. The computing device of claim 1 , wherein the instructions are further executable to dynamically change the search space based on a motion of the hand.

10. The computing device of claim 1 , wherein the tracking pattern comprises a rectangular center dark region adjoined on three sides by rectangular light outer regions and adjoined on one side by a rectangular dark outer region.

11. The computing device of claim 1 , wherein the operation is performed based on tracking the hand.

12. The computing device of claim 1 , wherein the instructions are further executable to recognize a hand gesture based on a gesture pattern.

13. The computing device of claim 12 , wherein recognizing a hand gesture further comprises:

determining whether a tracked hand has stopped moving for a period of time; and

scaling a tracked hand image to a fixed size.

14. The computing device of claim 12 , wherein the gesture pattern comprises a rectangular center dark region adjoined on two sides by rectangular light outer regions.

15. The computing device of claim 12 , wherein the instructions are further executable to perform an operation based on the recognized hand gesture.

16. The computing device of claim 12 , wherein the instructions are further executable to perform an operation when the hand gesture is recognized for a percentage of a number of frames.

17. The computing device of claim 12 , wherein the hand gesture is recognized based on an angle between a thumb and index finger, and wherein the instructions are further executable to move a cursor away from the hand based on the angle between the thumb and index finger.

18. The computing device of claim 1 , wherein the instructions are further executable to illuminate the projected image more brightly in a region around a hand.

19. The computing device of claim 1 , wherein the instructions are further executable to track a plurality of hands.

20. The computing device of claim 1 , wherein the instructions are further executable to recognize a plurality of hand gestures.

21. The computing device of claim 1 , wherein the instructions are further executable to remove dark portions of the image before tracking the hand to avoid tracking a shadow of the hand.

22. The computing device of claim 1 , wherein the instructions are further executable to provide improved focus and exposure of the image in an area of the projected image.

23. The computing device of claim 15 , wherein the operation releases a cursor.

24. A method for providing an interface, comprising:

projecting a projected image from a projector;

capturing an image using a camera, wherein the camera operates in a visible spectrum;

calibrating a computing device;

detecting, by the computing device, a hand;

tracking, by the computing device, the hand by scanning a search space within the captured image for a tracking pattern representative of the hand, the search space size being based on the hand size, the hand size being determined by a change in intensity of a portion of the captured image; and

performing an operation by the computing device.

25. The method of claim 24 , wherein calibrating the computing device comprises:

finding corners of the projected image; and

mapping coordinates of the projected image to a desktop space.

26. The method of claim 24 , wherein calibrating the computing device comprises determining the hand size.

27. The method of claim 26 , wherein the hand size is determined based on the projected image, wherein the projected image comprises a plurality of ellipses.

28. The method of claim 26 , wherein the hand size is determined based on facial recognition.

29. The method of claim 24 , wherein the hand is detected in an initial interaction area.

30. The method of claim 29 , further comprising:

tracking a body; and

moving the initial interaction area based on tracking the body.

31. The method of claim 24 , further comprising determining that the tracking pattern is matched to a hand.

32. The method of claim 24 , further comprising dynamically changing the search space dynamically based on a motion of the hand.

33. The method of claim 24 , wherein the tracking pattern comprises a rectangular center dark region adjoined on three sides by rectangular light outer regions and adjoined on one side by a rectangular dark outer region.

34. The method of claim 24 , wherein the operation is performed based on tracking the hand.

35. The method of claim 24 , further comprising recognizing a hand gesture based on a gesture pattern.

36. The method of claim 35 , wherein recognizing a hand gesture further comprises:

determining whether a tracked hand has stopped moving for a period of time; and

scaling a tracked hand image to a fixed size.

37. The method of claim 35 , wherein the gesture pattern comprises a rectangular center dark region adjoined on two sides by rectangular light outer regions.

38. The method of claim 35 , further comprising performing an operation based on the recognized hand gesture.

39. The method of claim 35 , further comprising performing an operation when the hand gesture is recognized for a percentage of a number of frames.

40. The method of claim 35 , wherein the hand gesture is recognized based on an angle between a thumb and index finger, and the method further comprises moving a cursor away from the hand based on the angle between the thumb and index finger.

41. The method of claim 24 , further comprising illuminating the projected image more brightly in a region around a hand.

42. The method of claim 24 , further comprising tracking a plurality of hands.

43. The method of claim 24 , further comprising recognizing a plurality of hand gestures.

44. The method of claim 24 , further comprising removing dark portions of the image before tracking the hand to avoid tracking a shadow of the hand.

45. The method of claim 24 , further comprising providing improved focus and exposure of the image in an area of the projected image.

46. The method of claim 38 , wherein the operation releases a cursor.

47. A computer-program product for providing an interface, the computer-program product comprising a non-transitory tangible computer-readable medium having instructions thereon, the instructions comprising:

code for causing a computing device to project a projected image from a projector;

code for causing the computing device to capture an image using a camera, wherein the camera operates in a visible spectrum;

code for causing the computing device to calibrate the computing device;

code for causing the computing device to detect a hand;

code for causing the computing device to track the hand by scanning a search space within the captured image for a tracking pattern representative of the hand, the search space size being based on the hand size, the hand size being determined by a change in intensity of a portion of the captured image; and

code for causing the computing device to perform an operation.

48. The computer-program product of claim 47 , wherein the hand is detected in an initial interaction area.

49. The computer-program product of claim 47 , wherein the instructions further comprise code for causing the computing device to recognize a hand gesture based on a gesture pattern.

50. An apparatus for providing an interface, comprising:

means for projecting a projected image from a projector;

means for capturing an image using a camera, wherein the camera operates in a visible spectrum;

means for calibrating the apparatus;

means for detecting a hand;

means for tracking the hand by scanning a search space within the captured image for a tracking pattern representative of the hand, the search space size being based on the hand size, the hand size being determined by a change in intensity of a portion of the captured image; and

means for performing an operation.

51. The apparatus of claim 50 , wherein the hand is detected in an initial interaction area.

52. The apparatus of claim 50 , further comprising means for recognizing a hand gesture based on a gesture pattern.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 25, 2011
From: FORUTANPOUR, BABAK; REN, JIANFENG
To: QUALCOMM INCORPORATED
Reel/Frame 025696/0110 →
Continuity (2)
Provisional Application 61320158 · Apr 1, 2010
Related Publication 20110243380A1 · Oct 6, 2011