IP Library Granted Patent US 10,268,339
Granted Patent B2
US 10,268,339 · App. 14/150,562 · Granted Apr 23, 2019

Enhanced camera-based input

Inventor: Evan Hildreth (Thornhill, CA)
Assignee: QUALCOMM Incorporated
G06F3/0482G06F3/011G06F3/017G06F3/03G06F3/0304G06F3/04842
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,268,339
App. No.
14/150,562
Granted
Apr 23, 2019
Kind
B2
Abstract

Enhanced camera-based input, in which a detection region surrounding a user is defined in an image of the user within a scene, and a position of an object (such as a hand) within the detection region is detected. Additionally, a control (such as a key of a virtual keyboard) in a user interface is interacted with based on the detected position of the object.

Claims (103)

1. A computer-implemented method comprising:

obtaining an image of a control object in a scene, the image captured by a camera;

determining a location of a first object in a user interface on a display, based on the location of the control object in the scene, wherein the first object comprises a control portion and a non-control portion;

determining a range of motion of the control portion of the first object based on a dimension of the non-control portion of the first object;

determining a first set of items; and

dynamically positioning one or more guide lines in the user interface on the display based on the determined location of the first object, the determined range of motion of the control portion of the first object, and a quantity of the items in the first set of items, wherein:

the items are shown on the display and aligned with the one or more guide lines in the user interface;

and

the one or more guide lines are positioned such that the items do not overlap with the non-control portion of the first object.

2. The computer-implemented method of claim 1 wherein the user interface is shown on a display of a mobile device.

3. The computer-implemented method of claim 1 wherein the user interface is shown on a television display.

4. The computer-implemented method of claim 1 wherein the first object comprises a representation of a user.

5. The computer-implemented method of claim 1 further comprising:

determining a location of a second object in the user interface; and

repositioning the one or more guide lines such that the items do not overlap with the second object.

6. The computer-implemented method of claim 5 further, wherein the repositioning comprises changing one or more of:

a color of an item;

a line thickness of an item;

a size of an item; or

a transparency of an item.

7. The computer-implemented method of claim 5 wherein the repositioning comprises repositioning the one or more guide lines such that items are outside a determined range of motion of a control portion of the second object.

8. The computer-implemented method of claim 1 farther comprising:

determining a new location of the first object in the user interface; and

repositioning the one or more guide lines based on the new location such that the items do not overlap with the non-control portion of the first object.

9. The computer-implemented method of claim 1 further comprising:

receiving a selection of an item; and

creating, based on the selection, a new guide line having a second set of items aligned therewith.

10. The computer-implemented method of claim 9 wherein:

a guide line component of the one or more guide lines is generally defined within the same X-Y plane as the user interface; and

creating the new guide line comprises creating the new guide line to be generally defined within a plane other than the X-Y plane.

11. The computer-implemented method of claim 10 wherein items aligned with the new guide line decrease in size from a first end of the new guide line to a second end of the new guide line.

12. A device comprising a processor configured to:

obtain an image of a control object in a scene, the image captured by a camera;

determine a location of a first object in a user interface on a display, based on a the location of the control object in the scene, wherein the first object comprises a control portion and a non-control portion;

determine a range of motion of the control portion of the first object based on a dimension of the non-control portion of the first object;

determine a first set of items; and

dynamically position one or more guide lines in the user interface on the display based on the determined location of the first object, the determined range of motion of the control portion of the first object, and a quantity of the items in the first set of items, wherein:

the items are shown on the display and aligned with the one or more guide lines in the user interface;

and

the one or more guide lines are positioned such that the items do not overlap with the non-control portion of the first object.

13. The device of claim 12 , wherein the device is a mobile device.

14. The device of claim 12 , wherein the processor is further configured to:

determine a location of a second object in the user interface; and

reposition the one or more guide lines such that the items do not overlap with the second object.

15. The device of claim 14 , wherein the repositioning comprises changing one or more of:

a color of an item;

a line thickness of an item;

a size of an item; or

a transparency of an item.

16. The device of claim 14 , wherein the processor is configured to reposition the one or more guide lines such that items are outside a determined range of motion of a control portion of the second object.

17. The device of claim 12 , wherein the processor is further configured to:

determine a new location of the first object in the user interface; and

reposition the one or more guide lines based on the new location such that the items do not overlap with the non-control portion of the first object.

18. The device of claim 12 , wherein the processor is further configured to:

receive a selection of an item; and

create, based on the selection, a new guide line having a second set of items aligned therewith.

19. The device of claim 18 , wherein the processor is further configured to:

cause a guide line component of the one or more guide lines to be generally defined within the same X-Y plane as the user interface; and

create the new guide line to be generally defined within a plane other than the X-Y plane.

20. The device of claim 18 , wherein the processor is configured to cause items aligned with the new guide line to be displayed in decreasing size from a first end of the new guide line to a second end of the new guide line.

21. An apparatus comprising:

means for obtaining an image of a control object in a scene, the image captured by image capturing means;

means for determining a location of a first object in a user interface on a display, based on the location of the control object in the scene, wherein the first object comprises a control portion and a non-control portion;

means for determining a range of motion of the control portion of the first object based on a dimension of the non-control portion of the first object;

means for determining a first set of items; and

means for dynamically positioning one or more guide lines in the user interface on the display based on the determined location of the first object, the determined range of motion of the control portion of the first object, and a quantity of the items in the first set of items, wherein:

the items are shown on the display and aligned with the one or more guide lines in the user interface;

and

the one or more guide lines are positioned such that the items do not overlap with the non-control portion of the first object.

22. The apparatus of claim 21 further comprising:

means for determining a location of a second object in the user interface; and

means for repositioning the one or more guide lines such that the items do not overlap with the second object.

23. The apparatus of claim 22 the means for repositioning comprises means for changing one or more of:

a color of an item;

a line thickness of an item;

a size of an item; or

a transparency of an item.

24. The apparatus of claim 22 wherein the means for repositioning comprises means for repositioning the one or more guide lines such that items are outside a determined range of motion of a control portion of the second object.

25. The apparatus of claim 21 further comprising:

means for determining a new location of the first object in the user interface; and

means for repositioning the one or more guide lines based on the new location such that the items do not overlap with the non-control portion of the first object.

26. The apparatus of claim 21 further comprising:

means for receiving a selection of an item; and

means for creating, based on the selection, a new guide line having a second set of items aligned therewith.

27. The apparatus of claim 26 wherein:

a guide line component of the one or more guide lines is generally defined within the same X-Y plane as the user interface; and

the means for creating the new guide line comprises means for creating the new guide line to be generally defined within a plane other than the X-Y plane.

28. The apparatus of claim 27 further comprising means for causing the items aligned with the new guide line to decrease in size from a first end of the new guide line to a second end of the new guide line.

29. A non-transitory computer readable medium encoded with instructions that, when executed, operate to cause a computer to perform operations comprising:

obtaining an image of a control object in a scene, the image captured by a camera;

determining a location of a first object in a user interface on a display, based on the location of the control object in the scene, wherein the first object comprises a control portion and a non-control portion;

determining a range of motion of the control portion of the first object based on a dimension of the non-control portion of the first object;

determining a first set of items; and

dynamically positioning one or more guide lines in the user interface on the display based on the determined location of the first object, the determined range of motion of the control portion of the first object, and a quantity of the items in the first set of items, wherein:

the items are shown on the display and aligned with the one or more guide lines in the user interface;

and

the one or more guide lines are positioned such that the items do not overlap with the non-control portion of the first object.

30. The computer readable medium of claim 29 further comprising instructions for causing the computer to:

determine a location of a second object in the user interface; and

reposition the one or more guide lines such that the items do not overlap with the second object.

31. The computer readable medium of claim 29 further comprising instructions for causing the computer to:

receive a selection of an item; and

create, based on the selection, a new guide line having a second set of items aligned therewith.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2014
From: GESTURETEK, INC.
To: QUALCOMM INCORPORATED
Reel/Frame 032044/0908 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2014
From: HILDRETH, EVAN
To: GESTURETEK, INC.
Reel/Frame 032124/0626 →
Continuity (3)
Continuation 12124375 · May 21, 2008
Provisional Application 60952448 · Jul 27, 2007
Related Publication 20140118249A1 · May 1, 2014