IP Library Granted Patent US 9,772,689
Granted Patent B2
US 9,772,689 · App. 12/041,927 · Granted Sep 26, 2017

Enhanced gesture-based image manipulation

Inventor: Evan Hildreth (Ottawa, CA)
Assignee: QUALCOMM Incorporated
G06F3/017G06F3/0304
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,772,689
App. No.
12/041,927
Granted
Sep 26, 2017
Kind
B2
Abstract

Enhanced image viewing, in which a user's gesture is recognized from first and second images, an interaction command corresponding to the recognized user's gesture is determined, and, based on the determined interaction command, an image object displayed in a user interface is manipulated.

Claims (66)

1. A computer-implemented method comprising:

causing a display to render an image object in a user interface;

determining a position of a user's torso;

defining, with a processing unit, a plane relative to the user's torso, based on the determined position;

ignoring, by the processing unit, movements of the user that are made on a far side of the plane from a camera capturing images of the user;

determining, with the processing unit, an extent to which the user is able to reach;

recognizing, from first and second images captured by the camera, a user's gesture performed between the camera and the plane;

determining, with the processing unit, an interaction command corresponding to the recognized user's gesture; and

manipulating, with a processing unit and based on the determined interaction command, the image object displayed in the user interface, wherein a magnitude of the manipulation is related to a proximity of the user's gesture to the determined extent to which the user is able to reach.

2. The method of claim 1 , wherein: the interaction command comprises a selection command, and manipulating the image object further comprises selecting the image object for further manipulation.

3. The method of claim 1 , wherein: the interaction command comprises an image pan command, and manipulating the image object further comprises panning the image object relative to the user interface.

4. The method of claim 3 , wherein recognizing the user's gesture further comprises:

detecting a first position of an arm of the user in the first image;

detecting a second position of the arm of the user in the second image; and

determining a magnitude and direction of a change between the first position and the second position.

5. The method of claim 4 , wherein, in a distance mode, manipulating the image object further comprises:

determining a displacement position of the image object correlating to the determined magnitude and direction; and

displaying the image object in the displacement position.

6. The method of claim 4 , wherein, in a velocity mode, the magnitude of the manipulation comprises a scroll magnitude, and the scroll magnitude and direction correlates to the determined magnitude and direction; and

manipulating the image object further comprises scrolling the image object based on the determined scroll magnitude and direction.

7. The method of claim 1 , wherein: the interaction command comprises an image zoom command, and manipulating the image object further comprises zooming the image object relative to the user interface.

8. The method of claim 7 , wherein recognizing the user's gesture further comprises:

detecting a first position of an arm of the user in the first image;

detecting a second position of the arm of the user in the second image; and

determining a magnitude and direction of a change between the first position and the second position.

9. The method of claim 8 , wherein, in a distance mode, manipulating the image object further comprises:

determining a magnification factor correlating to the determined magnitude and direction; and

applying the determined magnification factor to the image object.

10. The method of claim 8 , wherein, in a velocity mode, the magnitude of the manipulation comprises an adjustment magnitude, and the adjustment magnitude and direction correlates to the determined magnitude and direction; and

manipulating the image object further comprises iteratively adjusting a magnification factor of the image object based on the determined adjustment magnitude and direction.

11. The method of claim 1 , wherein: the interaction command comprises a rotation command, and manipulating the object further comprises rotating the image object relative to the user interface.

12. The method of claim 11 , wherein recognizing the user's gesture further comprises:

detecting a first orientation of a hand of the user in the first image;

detecting a second orientation of the hand of the user in the second image; and

determining an orientation change between the first position and the second position.

13. The method of claim 12 , wherein, in a distance mode, manipulating the image object further comprises:

determining displacement orientation of the image object correlating to the determined magnitude and direction; and

displaying the image object in the displacement orientation.

14. The method of claim 12 , wherein, in a velocity mode, manipulating the image object further comprises:

the magnitude of the manipulation comprises an adjustment magnitude, and the adjustment magnitude and direction correlates to the determined magnitude and direction; and

manipulating the image object further comprises iteratively adjusting an orientation of the image object based on the determined adjustment magnitude and direction.

15. The method of claim 1 , wherein the image object is manipulated in a manipulation direction mirroring a direction of the user's gesture.

16. The method of claim 1 , wherein: recognizing the user's gesture further comprises: recognizing a first selection gesture, recognizing a first interaction gesture, recognizing a de-selection gesture, recognizing a repositioning gesture, recognizing a second selection gesture, and recognizing a second interaction gesture; and manipulating the image object further comprises: selecting the image object based on recognizing the first selection gesture, adjusting, using a single adjustment technique associated with the first and second interaction gestures, the image object based on recognizing the first and second interaction gestures, and filtering the repositioning gesture.

17. The method of claim 1 , wherein: the interaction command comprises a preview image command, and manipulating the image object further comprises selecting, from a plurality of preview image objects, the image object.

18. The computer-implemented method if claim 1 , wherein causing the display to render the image object in the user interface comprises sending information to the display via a display interface of a computer.

19. The computer-implemented method if claim 1 , further comprising receiving the first and second images from the camera via a digital input interface of a computer.

20. An apparatus comprising:

a processor configured to:

cause a display to render an image object in a user interface;

determine a position of a user's torso;

define a plane relative to the user's torso, based on the determined position;

ignore movements of the user that are made on a far side of the plane from a camera capturing images of the user;

determine an extent to which the user is able to reach;

recognize, from first and second images captured by the camera, a user's gesture performed between the camera and the plane;

determine an interaction command corresponding to the recognized user's gesture; and

manipulate, based on the determined interaction command, the image object displayed in the user interface, wherein a magnitude of the manipulation is related to a proximity of the user's gesture to the determined extent to which the user is able to reach.

21. A non-transitory processor-readable medium comprising processor-readable instructions configured to cause a processor to:

causing a display to render an image object in a user interface;

determine a position of a user's torso;

define a plane relative to the user's torso, based on the determined position;

ignore movements of the user that are made on a far side of the plane from a camera capturing images of the user;

determine an extent to which the user is able to reach;

recognize, from first and second images captured by the camera, a user's gesture performed between the camera and the plane;

determine an interaction command corresponding to the recognized user's gesture; and

manipulate, based on the determined interaction command, the image object displayed in the user interface, wherein a magnitude of the manipulation is related to a proximity of the user's gesture to the determined extent to which the user is able to reach.

22. The computer-implemented method if claim 1 , wherein determining an extent of the user's reach comprises using an anatomical model.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 2, 2011
From: GESTURETEK, INC.
To: QUALCOMM INCORPORATED
Reel/Frame 026690/0421 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2008
From: HILDRETH, EVAN
To: GESTURETEK, INC.
Reel/Frame 020601/0541 →
Continuity (1)
Related Publication 20090228841A1 · Sep 10, 2009