IP Library Granted Patent US 9,836,201
Granted Patent B2
US 9,836,201 · App. 14/485,840 · Granted Dec 5, 2017

Zoom-based gesture user interface

Inventors: Martin Frey (Munich, DE); Marcus Hauer (Munich, DE); Dario Buzzini (New York, NY); Philipp Schaefer (Munich, DE); Adi Berenson (Tel Aviv, IL); Micha Galor (Tel Aviv, IL); Nili Metuki (Even-Yehuda, IL); Alexander Shpunt (Tel-Aviv, IL)
Assignee: APPLE INC.
G06F3/04845G06F3/013G06F3/017G06F3/0482G06F3/0485G06F3/04815G06F2203/04806
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,836,201
App. No.
14/485,840
Granted
Dec 5, 2017
Kind
B2
Abstract

A user interface method, including presenting by a computer executing a user interface, multiple interactive items on a display. A first sequence of images is captured indicating a position in space of a hand of a user in proximity to the display, and responsively to the position, one of the interactive items is associated with the hand. After associating the item, a second sequence of images is captured indicating a movement of the hand, and responsively to the movement, a size of the one of the items is changed on the display.

Claims (32)

1. A user interface method, comprising:

presenting by a computer executing a user interface, multiple interactive items on a display;

capturing a first sequence of images indicating a position in space of a hand of a user in proximity to the display;

responsively to the position, associating a given interactive item on the display with the hand;

after associating the given item, capturing a second sequence of images indicating a movement of the hand away from the display; and

responsively to the movement away from the display, changing a size of the given item on the display.

2. The method according to claim 1 , wherein the given interactive item is enlarged in response to the movement of the hand comprising motion away from the display.

3. The method according to claim 1 , wherein the given interactive item is associated with a function of the computer, and wherein the method comprises activating the function when the given interactive item has been enlarged, in response to the movement, to at least a predetermined threshold size.

4. The method according to claim 3 , wherein activating the function comprises playing a media item associated with the given interactive item.

5. The method according to claim 3 , wherein activating the function comprises executing a software application.

6. The method according to claim 3 , wherein activating the function comprises displaying an image over a full area of the display.

7. The method according to claim 1 , wherein associating the given interactive item comprises choosing the interactive item that is presented at a center of the display.

8. The method according to claim 1 , wherein associating the given interactive item comprises detecting, in the first sequence of images, a predetermined gesture of the hand, and initiating association of the given interactive item with the hand responsively to the gesture.

9. The method according to claim 1 , wherein at least some of the interactive items contain sub-items, which are not displayed as long as the interactive items presented on the display are smaller than a predetermined size, and wherein enlarging the given interactive item comprises displaying the sub-items contained by the given interactive item when the given interactive item is enlarged to at least the predetermined size.

10. The method according to claim 9 , wherein the given interactive item is enlarged in response the movement comprising significant motion away from the display.

11. The method according to claim 1 , wherein sensing the position comprises generating a three-dimensional (3D) map of at least a part of body of the user, including the hand.

12. The method according to claim 1 , wherein associating the item with the hand comprises identifying the item based on a gaze of the user.

13. The method according to claim 1 , wherein associating the item with the hand comprises identifying the item based on a pointing gesture performed the user.

14. The method according to claim 1 , wherein presenting the multiple interactive items comprises organizing the multiple interactive items as a grid on the display.

15. The method according to claim 1 , wherein changing the size of the one of the items comprises enlarging the one of the items on the display.

16. A user interface method, comprising:

presenting by a computer executing a user interface, multiple interactive items on a display;

capturing a first sequence of images indicating a position in space of a hand of a user in proximity to the display;

capturing a second sequence of images indicating a movement of the hand transverse to the display in a first direction; and

responsively to the movement, panning the interactive items on the display in a second direction, which is different from the first direction.

17. The method according to claim 16 , wherein the second direction comprises a direction opposite to the first direction.

18. A user interface method, comprising:

presenting by a computer executing a user interface, multiple interactive items on a display;

receiving, from a handheld remote control device coupled to the computer, a first signal indicating a position in space of a hand of a user in proximity to the handheld remote control device;

responsively to the position, associating one of the interactive items with the hand;

after associating the item, receiving from the handheld remote control device a second signal indicating a movement of the hand relative to the handheld remote control device; and

responsively to the movement, changing a size of the one of the interactive items on the display.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 15, 2014
From: FREY, MARTIN; HAUER, MARCUS; BUZZINI, DARIO; SCHAEFER, PHILIP; BERENSON, ADI; GALOR, MICHA; METUKI, NILI; SHPUNT, ALEXANDER
To: PRIMESENSE LTD.
Reel/Frame 033736/0609 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 15, 2014
From: PRIMESENSE LTD.
To: APPLE INC.
Reel/Frame 033736/0655 →
Continuity (5)
Continuation 13541786 · Jul 5, 2012
Provisional Application 61504339 · Jul 5, 2011
Provisional Application 61521448 · Aug 9, 2011
Provisional Application 61523349 · Aug 14, 2011
Related Publication 20140380241A1 · Dec 25, 2014