IP Library Granted Patent US 9,218,063
Granted Patent B2
US 9,218,063 · App. 13/592,352 · Granted Dec 22, 2015

Sessionless pointing user interface

Inventors: Micha Galor (Tel Aviv, IL); Jonathan Pokrass (Bat-Yam, IL); Amir Hoffnung (Tel Aviv, IL); Ofir Or (Ramat Gan, IL)
Assignee: APPLE INC.
G06F3/017G06F3/0304
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,218,063
App. No.
13/592,352
Granted
Dec 22, 2015
Kind
B2
Abstract

A method, including receiving, by a computer, a sequence of three-dimensional maps containing at least a hand of a user of the computer, and identifying, in the maps, a device coupled to the computer. The maps are analyzed to detect a gesture performed by the user toward the device, and the device is actuated responsively to the gesture.

Claims (34)

1. A method, comprising:

identifying, by a computer coupled to a three-dimensional (3D) sensing device and a display, different, respective locations of multiple controllable devices other than the display;

receiving, by the computer from the 3D sensing device, a sequence of three-dimensional maps containing at least a head and a hand of a user of the computer;

detecting in the maps a gaze direction of the user that is directed toward a given device among the multiple controllable devices;

defining a region in space with an apex at the head of the user and a base encompassing the given device;

defining an interaction zone within the region;

defining an angle threshold;

defining a minimum time period;

analyzing the maps to detect a gesture performed by the hand within the defined interaction zone that is directed toward an identified location of the given device among the multiple controllable devices,

wherein the gesture comprises extending an elbow associated with the hand at an angle greater than or equal to the angle threshold, extending the hand toward the device and pausing the hand for the minimum time period; and

actuating the given device responsively to the gesture.

2. The method according to claim 1 , wherein the gesture is selected from a list comprising a pointing gesture, a grab gesture and a release gesture.

3. The method according to claim 1 , and comprising receiving a vocal command from the user, and actuating the device in response to the gesture and the vocal command.

4. The method according to claim 1 , and comprising communicating between the computer and a further device upon detecting, in the maps, a subsequent gesture directed toward the further device.

5. The method according to claim 1 , wherein identifying the respective locations comprises performing, by the computer, an initialization step comprising:

identifying, by the computer, the controllable devices that are in proximity to the 3D sensing device; and

directing the user to point to each of the identified controllable devices.

6. An apparatus, comprising:

a three-dimensional sensing device;

a display; and

a computer configured to identify different, respective locations of multiple controllable devices other than the display, to receive from the three-dimensional sensing device a sequence of three-dimensional maps containing at least a head and a hand of a user of the computer, to detect in the maps a gaze direction of the user that is directed toward a given device among the multiple controllable devices, to define a region in space with an apex at the head of the user and a base encompassing the given device, to define an interaction zone within the region, to analyze the maps to detect a gesture performed by the hand within the defined interaction zone that is directed toward an identified location of the given device among the multiple controllable devices, and to actuate the given device responsively to the gesture,

wherein the computer is configured to define an angle threshold and a minimum time period, and wherein the gesture detected by the computer comprises extending an elbow associated with the hand at an angle greater than or equal to the angle threshold, extending the hand toward the given device, and pausing the hand for the minimum time period.

7. The apparatus according to claim 6 , wherein the computer is configured to select the gesture from a list comprising a pointing gesture, a grab gesture and a release gesture.

8. The apparatus according to claim 6 , wherein the computer is configured to receive a vocal command from the user, and to actuate the given device in response to the gesture and the vocal command.

9. The apparatus according to claim 6 , wherein the computer is configured to communicate with a further device among the multiple controllable devices upon detecting, in the maps, a subsequent gesture directed toward the further device.

10. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a non-tactile user interface and coupled to a display, cause the computer to:

identify different, respective locations of multiple controllable devices other than the display,

receive from a three-dimensional sensing device that is coupled to the computer a sequence of three-dimensional maps containing at least a head and a hand of a user of the computer,

detect in the maps a gaze direction of the user that is directed toward a given device among the multiple controllable devices,

define a region in space with an apex at the head of the user and a base encompassing the given device,

define an interaction zone within the region,

analyze the maps to detect a gesture performed by the hand within the defined interaction zone that is directed toward an identified location of the given device among the multiple controllable devices, and

actuate the given device responsively to the gesture,

wherein the instructions cause the computer to define an angle threshold and a minimum time period, and wherein the gesture detected by the computer comprises extending an elbow associated with the hand at an angle greater than or equal to the angle threshold, extending the hand toward the given device, and pausing the hand for the minimum time period.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICATION # 13840451 AND REPLACE IT WITH CORRECT APPLICATION # 13810451 PREVIOUSLY RECORDED ON REEL 034293 FRAME 0092. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded May 11, 2015
From: PRIMESENSE LTD.
To: APPLE INC.
Reel/Frame 035624/0091 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 19, 2014
From: PRIMESENSE LTD.
To: APPLE INC.
Reel/Frame 034293/0092 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 23, 2012
From: GALOR, MICHA; POKRASS, JONATHAN; HOFFNUNG, AMIR; OR, OFIR
To: PRIMESENSE LTD.
Reel/Frame 028832/0938 →
Continuity (2)
Provisional Application 61526692 · Aug 24, 2011
Related Publication 20130055120A1 · Feb 28, 2013