IP Library Granted Patent US 11,262,840
Granted Patent B2
US 11,262,840 · App. 16/012,802 · Granted Mar 1, 2022

Gaze detection in a 3D mapping environment

Inventors: Eyal Bychkov (Hod Hasharon, IL); Oren Brezner (Rishon Lezion, IL); Micha Galor (Tel Aviv, IL); Ofir Or (Ramat Gan, IL); Jonathan Pokrass (Bat Yam, IL); Amir Hoffnung (Tel Aviv, IL); Tamir Berliner (Beit Hashmonay, IL)
G06F3/013G06F3/011G06F3/017G06F2203/0381
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,262,840
App. No.
16/012,802
Granted
Mar 1, 2022
Kind
B2
Abstract

A method, including receiving a three-dimensional (3D) map of at least a part of a body of a user ( 22 ) of a computerized system, and receiving a two dimensional (2D) image of the user, the image including an eye ( 34 ) of the user. 3D coordinates of a head ( 32 ) of the user are extracted from the 3D map and the 2D image, and a direction of a gaze performed by the user is identified based on the 3D coordinates of the head and the image of the eye.

Claims (27)

1. An apparatus, comprising:

at least one sensing device configured to produce a sequence of three dimensional (3D) maps of at least a part of a body of a user of a computerized system and to detect light reflected off an element of an eye of the user; and

a processor coupled to the at least one sensing device and configured to process the 3D maps in order to identify a gesture performed by the user and to identify a direction of a gaze of the user based on the light reflected off the element of the eye, and to control a function of the computerized system responsively to the gesture and the direction of the gaze.

2. The apparatus according to claim 1 , wherein the element is selected from a list comprising a pupil, an iris and a cornea.

3. The apparatus according to claim 1 , wherein the processor is configured to identify the gesture responsively to a position of a finger in the 3D maps.

4. The apparatus according to claim 1 , wherein controlling the function of the computerized system comprises identifying an interactive item presented on a display coupled to the computerized system and in the direction of the gaze, and changing a state of the interactive item.

5. The apparatus according to claim 1 , wherein controlling the function of the computerized system comprises identifying a device coupled to the computerized system and in the direction of the gaze, and changing a state of the device.

6. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer, cause the computer to receive a sequence of three-dimensional (3D) maps of at least a part of a body of a user of the computer, and to receive an input indicative of light reflected off an element of an eye of the user,

wherein the instructions cause the computer to process the 3D maps in order to identify a gesture performed by the user and to identify a direction of a gaze of the user based on the light reflected off the element of the eye, and to control a function of the computerized system responsively to the gesture and the direction of the gaze.

7. A method, comprising:

receiving a sequence of three-dimensional (3D) maps of at least a part of a body of a user of a computerized system;

processing the 3D maps in order to identify a gesture performed by the user;

identifying a direction of a gaze of the user by analyzing light reflected off an element of the eye; and

controlling a function of the computerized system responsively to the gesture and the direction of the gaze.

8. The method according to claim 7 , wherein the element is selected from a list comprising a pupil, an iris and a cornea.

9. The method according to claim 7 , wherein processing the 3D maps comprises identifying a position of a finger in the 3D maps.

10. The method according to claim 9 , wherein identifying the position of the finger comprises detecting a point-select gesture with respect to an interactive item, in which the finger moves toward the interactive item, and then moves along a plane from the interactive item.

11. The method according to claim 9 , wherein identifying the position of the finger comprises detecting a point-touch gesture with respect to an interactive item, in which the finger makes an initial movement toward the interactive item, and then moves in a direction perpendicular to the initial movement.

12. The method according to claim 9 , wherein identifying the position of the finger comprises detecting respective locations of a tip and a knuckle of the finger, and identifying a direction in which the finger is pointing responsively to the detected locations.

13. The method according to claim 7 , wherein controlling the function of the computerized system comprises identifying an interactive item presented on a display coupled to the computerized system and in the direction of the gaze, and changing a state of the interactive item.

14. The method according to claim 13 , wherein the state of the interactive item is changed responsively to the gaze.

15. The method according to claim 13 , wherein the state of the interactive item is changed responsively to a vocal command received from the user.

16. The method according to claim 13 , wherein the state of the interactive item is changed responsively to the gesture.

17. The method according to claim 13 , wherein identifying the position of the finger comprises detecting a pointing motion of a finger, and wherein changing the state comprises moving the identified interactive item on the display in a direction pointed to by the finger.

18. The method according to claim 7 , wherein controlling the function of the computerized system comprises performing an action associated with an interactive item presented in the direction of the gaze on a display coupled to the computerized system, upon detecting that the gesture comprises a motion of a limb toward the display, followed by a deceleration of the motion of the limb toward the display, and then followed by a motion of the limb away from the display.

19. The method according to claim 7 , wherein controlling the function of the computerized system comprises activating, by the computerized system, a power saving technique upon the user directing the gaze away from the display.

20. The method according to claim 7 , wherein controlling the function of the computerized system comprises identifying a device coupled to the computerized system and in the direction of the gaze, and changing a state of the device.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2018
From: BYCHKOV, EYAL; BREZNER, OREN; GALOR, MICHA; OR, OFIR; POKRASS, JONATHAN; HOFFNUNG, AMIR; BERLINER, TAMIR
To: PRIMESENSE LTD.
Reel/Frame 046136/0840 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2018
From: PRIMESENSE LTD.
To: APPLE INC.
Reel/Frame 046136/0860 →
Continuity (7)
Continuation 15256654 · Sep 5, 2016
Continuation 13960822 · Aug 7, 2013
Continuation PCTIB2012050577 · Feb 9, 2012
Provisional Application 61538867 · Sep 25, 2011
Provisional Application 61526692 · Aug 24, 2011
Provisional Application 61440877 · Feb 9, 2011
Related Publication 20180314329A1 · Nov 1, 2018
Cited By (4)
US 12,405,704 US 12,443,286 US 12,657,958 US 12,706,072