IP Library Granted Patent US 10,855,938
Granted Patent B2
US 10,855,938 · App. 16/532,254 · Granted Dec 1, 2020

Gaze tracking variations using selective illumination

Inventor: Eric J. Larsen (Pacifica, CA)
Assignee: SONY INTERACTIVE ENTERTAINMENT INC.
H04N5/33G06K9/00604H04N5/2351H04N5/2354G06K9/2027
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,855,938
App. No.
16/532,254
Granted
Dec 1, 2020
Kind
B2
Abstract

Aspects of the present disclosure relate to eye tracking systems and methods which track eyes by illuminating the eyes using a light source and detecting the eye illuminations using a sensor. Implementations of the present disclosure may utilize wide angle lighting via a plurality of individual light sources which are each oriented in different orientations. A wide area may be illuminated by the different light sources, and these light sources may be selectively turned on and off based on a current location of a user.

Claims (81)

1. a) emitting light from an eye tracking device, wherein said emitting the light includes initially emitting the light from all light sources of a plurality of light sources of the eye tracking device, each said light source of the plurality being oriented to emit light at a different angle;

b) detecting an area illuminated by the light with a sensor of the eye tracking device;

identifying one or more of the light sources of the plurality as contributing light sources from analysis of data gathered from said detecting the area, said identifying the light sources as the contributing light sources being based on a determination that the contributing light sources illuminate one or more eyes, wherein identifying one or more of the lights sources of the plurality as contributing light sources comprises pulsing one or more of the plurality at a different pulse frequency than a pulse frequency of the rest of the light sources of the plurality and detecting the presence of reflections at the different pulse frequency or the pulse frequency of the rest of the light sources of the plurality;

c) turning off one or more light sources of the plurality not identified as the contributing light sources while emitting the light from the contributing light sources; and

subsequently repeating a), b), and c) one or more times.

2. The method of claim 1 ,

wherein the sensor is a camera,

wherein the data is one or more images, and

wherein said detecting the area includes capturing the images with the camera.

3. The method of claim 1 ,

wherein the sensor is a camera,

wherein the data is one or more images,

wherein said detecting the area includes capturing the images with the camera, and

wherein said identifying the contributing light sources includes identifying a location of one or more illuminated eyes within the images and identifying the contributing light sources based on correspondence between the location and known orientations of each said light source.

4. The method of claim 1 ,

wherein the sensor is a camera,

wherein the data is one or more images,

wherein said detecting the area includes capturing the images with the camera, and

wherein the method further comprises, after said turning off the light sources, analyzing the images to determine one or more gaze characteristics of the one or more eyes illuminated by said emitting the light.

5. The method of claim 1 ,

wherein each said light source is an infrared LED,

wherein the sensor is an infrared camera,

wherein the data is one or more infrared images, and

wherein said detecting the area includes capturing the infrared images with the infrared camera.

6. The method of claim 1 , wherein said turning off the light sources includes sending a control signal to the eye tracking device which causes said turning off of the light sources.

7. A system comprising:

an eye tracking device having a sensor and a plurality of light sources, each said light source being oriented to emit light at a different angle; and

a computing device, wherein the system is configured to perform a method, the method comprising:

a) emitting light from the eye tracking device, wherein said emitting the light includes initially emitting the light from all light sources of the plurality of light sources;

b) detecting an area illuminated by the light with the sensor;

c) identifying, with the computing device, one or more of the light sources of the plurality as contributing light sources from analysis of data gathered from said detecting the area, said identifying the light sources of the plurality as the contributing light sources being based on a determination that the contributing light sources illuminate one or more eyes, wherein identifying one or more of the lights sources of the plurality as contributing light sources comprises pulsing one or more of the plurality at a different pulse frequency than a pulse frequency of the rest of the light sources of the plurality and detecting the presence of reflections at the different pulse frequency or the pulse frequency of the rest of the light sources of the plurality;

d) turning off one or more light sources of the plurality not identified as the contributing light sources while emitting the light from the contributing light sources; and

subsequently repeating a), b), c) and d) one or more times.

8. The system of claim 7 , wherein each said light source is an infrared LED.

9. The system of claim 7 , wherein each said light source is an infrared LED, wherein each said infrared LED is fixed to a casing of the eye tracking device at a different angle, wherein each infrared LED is configured to emit light in a different non-overlapping region in front of the eye tracking device.

10. The system of claim 7 ,

wherein the sensor is a camera,

wherein the data is one or more images, and

wherein said detecting the area includes capturing the images with the camera.

11. The system of claim 7 ,

wherein the sensor is a camera,

wherein the data is one or more images,

wherein said detecting the area includes capturing the images with the camera, and

wherein said identifying the contributing light sources includes identifying a location of one or more illuminated eyes within the images and identifying the contributing light sources based on correspondence between the location and known orientations of each said light source.

12. The system of claim 7 ,

wherein the sensor is a camera,

wherein the data is one or more images,

wherein said detecting the area includes capturing the images with the camera, and

wherein the method further comprises, after said turning off the light sources, analyzing the images to determine one or more gaze characteristics of the one or more eyes illuminated by said emitting the light.

13. The system of claim 7 ,

wherein each said light source is an infrared LED,

wherein the sensor is an infrared camera,

wherein the data is one or more infrared images, and

wherein said detecting the area includes capturing the infrared images with the infrared camera.

14. The system of claim 7 , wherein said turning off the light sources includes sending a control signal from the computing device to the eye tracking device which causes said turning off of the light sources.

15. A non-transitory computer readable medium having processor-executable instructions embodied therein, wherein execution of the instructions by a processor causes the processor to perform a method, the method comprising:

a) emitting light from an eye tracking device, wherein said emitting the light includes initially emitting the light from all light sources of a plurality of light sources of the eye tracking device, each said light source being oriented to emit light at a different angle;

b) detecting an area illuminated by the light with a sensor of the eye tracking device;

identifying one or more of the light sources of the plurality as contributing light sources from analysis of data gathered from said detecting the area, said identifying the one or more light sources of the plurality as the contributing light sources being based on a determination that the contributing light sources illuminate one or more eyes, wherein identifying one or more of the lights sources of the plurality as contributing light sources comprises pulsing one or more of the plurality at a different pulse frequency than a pulse frequency of the rest of the light sources of the plurality and detecting the presence of reflections at the different pulse frequency or the pulse frequency of the rest of the light sources of the plurality;

c) turning off one or more light sources of the plurality not identified as the contributing light sources while emitting the light from the contributing light sources; and

subsequently repeating a), b), and c) one or more times.

16. The non-transitory computer readable medium of claim 15 ,

wherein the sensor is a camera,

wherein the data is one or more images, and

wherein said detecting the area includes capturing the images with the camera.

17. The non-transitory computer readable medium of claim 15 ,

wherein the sensor is a camera,

wherein the data is one or more images,

wherein said detecting the area includes capturing the images with the camera, and

wherein said identifying the contributing light sources includes identifying a location of one or more illuminated eyes within the images and identifying the contributing light sources based on correspondence between the location and known orientations of each said light source.

18. The non-transitory computer readable medium of claim 15 ,

wherein the sensor is a camera,

wherein the data is one or more images,

wherein said detecting the area includes capturing the images with the camera, and

wherein the method further comprises, after said turning off the light sources, analyzing the images to determine one or more gaze characteristics of the one or more eyes illuminated by said emitting the light.

19. The non-transitory computer readable medium of claim 15 ,

wherein each said light source is an infrared LED,

wherein the sensor is an infrared camera,

wherein the data is one or more infrared images, and

wherein said detecting the area includes capturing the infrared images with the infrared camera.

20. The non-transitory computer readable medium of claim 15 , wherein said turning off the light sources includes sending a control signal to the eye tracking device which causes said turning off of the light sources.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2019
From: LARSEN, ERIC J.
To: SONY COMPUTER ENTERTAINMENT INC.
Reel/Frame 049962/0716 →
CHANGE OF NAME Recorded Aug 5, 2019
From: SONY COMPUTER ENTERTAINMENT INC.
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 049964/0955 →
Continuity (4)
Continuation 15713466 · Sep 22, 2017
Continuation 14493738 · Sep 23, 2014
Provisional Application 61881660 · Sep 24, 2013
Related Publication 20190364228A1 · Nov 28, 2019