IP Library Granted Patent US 10,591,990
Granted Patent B2
US 10,591,990 · App. 16/232,591 · Granted Mar 17, 2020

Fast wake-up in gaze tracking system

Inventors: Mårten Skogö (Danderyd, SE); Anders Olsson (Danderyd, SE); John Mikael Elvesjö (Danderyd, SE); Aron Yu (Danderyd, SE)
Assignee: Tobii AB
G06F3/013G06F1/325
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,591,990
App. No.
16/232,591
Granted
Mar 17, 2020
Kind
B2
Abstract

A gaze tracking system, leaving a low power mode in response to an activation signal, initial burst of eye pictures in short time by restricting the image area of a sensor, purpose of enabling an increased frame rate. Subsequent eye pictures are captured at le rate. The first gaze point value is computed memorylessly based on the initial burst res and no additional imagery, while subsequent values may be computed recursively to account previous gaze point values or information from previous eye pictures. The restriction of the image area may be guided by a preliminary overview picture captured using the different sensor. From the gaze point values, the system may derive a control signal to a computer device with a visual display.

Claims (52)

1. A method in a gaze tracking system for controlling a computer device associated with a visual display, wherein the gaze tracking system tracks a gaze point on said visual display, said tracking system comprising a first sensor with a light-sensitive surface, wherein:

in response to a gaze tracking activation signal from input means of said computer device at said gaze tracking system, leaving a low power mode of said gaze tracking system;

capturing, an overview picture of at least a portion of the head of the viewer, and locating an eye region of the viewer in the overview picture, wherein the overview picture is captured as a depth map by a second sensor either alone or in combination with the first sensor;

capturing, at a first frame rate, a plurality of pictures of at least a portion of the viewer's head using said first sensor;

determining a first gaze point value of said viewer based on information from said plurality of pictures and optionally on additional information from the overview picture;

providing a first control signal based on said first gaze point value;

capturing, at a second frame rate, being lower than said first frame rate, further pictures of at least a portion of the viewer's head using a smaller portion or a restricted region of said first sensor;

determining a further gaze point value of said viewer based on information from at least one of said further pictures; and

providing a further control signal based on said further gaze point.

2. The method according to claim 1 , wherein:

said first sensor includes a light-sensitive surface arranged to receive light from the head of the viewer;

said plurality of pictures is captured by reading out data from said first sensor in restricted regions of a light-sensitive surface; and

said further pictures are captured by reading out data from a less restricted or unrestricted region of the light-sensitive surface.

3. The method according to claim 1 , wherein:

said first sensor includes a light-sensitive surface arranged to receive light from the head of the viewer;

said plurality of pictures is captured by reading out data from said first sensor in restricted regions of a light-sensitive surface, which are positioned in accordance with the eye region location and the union of which occupies at most 10 percent of the area of the light-sensitive surface; and

said further pictures are captured by reading out data from a less restricted or unrestricted region of the light-sensitive surface.

4. The method according to claim 1 , wherein the determination of the first gaze point is based on information from said plurality of pictures and on additional information from the overview picture.

5. The method according to claim 1 , the determination of a further gaze point is based on information from said further picture and additionally on information from at least one previous picture.

6. The method according to claim 1 , wherein the restricted region comprises eye region location derived from the overview picture.

7. The method according to claim 1 , wherein the first sensor is CMOS-type sensor and the second sensor is CMOS-type sensor or a time of flight sensor.

8. The method according to claim 1 , wherein the depth map is created by triangulation of the outputs of the first and second sensor.

9. The method according to claim 1 , wherein the depth map is created by operating the second sensor while simultaneously illuminating the viewer's head by structured or coded light from a light source.

10. The method according to claim 9 , wherein the first sensor is a higher-resolution CMOS-type sensor and the second sensor is a lower-resolution CMOS-type sensors, wherein a depth map is created by operating the second sensor while simultaneously illuminating the viewer's head by structured or coded light from a light source.

11. The method according to claim 1 , wherein the first sensor is CMOS-type sensor and the second sensor is a time-of-flight sensor.

12. A gaze tracking system for controlling a computer device associated with a visual display, wherein the gaze tracking system tracks a gaze point on said visual display, the system comprising:

a first sensor with a light-sensitive surface arranged to receive light from the head of a viewer, the first sensor being selectively operable at a second frame rate or a first frame rate; and

a processor configured to respond to receipt, in a low power mode of the eye tracking system, of a gaze tracking activation signal by performing the following:

causing the eye tracking system to leave the low power mode;

capturing an overview picture of at least a portion of the head of the viewer and locating an eye region of the viewer in the overview picture, wherein the overview picture is captured as a depth map by a second sensor either alone or in combination with the first sensor;

capturing a plurality of pictures of at least a portion of the viewer's head using said first sensor, from which data are read out at a first frame rate;

determining a first gaze point value of said viewer based on information from said plurality of pictures;

providing a first control signal based on said first gaze point value;

capturing further pictures of at least a portion of the viewer's head using a smaller portion of the or a restricted region of said first sensor, from which data are read out at a second frame rate, being lower than said first frame rate;

determining, for each further picture, a further gaze point value of said viewer on the basis of at least one of said further pictures; and

providing a further control signal based on said further gaze point.

13. A gaze tracking system according to claim 12 , wherein:

the first sensor receives light from the head of a viewer of the visual display, and the first sensor is selectively operable to read out the light-sensitive surface at a second frame rate or a restricted region at a first frame rate, being higher than said second frame rate; and

wherein the capturing of the plurality of pictures from which data are read out at a first frame rate and in restricted regions;

wherein the determining a first gaze point value of said viewer on the visual display based on information from said plurality of pictures and optionally on additional information from the overview picture; and

determining, for each further picture, a further gaze point value of said viewer on the basis said further picture and additionally of at least one previous picture.

14. The system according to claim 12 , wherein the first sensor is CMOS-type sensor and the second sensor is CMOS-type sensor or a time of flight sensor.

15. The system according to claim 12 , wherein the depth map is configured to be created by triangulation of the outputs of the first and second sensor.

16. The method according to claim 15 , wherein the first sensor is a higher-resolution CMOS-type sensor and the second sensor is a lower-resolution CMOS-type sensors, wherein a depth map is configured to be created by operating the second sensor while simultaneously illuminating the viewer's head by structured or coded light from a light source.

17. The method according to claim 12 , wherein the depth map is configured to be created by operating the second sensor while simultaneously illuminating the viewer's head by structured or coded light from a light source.

18. The method according to claim 12 , wherein the first sensor is CMOS-type sensor and the second sensor is a time-of-flight sensor.

19. A non-transitory computer readable medium having instructions stored thereon executable by a computing device for controlling a computer device associated with a visual display, the gaze tracking system tracking a gaze point on said visual display, that when executed cause the computing device to perform operations comprising:

leaving a low power mode of said gaze tracking system in response to a gaze tracking activation signal from input means of said computer device at said gaze tracking system;

capturing, an overview picture of at least a portion of the head of the viewer, and locating an eye region of the viewer in the overview picture, wherein the overview picture is captured as a depth map by a second sensor either alone or in combination with the first sensor;

capturing, at a second frame rate, a plurality of pictures of at least a portion of the viewer's head using a smaller portion or a restricted region of said first sensor;

determining a first gaze point value of said viewer based on information from said plurality of pictures and optionally on additional information from the overview picture;

providing a first control signal based on said first gaze point value.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 26, 2018
From: SKOGÖ, MÅRTEN; OLSSON, ANDERS; ELVESJÖ, JOHN MIKAEL; YU, ARON
To: TOBII TECHNOLOGY AB
Reel/Frame 047854/0235 →
CHANGE OF NAME Recorded Dec 26, 2018
From: TOBII TECHNOLOGY AB
To: TOBII AB
Reel/Frame 048000/0989 →
Priority Claims (1)
EP 12179867 · Aug 9, 2012 · regional
Continuity (3)
Continuation 15707030 · Sep 18, 2017
Continuation 13962151 · Aug 8, 2013
Related Publication 20190265786A1 · Aug 29, 2019