Method and system for glint-based eye detection in a remote eye tracking system
The invention is related a method and a remote eye tracking system for determining a position of at least one eye of a subject. The method comprises the steps of: illuminating a face of a subject using at least one infrared (IR) illuminator off-axis from at least one image sensor; capturing at least one image of the face using the at least one image sensor at a time instant; and using a processing circuitry of the remote eye tracking system, performing processing steps of: determining at least one pupil candidate associated with the at least one image; determining at least one glint candidate associated with the at least one image; determining at least one pupil-glint candidate group, comprising at least one pupil candidate and at least one corresponding glint candidate; generating a score value for each of the at least one pupil-glint candidate groups; and determining an eye position of at least one eye of the subject in the at least one image based on the pupil-glint candidate group with the highest score value.
1. A method for determining a position of at least one eye of a subject in images captured by a remote eye tracking system, the method comprising:
illuminating a face of a subject using at least one infrared (IR) illuminator off-axis from at least one image sensor;
capturing at least one image of the face using the at least one image sensor at a time instant; and
using a processing circuitry of the remote eye tracking system, performing processing steps of:
determining at least one pupil candidate associated with the at least one image;
determining at least one glint candidate associated with the at least one image, wherein the at least one glint candidate is determined by detection of a bright area of pixels in the at least one image;
determining at least one further glint candidate associated with the at least one image, wherein determining the at least one further glint candidate comprises determining an angle of a line between the at least one glint candidate and the at least one further glint candidate satisfying an angle of line threshold;
determining at least one pupil-glint candidate group, comprising at least one pupil candidate and at least one corresponding glint candidate;
generating a score value for each of the at least one pupil-glint candidate groups; and
determining an eye position of at least one eye of the subject in the at least one image based on the pupil-glint candidate group with the highest score value.
2. The method of claim 1 , wherein the brightness of said bright area is larger than a glint brightness threshold.
3. The method of claim 1 , wherein the radius of said bright area satisfies a glint radius threshold.
4. The method of claim 1 , wherein the pupil candidate is determined by detection of a bright area of pixels in an inverted at least one image.
5. The method of claim 4 , wherein the brightness of the bright area of pixels in the inverted at least one image is larger than a pupil brightness threshold.
6. The method of claim 4 , wherein the radius of said bright or dark area satisfies a pupil radius threshold.
7. The method of claim 1 , wherein the pupil candidate is determined by detection of a dark area of pixels in the at least one image.
8. The method of claim 7 , wherein the brightness of said dark area is smaller than the pupil brightness threshold.
9. The method of claim 1 , wherein in the method, a glint candidate is considered to be adjacent to a pupil candidate if the distance between the glint and the pupil candidates is below a distance threshold.
10. The method of claim 1 , wherein the step of generating the score value for each of the pupil-glint candidate groups is based on determining at least one of a distance between each glint candidate and each pupil candidate or brightness of the glint candidate.
11. The method of claim 1 , wherein the maximum number of glint candidates for each pupil candidates in the pair of pupil-glint candidate groups corresponds to a number of IR illuminators in the remote eye tracking system.
12. The method of claim 1 , wherein the method is performed by capturing a first image using a first image sensor and a second image using a second image sensor, the first and second images being captured at the time instant, producing stereoscopic images.
13. The method of claim 12 , wherein generating the score value is based on determining a distance between each pupil-glint candidate group in stereoscopic images respectively.
14. The method of claim 1 , wherein the at least one image is a full-frame image.
15. A non-transitory computer-readable storage medium storing instructions which, when executed by processing circuitry of a remote eye tracking system, cause the eye tracking system to perform the method steps of claim 1 .
16. A remote eye tracking system for determining a position of at least one eye of a subject in images captured by at least one image sensor associated with the remote eye tracking system, the remote eye tracking system comprising at least one infrared (IR) illuminator off-axis from at least one image sensor and comprising a processing circuitry being configured to:
determine at least one pupil candidate associated with the at least one image;
determine at least one glint candidate associated with the at least one image, wherein the at least one glint candidate is determined by detection of a bright area of pixels in the at least one image;
determine at least one further glint candidate associated with the at least one image, wherein determining the at least one further glint candidate comprises determining an angle of a line between the at least one glint candidate and the at least one further glint candidate satisfying an angle of line threshold;
determine at least one pupil-glint candidate group, comprising at least one pupil candidate and at least one corresponding glint candidate;
generate a score value for each of the at least one pupil-glint candidate groups; and
determine an eye position of at least one eye of the subject in the at least one image based on the pupil-glint candidate group with the highest score value.
17. The remote eye tracking system of claim 16 , wherein the system is configured to be used with a subject being a non-human primate (NHP) subject, for example macaques, chimpanzees, bonobos or gorillas.