Compact remote eye tracking system including depth sensing capacity
The present disclosure describes compact remote eye tracking systems that include depth sensing capacity. In one aspect, a method of eye tracking includes illuminating a subject's eye with illumination; detecting, in a depth sensor, optical signals reflected from the subject's face in response to the illumination; generating a depth map based on the optical signals detected by the depth sensor; acquiring an intensity image based on optical signals reflected from the subject's face in response to the illumination; determining, based on a portion of the depth map corresponding to the subject's eye, an estimated distance between the subject's eye and a target screen; determining an estimated gaze direction of the subject's eye based on information obtained from the intensity image; and determining, based on the estimated gaze direction and the estimated distance between the subject's eye and the target screen, a gaze point for the subject's eye.
1. A method of eye tracking comprising:
illuminating a subject's eye with illumination;
detecting, in a depth sensor, optical signals reflected from the subject's face in response to the illumination;
generating a depth map based on the optical signals detected by the depth sensor;
acquiring an intensity image based on optical signals reflected from the subject's face in response to the illumination;
determining, based on a portion of the depth map corresponding to the subject's eye, an estimated distance between the subject's eye and a target screen;
determining an estimated gaze direction of the subject's eye based on information obtained from the intensity image; and
determining, based on the estimated gaze direction and the estimated distance between the subject's eye and the target screen, a gaze point for the subject's eye.
2. The method of claim 1 including acquiring the intensity image using the depth sensor.
3. The method of claim 1 wherein integration timing of the depth sensor is correlated to timing of the illumination.
4. The method of claim 1 including acquiring the intensity image using an infra-red camera different from the depth sensor.
5. The method of claim 4 including:
applying a face segmentation to identify a portion of the intensity image representing the subject's eye; and
determining which signals acquired by the depth sensor correspond to the subject's eye based on a mapping between pixels of the depth sensor and pixels of the infra-red camera.
6. The method of claim 4 wherein integration timings of the depth sensor and the infra-red camera are correlated with timing of the illumination.
7. The method of claim 1 including determining the estimated gaze direction of the subject's eye using an estimated pupil center and an estimated corneal reflection obtained from information in the intensity image.
8. The method of claim 1 wherein the depth sensor comprises a time-of-flight sensor.
9. The method of claim 1 including dynamically controlling a field of the illumination based on a location and distance of the subject's face and/or eyes as determined in one or more previous frames.
10. An eye tracking apparatus comprising:
an illuminator operable to illuminate a subject's eye with illumination;
a depth sensor operable to detect optical signals reflected from the subject's face in response to the illumination;
an image sensor operable to acquire an intensity image based on optical signals reflected from the subject's face in response to the illumination; and
one or more processors configured to:
generate a depth map based on the optical signals detected by the depth sensor;
determine, based on a portion of the depth map corresponding to the subject's eye, an estimated distance between the subject's eye and a target screen;
determine an estimated gaze direction of the subject's eye based on information obtained from the intensity image; and
determine, based on the estimated gaze direction and the estimated distance between the subject's eye and the target screen, a gaze point for the subject's eye.
11. The eye tracking apparatus of claim 10 wherein the depth sensor comprises a time-of-flight sensor.
12. The apparatus of claim 11 wherein the one or more processors are configured to:
determine the estimated gaze direction of the subject's eye using an estimated pupil center and an estimated corneal reflection obtained from information in the intensity image.
13. The apparatus of claim 11 including a controller configured to control a field of the illumination by the illuminator dynamically based on a location and distance of the subject's face and/or eye as determined in one or more previous frames.
14. The eye tracking apparatus of claim 10 wherein both of the depth sensor and the image sensor are implemented by a same time-of-flight sensor.
15. The apparatus of claim 10 wherein integration timing of the depth sensor is correlated to timing of the illumination by the illuminator.
16. The apparatus of claim 10 wherein the image sensor is implemented by an infra-red camera different from the depth sensor.
17. The apparatus of claim 16 wherein the one or more processors are configured to:
apply a face segmentation to identify a portion of the intensity image representing the subject's eye; and
determine which signals acquired by the depth sensor correspond to the subject's eye based on a mapping between pixels of the depth sensor and pixels of the infra-red camera.
18. The apparatus of claim 16 wherein integration timings of the depth sensor and the infra-red camera are correlated with timing of the illumination by the illuminator.