Image cropping based on eye position for a video-based eye tracker
Systems, methods, and computer programs for eye tracking. In one embodiment, an eye tracking system includes a head-mounted device that takes measurements indicative of a position of the eye of a user. A head-mounted camera captures an image of the eye. A computer calculates the position of the eye based on the measurements, utilizes the position of the eye to crop the image around the pupil, and provides a cropped image to a video-based eye tracker. Optionally, the size of the cropped image is less than a third of the size of its respective uncropped image. Optionally, the head-mounted device includes one or more of the following: a photosensor-oculography device (PSOG), an electrooculography device (EOG), an electromyography device (EMG), an optical flow sensor, and a range sensor. Optionally, the image-based eye tracker calculates, based on the cropped image, at least one of pupil diameter and pupillary response.
1. An eye tracking system comprising:
a head-mounted device configured to take measurements indicative of a position of an eye of a user (eye position);
a head-mounted camera configured to capture an image of the eye; and
a computer configured to:
calculate the eye position based on the measurements;
utilize the eye position to crop the image around the eye's pupil;
provide the cropped image to a video-based eye tracker; and
wherein calculating the eye position based on the measurements is performed at an average rate that is higher than an average rate at which images are captured by the head-mounted camera.
2. The eye tracking system of claim 1 , wherein the device comprises a photosensor-oculography device (PSOG), the measurements are of reflections of light emitted by the PSOG towards the eye, and calculating the eye position based on the measurements is performed at a rate that is at least ten times higher than the rate at which images are captured by the head-mounted camera.
3. The eye tracking system of claim 2 , further comprising head-mounted light sources configured to emit light that generates glints on the eye; and wherein the computer is further configured to utilize the eye position to select a subset of the light sources that are expected to generate one or more glints on the cornea, and operate the subset of the light sources at a higher intensity compared to the rest of the light sources.
4. The eye tracking system of claim 2 , further comprising head-mounted light sources configured to emit light that generates glints on the eye; and wherein the computer is further configured to: calculate positions of the eyelids based on the measurements of the reflections, utilize the positions of the eyelids to select a subset of the light sources that are expected to generate one or more glints on an area of the cornea not covered by the eyelids, and operate the subset of the light sources at a higher intensity compared to the rest of the light sources.
5. The eye tracking system of claim 1 , wherein size of the cropped image is less than a third of size of its respective uncropped image.
6. The eye tracking system of claim 1 , wherein the cropped image covers an area that is not greater than two times the area of a square that surrounds the iris tightly.
7. The eye tracking system of claim 1 , wherein the video-based eye tracker is configured to calculate, based on the cropped image, at least one of pupil diameter and pupillary response.
8. The eye tracking system of claim 1 , wherein the device comprises an electrooculography device, the measurements comprise a value of an electrical potential between electrodes placed close to the eye, and calculating the eye position based on the measurements is performed at a rate that is at least ten times higher than the rate at which images are captured by the head-mounted camera.
9. The eye tracking system of claim 8 , further comprising head-mounted light sources configured to emit light that generates glints on the eye; and wherein the computer is further configured to utilize the eye position to select a subset of the light sources that are expected to generate one or more glints on the cornea, and operate the subset of the light sources at a higher intensity compared to the rest of the light sources.
10. The eye tracking system of claim 1 , wherein the device comprises an electromyography device, the measurements comprise a value of an electrical potential generated by muscle cells, and calculating the eye position based on the measurements is performed at a rate that is at least ten times higher than the rate at which images are captured by the head-mounted camera.
11. The eye tracking system of claim 10 , further comprising head-mounted light sources configured to emit light that generates glints on the eye; and wherein the computer is further configured to utilize the eye position to select a subset of the light sources that are expected to generate one or more glints on the cornea, and operate the subset of the light sources at a higher intensity compared to the rest of the light sources.
12. The eye tracking system of claim 1 , wherein the device comprises an optical flow sensor, the measurements comprise values of optical flow and/or visual motion, the eye position is calculated based on an optical flow algorithm, and calculating the eye position based on the measurements is performed at a rate that is at least ten times higher than the rate at which images are captured by the head-mounted camera.
13. The eye tracking system of claim 12 , further comprising head-mounted light sources configured to emit light that generates glints on the eye; and wherein the computer is further configured to utilize the eye position to select a subset of the light sources that are expected to generate one or more glints on the cornea, and operate the subset of the light sources at a higher intensity compared to the rest of the light sources.
14. The eye tracking system of claim 1 , wherein the device comprises a range sensor, the measurements comprise a value of a range between the range sensor and the eye, and calculating the eye position based on the measurements is performed at a rate that is at least ten times higher than the rate at which images are captured by the head-mounted camera.
15. The eye tracking system of claim 14 , further comprising head-mounted light sources configured to emit light that generates glints on the eye; and wherein the computer is further configured to utilize the eye position to select a subset of the light sources that are expected to generate one or more glints on the cornea, and operate the subset of the light sources at a higher intensity compared to the rest of the light sources.
16. A method comprising:
taking, with a head-mounted device, measurements indicative of a position of an eye of a user (eye position);
capturing an image of the eye by a head-mounted camera;
calculating the eye position based on the measurements at an average rate that is higher than an average rate at which images are captured by the head-mounted camera;
utilizing the eye position for cropping the image around the eye's pupil; and
providing the cropped image to a video-based eye tracker.
17. The method of claim 16 , wherein the device comprises a photosensor-oculography device (PSOG), the measurements are of reflections of light emitted by the PSOG towards the eye, and calculating the eye position based on the measurements is performed at a rate that is at least ten times higher than the rate at which images are captured by the head-mounted camera.
18. The method of claim 16 , wherein the device comprises an electrooculography device, the measurements comprise a value of an electrical potential between electrodes placed close to the eye, and calculating the eye position based on the measurements is performed at a rate that is at least ten times higher than the rate at which images are captured by the head-mounted camera.
19. The method of claim 16 , wherein the device comprises an electromyography device, the measurements comprise a value of an electrical potential generated by muscle cells, and calculating the eye position based on the measurements is performed at a rate that is at least ten times higher than the rate at which images are captured by the head-mounted camera.
20. A non-transitory computer readable medium storing one or more computer programs configured to cause a processor-based system to execute steps comprising:
taking, with a head-mounted device, measurements indicative of a position of an eye of a user (eye position);
capturing an image of the eye by a head-mounted camera;
calculating the eye position based on the measurements at an average rate that is higher than an average rate at which images are captured by the head-mounted camera;
utilizing the eye position for cropping the image around the eye's pupil; and
providing the cropped image to a video-based eye tracker.