Operating an eye tracker camera according to eye position
Systems, methods, and computer programs for efficient use of head-mounted cameras in eye tracking. In one embodiment, a system includes a head-mounted device that takes measurements indicative of positions of an eye of a user, and a head-mounted camera that captures images of the eye. A first range of eye positions trackable from the images is narrower than a second range of eye positions trackable from the measurements. A computer calculates the eye positions based on the measurements. When the eye positions fall within the first range, the computer reads images at a first bitrate from the head-mounted camera. And when the eye positions fall outside the first range, the computer refrains from reading images from the head-mounted camera, or reads the images from the head-mounted camera at a second bitrate that is less than half the first bitrate.
1 . An eye tracking system comprising:
a head-mounted device configured to take measurements indicative of positions of an eye of a user (eye positions);
a head-mounted camera configured to capture images of the eye; whereby a first range of eye positions trackable from the images is narrower than a second range of eye positions trackable from the measurements; and
a computer configured to:
calculate eye positions based on the measurements;
read the images from the head-mounted camera at a first bitrate when the eye positions fall within the first range;
refrain from reading images from the head-mounted camera, or read the images from the head-mounted camera at a second bitrate that is less than half the first bitrate, when the eye positions fall outside the first range; and
wherein the eye positions that fall outside the first range span at least 20° of the eve's field of view, and the computer is further configured to calculate the eye positions based on the measurements at a rate that is higher than a rate at which the images are captured by the head-mounted camera.
2 . The eye tracking system of claim 1 , wherein the computer is further configured to calculate the eye positions based on the measurements at a rate that is at least ten times higher than a rate at which the images are captured by the head-mounted camera, and to command the head-mounted camera to operate in a low-power mode for a longer percent of time while the eye positions fall outside the first range compared to a percent of time it operates in low-power mode while the eye positions fall within the first range.
3 . The eye tracking system of claim 1 , further comprising a second head-mounted camera configured to capture a second set of images of the eye from a different position than the head-mounted camera; and wherein the computer is further configured to read the second set of images from the second head-mounted camera at a higher bitrate when the eye positions fall outside the first range compared to when the eye positions fall within the first range.
4 . The eye tracking system of claim 3 , wherein the computer is further configured to command the head-mounted camera to operate in a low-power mode for a longer percent of time while the eye positions fall outside the first range compared to a percent of time it operates in the low-power mode while the eye positions fall within the first range; and wherein the computer is further configured to command the second head-mounted camera to operate in a low-power mode for a longer percent of time while the eye positions fall within the first range compared to a percent of time it operates in the low-power mode while the eye positions fall outside the first range.
5 . The eye tracking system of claim 3 , wherein the head-mounted camera and the second head-mounted camera are connected to the computer over a bus that does not have sufficient bandwidth to transfer images from both cameras at a maximum bitrate at which the computer is capable of reading from each camera; and wherein the head-mounted camera and the second head-mounted camera are positioned at least 2 cm apart horizontally.
6 . The eye tracking system of claim 1 , wherein the eye is the user's right eye, the head-mounted camera is located to the right of the lateral canthus of the eye, and eye positions in vicinity of the eye's medial canthus are outside the first range.
7 . The eye tracking system of claim 1 , wherein the head-mounted camera is capable of tilting and/or shifting its sensor relative to its optics according to the Scheimpflug principle; and the computer is further configured to command the head-mounted camera to perform the tilting and/or shifting according to the eye positions.
8 . The eye tracking system of claim 1 , wherein the head-mounted device comprises a photosensor-oculography device (PSOG), and the measurements are of reflections of light emitted by the PSOG towards the eye.
9 . The eye tracking system of claim 8 , wherein the computer is further configured to generate feature values based on data comprising measurements of the reflections and the images, and to utilize a model to calculate the eye positions based on the feature values.
10 . The eye tracking system of claim 8 , wherein the computer is further configured to (i) extract a set of eyelid positions based on analyzing the images, and (ii) provide labels that are based on the set of eyelid positions, and corresponding feature values that are based on the emitted light and measurements of the reflections, to train a model for calculating eyelid position based on additional measurements of reflections.
11 . The eye tracking system of claim 1 , wherein the head-mounted device comprises an electrooculography device, and the measurements comprise values of electrical potentials between electrodes placed close to the eye.
12 . The eye tracking system of claim 1 , wherein the head-mounted device comprises an electromyography device, and the measurements comprise values of electrical potentials generated by muscle cells.
13 . The eye tracking system of claim 1 , wherein the head-mounted device comprises an optical flow sensor, the measurements comprise values of optical flow and/or visual motion, and the eye positions are calculated based on an optical flow algorithm.
14 . The eye tracking system of claim 1 , wherein the head-mounted device comprises range sensors, and the measurements comprise values indicative of ranges between the range sensors and the eye.
15 . A method comprising:
taking, with a head-mounted device, measurements indicative of positions of an eye of a user (eye positions);
capturing images of the eye by a head-mounted camera; whereby a first range of eye positions trackable from the images is narrower than a second range of eye positions trackable from the measurements;
calculating eye positions based on the measurements;
reading the images from the head-mounted camera at a first bitrate when the eye positions fall within the first range;
refraining from reading the images from the head-mounted camera, or reading the images from the head-mounted camera at a second bitrate that is less than half the first bitrate, when the eye positions fall outside the first range;
capturing, by a second head-mounted camera, a second set of images of the eye from a different position than the head-mounted camera; and
reading the second set of images from the second head-mounted camera at a higher bitrate when the eye positions fall outside the first range compared to when the eye positions fall within the first range.
16 . The method of claim 15 , further comprising commanding the head-mounted camera to operate in a low-power mode for a longer percent of time while the eye positions fall outside the first range compared to a percent of time it operates in low-power mode while the eye positions fall within the first range.
17 . A non-transitory computer readable medium storing one or more computer programs configured to cause a processor-based system to execute steps comprising:
taking, with a head-mounted device, measurements indicative of positions of an eye of a user (eye positions);
capturing images of the eye by a head-mounted camera; whereby a first range of eye positions trackable from the images is narrower than a second range of eye positions trackable from the measurements;
calculating eye positions based on the measurements;
reading the images from the head-mounted camera at a first bitrate when the eye positions fall within the first range;
refraining from reading the images from the head-mounted camera, or reading the images from the head-mounted camera at a second bitrate that is less than half the first bitrate, when the eye positions fall outside the first range;
capturing, by a second head-mounted camera, a second set of images of the eye from a different position than the head-mounted camera; and
reading the second set of images from the second head-mounted camera at a higher bitrate when the eye positions fall outside the first range compared to when the eye positions fall within the first range.