Systems and methods of eye tracking calibration
An image of a user's eyes and/or face, captured by a camera on the computing device or on a device coupled to the computing device, may be analyzed using computer-vision algorithms, such as eye tracking and gaze detection algorithms, to determine the location of the user's eyes and estimate the gaze information associated with the user. A user calibration process may be conducted to calculate calibration parameters associated with the user. These calibration parameters may be taken into account to accurately determine the location of the user's eyes and estimate the location on the display at which the user is looking. The calibration process may include determining a plane on which the user's eyes converge and relating that plane to a plane of a screen on which calibration targets are displayed.
1. A method comprising:
displaying one or more objects in sequence on a display of a computing device in communication with an eye tracking device, each object being associated with a calculation of eye parameters relating to a calculation of gaze information of a user of the computing device;
while each object of the one or more objects is displayed, receiving, from the eye tracking device, a corresponding image of at least one eye of the user;
determining, for each of the one or more corresponding images, eye feature information associated with the at least one eye of the user; and
after all of the one or more objects have been displayed and the corresponding images have been received, calculating one or more of the eye parameters and one or more geometry parameters based on the eye feature information, wherein the one or more geometry parameters indicate information associated with a location and orientation of the display relative to the eye tracking device;
computing a cost function based on the calculated eye parameters and geometry parameters; and
updating the calculated eye parameters and geometry parameters by minimizing the cost function.
2. The method of claim 1 , further comprising:
displaying a second one or more objects in sequence on the display of the computing device;
while each second object of the one or more second objects is displayed, receiving, from the eye tracking device, a corresponding second image of the at least one eye of the user;
determining, for each of the one or more corresponding second images, corresponding second eye information associated with the user based on said each second image, the one or more eye parameters, and the one or more geometry parameters; and
adjusting one or more of the eye parameters or the geometry parameters based on the second eye information and locations of the one or more second objects on the display of the computing device.
3. The method of claim 2 , further comprising:
calculating an error measurement based on the one or more eye parameters, the one or more geometry parameters, the eye feature information of the user, and locations of the one or more objects on the display;
determining that the error measurement exceeds a threshold; and
wherein the displaying of the second one or more objects is based on the error measurement exceeding the threshold.
4. The method of claim 1 , wherein the minimizing of the cost function minimizes an error between an estimated point of regard and display coordinates of the object.
5. The method of claim 1 , wherein the minimizing of the cost function minimizes a distance between a set of visual axes and 3-D coordinates of the object.
6. The method of claim 1 , wherein the one or more objects are a single object and the one or more corresponding images are a single image.
7. A non-transitory machine-readable storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
displaying one or more objects in sequence on a display of a computing device in communication with an eye tracking device, each object being associated with a calculation of eye parameters relating to a calculation of gaze information of a user of the computing device;
while each object of the one or more objects is displayed, receiving, from the eye tracking device, a corresponding image of at least one eye of the user;
determining, for each of the one or more corresponding images, eye feature information associated with the at least one eye of the user; and
after all of the one or more objects have been displayed and the corresponding images have been received, calculating one or more of the eye parameters and one or more geometry parameters based on the eye feature information, wherein the one or more geometry parameters indicate information associated with a location and orientation of the display relative to the eye tracking device;
computing a cost function based on the calculated eye parameters and geometry parameters; and
updating the calculated eye parameters and geometry parameters by minimizing the cost function.
8. The non-transitory machine-readable storage medium of claim 7 , wherein the operations further comprise:
displaying a second one or more objects in sequence on the display of the computing device;
while each second object of the one or more second objects is displayed, receiving, from the eye tracking device, a corresponding second image of the at least one eye of the user;
determining, for each of the one or more corresponding second images, corresponding second eye information associated with the user based on the second image, the one or more eye parameters, and the one or more geometry parameters; and
adjusting one or more of the eye parameters or the geometry parameters based on the second eye information and locations of the one or more second objects on the display of the computing device.
9. The non-transitory machine-readable storage medium of claim 8 , wherein the operations further comprise:
calculating an error measurement based on the eye parameters, the geometry parameters, the eye feature information of the user, and locations of the one or more objects on the display;
determining that the error measurement exceeds a threshold; and
wherein the displaying of the second one or more objects is based on the error measurement exceeding the threshold.
10. The non-transitory machine-readable storage medium of claim 7 , wherein the minimizing of the cost function minimizes an error between an estimated point of regard and display coordinates of the object.
11. The non-transitory machine-readable storage medium of claim 7 , wherein the minimizing of the cost function minimizes a distance between a set of visual axes and 3-D coordinates of the object.
12. The non-transitory machine-readable storage medium of claim 7 , wherein the one or more objects are a single object and the one or more corresponding images are a single image.
13. A device comprising:
a display;
a hardware-implemented display module configured to:
display one or more objects in sequence on a display of a computing device in communication with an eye tracking device, each object being associated with a calculation of eye parameters relating to a calculation of gaze information of a user of the computing device; and
a hardware-implemented eye tracking module configured to:
while each object of the one or more objects is displayed, receive, from the eye tracking device, a corresponding image of at least one eye of the user;
determine, for each of the one or more corresponding images, eye feature information associated with the at least one eye of the user; and
after all of the one or more objects have been displayed and the corresponding images have been received, calculate one or more of the eye parameters and one or more geometry parameters based on the eye feature information, wherein the one or more geometry parameters indicate information associated with a location and orientation of the display relative to the eye tracking device;
compute a cost function based on the calculated eye parameters and geometry parameters; and
update the calculated eye parameters and geometry parameters by minimizing the cost function.
14. The device of claim 13 , wherein:
the hardware-implemented display module is further configured to:
display a second one or more objects in sequence on the display of the device;
the hardware-implemented eye tracking module is further configured to:
while each second object of the second one or more objects is displayed, receive, from the eye tracking device, a corresponding second image of the at least one eye of the user;
determine, for each of the one or more corresponding second images, corresponding second eye information associated with the user based on the second image and the one or more eye parameters, and the one or more geometry parameters; and
adjust one or more of the eye parameters or the geometry parameters based on the second eye information and locations of the second one or more objects on the display of the device.
15. The device of claim 14 , wherein the hardware-implemented eye tracking module is further configured to:
calculate an error measurement based on the one or more eye parameters, the one or more geometry parameters, the eye feature information of the user, and locations of the one or more objects on the display;
determine that the error measurement exceeds a threshold; and
wherein the displaying of the second one or more objects is based on the error measurement exceeding the threshold.
16. The device of claim 13 , wherein the minimizing of the cost function minimizes an error between an estimated point of regard and display coordinates of the object.
17. The device of claim 13 , wherein the minimizing of the cost function minimizes a distance between a set of visual axes and 3-D coordinates of the one or more objects.
18. The device of claim 13 , wherein the one or more objects are a single object and the one or more corresponding images are a single image.