User eye model match detection
Methods and apparatus for providing eye model matching in a device are disclosed. When a user activates a device and the presence of the user's eye is detected, an image of the user's eye is captured. An eye model matching process is then implemented to determine a stored eye model (e.g., an eye model stored after enrollment of the eye on the device) that best matches the eye in the captured image. Determination of the best matching eye model may be based on matching between properties of the user's eye in the captured image (such as cornea and pupil features) and properties of the user's eye determined by the eye model. The best matching eye model may then be implemented in, for example, an eye gaze tracking process. In certain instances, the best matching eye model satisfies a threshold for matching before being implemented in the downstream process.
1 . A system, comprising:
one or more cameras configured to capture images of an eye;
a memory storing a set of eye models; and
a controller comprising one or more processors configured to:
capture one or more images of the eye by the one or more cameras;
determine one or more properties of the eye from the captured images;
determine a best match eye model from the set of eye models, wherein the best match eye model is the eye model determined to be a best match to the eye based on assessment of the determined eye properties against eye properties of the eye models; and
implement the best match eye model in a gaze tracking process in which features of the eye are derived from one or more images of the eye captured by the one or more cameras and a gaze direction is estimated based on the derived features and the implemented best match eye model.
2 . The system of claim 1 , wherein the controller is configured to implement the best match eye model in the gaze tracking process when the best match eye model satisfies a threshold.
3 . The system of claim 1 , wherein the best match eye model is the eye model with a highest matching score that satisfies a threshold.
4 . The system of claim 3 , wherein a matching score for the eye model is determined based on residual errors between the eye properties of the eye model and the eye properties of the eye determined from the captured images.
5 . The system of claim 1 , wherein the controller is further configured to implement an eye model enrollment process when the best match eye model does not satisfy a threshold.
6 . The system of claim 1 , wherein the controller is configured to determine the best match eye model in the set of eye models in response to detecting a presence of the eye in the captured images.
7 . The system of claim 1 , wherein the one or more properties of the eye determined from the captured images include eye features.
8 . The system of claim 7 , wherein the one or more processors are configured to:
determine two or more eye poses from two or more images captured of the eye in different orientations by the one or more cameras, wherein an eye pose indicates a current eye location and orientation with respect to the one or more cameras; and
calculate the eye features based at least in part on two or more eye poses.
9 . The system of claim 7 , wherein the eye features include cornea contour and pupil features of the eye.
10 . The system of claim 1 , wherein the system is a head-mounted device (HMD), a handheld device, or a wall-mounted device.
11 . The system of claim 1 , further comprising:
one or more second cameras configured to capture images of a second eye;
wherein the controller is further configured to:
capture one or more second images of the second eye by the second cameras;
determine one or more properties of the second eye from the captured second images;
determine a best match second eye model in the set of eye models, wherein the best match second eye model is the eye model determined to be a best match to the second eye based on assessment of the determined second eye properties against eye properties of the eye models;
and implement the best match second eye model in a second gaze tracking process in which features of the second eye are derived from one or more second images of the second eye captured by the one or more second cameras and a second gaze direction is estimated based on the derived features and the implemented best match second eye model.
12 . A method, comprising:
capturing one or more images of an eye using one or more cameras positioned in front of the eye, the cameras being located on an apparatus configured to be positioned proximate a head;
determining, by a computer processor coupled to the head-mounted apparatus, one or more features of the eye from the captured images;
determining, by the computer processor, a matching score for each eye model in a set of eye models stored in a memory coupled to the computer processor, wherein the matching score is determined from a comparison of the determined eye features from the captured images and eye features in the eye model;
determining, by the computer processor, a best match eye model from the set of eye models, wherein the best match eye model is the eye model determined to have a highest matching score; and
implementing, by the cameras in the apparatus, a gaze tracking process, wherein the gaze tracking process utilizes the best match eye model, and wherein the gaze tracking process derives features of the eye from one or more images of the eye captured by the one or more cameras and a gaze direction is estimated based on the derived features and the best match eye model.
13 . The method of claim 12 , further comprising utilizing the best match eye model in the gaze tracking process when a matching score of the best match eye model satisfies a threshold.
14 . The method of claim 12 , wherein the eye features include cornea contour and pupil features of the eye.
15 . The method of claim 12 , wherein the matching score for each eye model is determined based on residual errors between the eye features in the eye model and the eye features determined from the captured images.
16 . The method of claim 12 , wherein determining the one or more features of the eye from the captured images includes:
determining two or more eye poses from two or more captured images of the eye in different orientations, wherein an eye pose indicates a current eye location and orientation with respect to the one or more cameras; and
calculating the eye features based at least in part on two or more eye poses.
17 . A system, comprising:
one or more cameras configured to capture images of a right eye and a left eye;
a memory storing a set of eye models for the right eye and the left eye; and
a controller comprising one or more processors configured to:
capture one or more images of the right eye and the left eye by the one or more cameras;
determine one or more eye features of the right eye from the captured images;
determine a first matching score for each eye model for the right eye in the set of eye models, wherein the matching score is determined from a comparison of the determined eye features of the right eye from the captured images and eye features of the right eye in the eye model;
determine a best match eye model for the right eye from the set of eye models, wherein the best match eye model for the right eye is the eye model determined to have a highest first matching score;
implement a first gaze tracking process for the right eye utilizing the best match eye model for the right eye, wherein the first gaze tracking process derives features of the right eye from one or more images of the right eye captured by the one or more cameras and a gaze direction is estimated based on the derived features and the best match eye model for the right eye;
determine one or more eye features of the left eye from the captured images;
determine a second matching score for each eye model for the left eye in the set of eye models, wherein the matching score is determined from a comparison of the determined eye features of the left eye from the captured images and eye features of the left eye in the eye model;
determine a best match eye model for the left eye from the set of eye models, wherein the best match eye model for the left eye is the eye model determined to have a highest second matching score; and
implement a second gaze tracking process for the left eye utilizing the best match eye model for the left eye, wherein the second gaze tracking process derives features of the left eye from one or more images of the left eye captured by the one or more cameras and a gaze direction is estimated based on the derived features and the best match eye model for the left eye.
18 . The system of claim 17 , wherein the one or more cameras includes at least a first camera and a second camera, the first camera being configured to capture a first image of the right eye and the second camera being configured to capture a second, separate image of the left eye.
19 . The system of claim 17 , further comprising an illumination source comprising a plurality of light-emitting elements configured to emit light towards the right eye and the left eye to be imaged by the camera.
20 . The system of claim 19 , wherein the light-emitting elements include infrared (IR) light sources, and wherein the one or more cameras are infrared cameras.