Methods, systems, and apparatuses for simultaneous eye characteristic tracking and eye model updating
Various implementations disclosed herein include an inconspicuous eye enrollment for user devices through the simultaneous or near-simultaneous eye characteristic tracking and eye model updating. The implementations disclosed herein direct light towards an eye of a user to produce glint reflections and obtain, via one or more sensors, image frames comprising depictions of the glint reflections. Based on those image frames, the implementations disclosed herein track eye characteristics and update an eye model of the eye during a period of time between obtaining a first image frame of the image frames and obtaining a second image frame of the image frames.
1 . A method comprising:
directing light towards an eye using one or more light sources;
obtaining, via one or more sensors, image frames comprising depictions of reflections of the light via the eye; and
based on the image frames, tracking an eye characteristic and updating an at least three-dimensional (3D) eye model of the eye, wherein the tracking and the updating occur during a period of time between obtaining a first image frame of the image frames and obtaining a second image frame of the image frames, and wherein the at least 3D eye model is updated based on a subset of less than all of the image frames, the subset selected based on comparing portions of the eye represented in the subset and portions of the eye already represented in the at least 3D eye model.
2 . The method of claim 1 , wherein there is one image frame between the first image frame the second image frame.
3 . The method of claim 1 , wherein the tracking the eye characteristic and the updating the at least 3D eye model of the eye are both based on a first subset of the image frames each having data that satisfies a threshold amount of eye data.
4 . The method of claim 1 , wherein the image frames comprise a sequence of image frames.
5 . The method of claim 1 , further comprising causing display of a stimulus to attract the eye towards the stimulus.
6 . The method of claim 5 , further comprising estimating a visual axis of the eye based on the display of the stimulus.
7 . The method of claim 1 , wherein the tracking the eye characteristic comprising tracking respective eye glints or respective pupil occupations from the image frames.
8 . The method of claim 7 , further comprising determining, based on the at least three-dimensional eye model, an eye coverage map.
9 . The method of claim 8 , further comprising:
comparing the eye coverage map with the respective eye glints or the respective pupil occupations; and
determining a second subset of image frames associated with an eye glint or a pupil occupation that satisfies a threshold associated with the eye coverage map.
10 . The method of claim 1 , wherein the tracking the eye characteristic and the updating the at least three-dimensional eye model occur with minimum user input.
11 . The method of claim 1 , wherein the obtaining the image frames comprises obtaining images from multiple infrared cameras.
12 . The method of claim 1 , wherein the obtaining the image frames comprises obtaining images from an infrared camera and images reflected off of a mirror.
13 . A system comprising:
a non-transitory computer-readable storage medium; and
one or more processors in communication with the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
directing light towards an eye using one or more light sources;
obtaining, via one or more sensors, image frames comprising depictions of reflections of the light via the eye; and
based on the image frames, tracking an eye characteristic and updating an at least three-dimensional (3D) eye model of the eye, wherein the tracking and the updating occur during a period of time between obtaining a first image frame of the image frames and obtaining a second image frame of the image frames, and wherein the at least 3D eye model is updated based on a subset of less than all of the image frames, the subset selected based on comparing portions of the eye represented in the subset and portions of the eye already represented in the at least 3D eye model.
14 . The system of claim 13 , wherein there is one image frame between the first image frame the second image frame.
15 . The system of claim 13 , wherein the tracking the eye characteristic and the updating the at least 3D eye model of the eye are both based on a first subset of the image frames each having data that satisfies a threshold amount of eye data.
16 . The system of claim 13 , wherein the image frames comprise a sequence of image frames.
17 . The system of claim 13 , wherein the instructions, when executed on the one or more processors, cause the system to perform operations comprising:
causing display of a stimulus to attract the eye towards the stimulus.
18 . The system of claim 17 , wherein the instructions, when executed on the one or more processors, cause the system to perform operations comprising:
estimating a visual axis of the eye based on the display of the stimulus.
19 . The system of claim 13 , wherein tracking the eye characteristic comprising tracking respective eye glints or respective pupil occupations from the image frames.
20 . A non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations comprising:
directing light towards an eye using one or more light sources;
obtaining, via one or more sensors, image frames comprising depictions of reflections of the light via the eye; and
based on the image frames, tracking an eye characteristic and updating an at least three-dimensional (3D) eye model of the eye, wherein the tracking and the updating occur during a period of time between obtaining a first image frame of the image frames and obtaining a second image frame of the image frames, and wherein the at least 3D eye model is updated based on a subset of less than all of the image frames, the subset selected based on comparing portions of the eye represented in the subset and portions of the eye already represented in the at least 3D eye model.