Enhanced eye tracking for augmented or virtual reality display systems
Techniques are described for enhanced eye tracking for display systems, such as augmented or virtual reality display systems. The display systems may include a light source configured to output light, a moveable diffractive grating configured to reflect light from the light source, the reflected light forming a scan pattern on the eye of the user, and light detectors to detect light reflected from the eye. The orientation of the diffractive grating can be moved such that the light reflected from the diffractive grating is scanned across the eye according to the scan pattern. Light intensity pattern(s) are obtained via the light detectors, with a light intensity pattern representing a light detector signal obtained by detecting light reflected by the eye as the light is scanned across the eye. One or more physiological characteristics and/or a rotation speed of the eye are determined based on detected light intensity pattern(s).
1. A display system configured to present virtual content to a user, the display system comprising:
a light source configured to output polychromatic light;
a movable reflector configured to reflect the outputted polychromatic light to the eye of the user according to a scan pattern, wherein the scan pattern comprises:
a first leg; and
a second leg, wherein the first leg extends vertically in a first direction and the second leg extends vertically in a second direction, wherein the first direction crosses the second direction, and wherein the first and second legs are formed by light of different wavelengths;
a plurality of light detectors configured to detect reflections of the light scanned across the eye; and
one or more processors configured to perform operations comprising:
causing adjustment of the orientation of the moveable reflector, such that the reflected light is scanned across the eye according to the scan pattern;
obtaining respective light intensity patterns detected by the light detectors, wherein a light intensity pattern represents light detector signals determined at different times during scanning of the reflected light across the eye; and
determining, based on the light intensity patterns, a speed of rotation of the eye.
2. The display system of claim 1 , wherein determining the speed of rotation of the eye includes determining a saccadic velocity of the eye.
3. The display system of claim 2 , wherein the operations further comprise predicting a pose of the eye based on the saccadic velocity.
4. The display system of claim 1 , wherein the movable reflector comprises a diffractive grating, wherein the diffractive grating is configured to convert an incident beam of the polychromatic light from the light source into the scan pattern.
5. The display system of claim 1 , wherein the movable reflector comprises a plurality of diffractive gratings, each diffractive grating configured to form a different light pattern for scanning across the eye.
6. The display system of claim 5 , wherein the diffractive gratings are positioned on, or form part of, a MEMS mirror, and wherein the position information indicates an orientation of the MEMS mirror, the MEMS mirror being adjustable by the display system.
7. The display system of claim 1 , wherein the light detectors are photodiodes, and wherein each light intensity pattern represents a plot of electrical current versus position information associated with a position of the movable reflector.
8. The display system of claim 1 , wherein the light source is one of two light sources configured to output light to the movable reflector, wherein each of the light sources is configured to form a respective portion of the scan pattern of the light for scanning across the eye.
9. The display system of claim 1 , further comprising a waveguide, wherein the waveguide is one of a stack of waveguides, wherein some waveguides of the stack have out-coupling optical elements configured to output light with different amounts of wavefront divergence than out-coupling optical element of other waveguides of the stack, wherein the different amounts of wavefront divergence correspond to different depth planes.
10. The display system of claim 1 , wherein determining the speed of rotation of the eye is based on a difference between successively determined eye poses of the eye that are determined based on at least two successive scans of the eye using the scan pattern.
11. The display system of claim 1 , wherein determining the speed of rotation of the eye is based on a movement of one or more physiological features of the eye that is detected based on at least two successive scans of the eye using the scan pattern.
12. A method implemented by a display system configured to present virtual content to a user, wherein the method comprises:
controlling a movable reflector to direct light onto an eye of the user according to a scan pattern comprising:
a first leg; and
a second leg, wherein the first leg extends vertically in a first direction and the second leg extends vertically in a second direction, wherein the first direction crosses the second direction, and wherein the first and second legs are formed by light of different wavelengths;
obtaining a plurality of light intensity patterns representing light detector signals at different times, the light detector signals obtained from respective light detectors during the directing of the light onto the eye according to the scan pattern; and
determining, based on the light intensity patterns, a speed of rotation of the eye.
13. The method of claim 12 , wherein determining the speed of rotation of the eye includes determining a saccadic velocity of the eye.
14. The method of claim 13 , wherein the operations further comprise predicting a pose of the eye based on the saccadic velocity.
15. The method of claim 12 , wherein directing the light onto the eye according to the scan pattern comprises moving the moveable reflector such that the scan pattern is moved from a first portion of the eye to a second portion of the eye along an axis.
16. The method of claim 15 , wherein the movable reflector comprises a diffractive grating, wherein the diffractive grating is configured to convert an incident beam of light from a light source into the scan pattern.
17. The method of claim 12 , wherein determining the speed of rotation of the eye comprises:
employing at least one machine learning technique to generate a machine learning model associated with a saccadic velocity of the eye.
18. The method of claim 12 , wherein determining the speed of rotation of the eye is based on a difference of a position of at least one physiological characteristic of the eye that is determined based on at least two successive scans of the eye using the scan pattern, and wherein determining the at least one physiological characteristic comprises:
accessing information identifying stored light intensity patterns, the stored light intensity patterns being associated with respective physiological characteristics; and
determining the at least one physiological characteristic based at least partly on comparing the obtained light intensity patterns with the stored light intensity patterns.
19. The method of claim 18 , wherein comparing the obtained light intensity patterns with the stored light intensity patterns includes comparing positions of one or more features in the light intensity patterns, wherein the one or more features include at least one of a peak or a valley.
20. Non-transitory computer storage media storing instructions that when executed by one or more processors, cause the one or more processors to perform operations comprising:
controlling a movable reflector to direct light onto an eye of the user according to a scan pattern comprising:
a first leg; and
a second leg, wherein the first leg extends vertically in a first direction and the second leg extends vertically in a second direction, wherein the first direction crosses the second direction, and wherein the first and second legs are formed by light of different wavelengths;
obtaining a plurality of light intensity patterns representing light detector signals at different times, the light detector signals obtained from respective light detectors during the directing of the light onto the eye according to the scan pattern; and
determining, based on the light intensity patterns, a speed of rotation of the eye.