Gaze tracking for a retinal projection display system
A retinal projection display system includes at least one visible light source for projecting a visible light image, an infrared light source for projecting infrared light, a scanning mirror having a field of view larger than the visible light image, a reflective surface on which the visible light image is projected and on which the infrared light is reflected at least partially towards an eye of a user, wherein the reflective surface is larger than the visible light image, at least one infrared photodetector for receiving reflected infrared light that reflects off of the eye of the user, and a hardware computation module comprising a processor and a memory, the hardware computation module configured to determine a gaze direction of the user based at least in part on the reflected infrared light.
1 . A retinal projection display system comprising:
at least one visible light source for projecting a visible light image;
an infrared light source for projecting infrared light;
a scanning mirror having a field of view larger than the visible light image;
a reflective surface on which the visible light image is projected and on which the infrared light is reflected at least partially towards an eye of a user, wherein the reflective surface is larger than the visible light image, wherein the visible light image is projected onto the reflective surface using a first scanning pattern and the infrared light is projected onto the reflective surface using a second scanning pattern, and wherein the first scanning pattern and the second scanning pattern are different scanning patterns;
at least one infrared photodetector for receiving reflected infrared light that reflects off of the eye of the user; and
a hardware computation module comprising a processor and a memory, the hardware computation module configured to determine a gaze direction of the user based at least in part on the reflected infrared light, the hardware computation module further configured to coordinate operation of scanning mirror and the at least one visible light source for projecting the visible light image onto the reflective surface using the first scanning pattern based on the gaze direction such that the visible light image is directed through a pupil and projected onto a retina of the user.
2 . The retinal projection display system of claim 1 , wherein the at least one visible light source and the infrared light source are in alignment, such that the hardware computation module is further configured to control the scanning mirror to project the visible light image onto the reflective surface toward the gaze direction.
3 . The retinal projection display system of claim 1 , wherein the at least one visible light source and the infrared light source are not in alignment, such that the hardware computation module is further configured to control the scanning mirror to compensate for displacement between the at least one visible light source and the infrared light source to determine the gaze direction and to project the visible light image onto the reflective surface toward the gaze direction.
4 . The retinal projection display system of claim 3 , wherein a pupillary distance alignment is determined during a calibration operation for the user, wherein the pupillary distance alignment identifies a viewable region of the reflective surface for a known gaze direction of the user.
5 . The retinal projection display system of claim 4 , wherein the displacement between the at least one visible light source and the infrared light source is based at least in part on the gaze direction and the pupillary distance alignment.
6 . The retinal projection display system of claim 3 , wherein the displacement between the at least one visible light source and the infrared light source is determined during a manufacturing calibration operation and stored in the memory.
7 . The retinal projection display system of claim 1 , wherein the at least one visible light source comprises a plurality of visible light sources, and wherein a visible light source displacement between the plurality of visible light sources is determined during a manufacturing calibration operation and stored in the memory.
8 . The retinal projection display system of claim 7 , wherein the hardware computation module is configured to align the plurality of visible light sources based at least in part on the visible light source displacement.
9 . The retinal projection display system of claim 1 , wherein the hardware computation module is further configured to:
scan the infrared light over the field of view of the reflective surface;
receive reflected infrared light that reflects off of the eye of the user at the at least one infrared photodetector;
measure an amount of the reflected infrared light over the field of view of the scanning mirror on the reflective surface; and
map the amount of the reflected infrared light over the field of view of the scanning mirror on the reflective surface to generate an infrared reflectivity map of the field of view of the scanning mirror, wherein the infrared reflectivity map identifies the gaze direction.
10 . The retinal projection display system of claim 1 , wherein the reflective surface is at least partially transparent.
11 . The retinal projection display system of claim 1 , further comprising:
an eyeglasses frame configured to be worn by the user; and
at least one lens mounted in the eyeglasses frame, where the reflective surface is positioned on at least a portion of the at least one lens.
12 . The retinal projection display system of claim 11 , wherein the at least one infrared photodetector is positioned on the eyeglasses frame.
13 . The retinal projection display system of claim 11 , wherein the at least one infrared photodetector is positioned inside a module comprising the at least one visible light source and the infrared light source.
14 . A method of retinal projection display, the method comprising:
projecting a visible light image from at least one visible light source onto a reflective surface using a scanning mirror having a field of view larger than the visible light image, wherein the reflective surface is larger than the visible light image, wherein the visible light image is projected onto the reflective surface using a first scanning pattern;
projecting infrared light from an infrared light source onto the reflective surface using the scanning mirror, wherein the infrared light is projected over the field of view of the scanning mirror and reflected off the reflective surface at least partially towards an eye of a user, wherein the infrared light is projected onto the reflective surface using a second scanning pattern, and wherein the first scanning pattern and the second scanning pattern are different scanning patterns;
receiving reflected infrared light that reflects off of the eye of the user at at least one infrared photodetector;
determining a gaze direction of the user based at least in part on the reflected infrared light; and
coordinating operation of scanning mirror and the at least one visible light source for projecting the visible light image onto the reflective surface using the first scanning pattern based on the gaze direction such that the visible light image is directed through a pupil and projected onto a retina of the user.
15 . The method of claim 14 , wherein the at least one visible light source and the infrared light source are in alignment, the method further comprising:
controlling the scanning mirror to project the visible light image onto the reflective surface toward the gaze direction.
16 . The method of claim 14 , wherein the at least one visible light source and the infrared light source are not in alignment, the method further comprising:
determining displacement between the at least one visible light source and the infrared light source; and
controlling the scanning mirror to compensate for the displacement between the at least one visible light source and the infrared light source to determine the gaze direction and to project the visible light image onto the reflective surface toward the gaze direction.
17 . The method of claim 16 , wherein the determining the displacement between the at least one visible light source and the infrared light source comprises:
retrieving a pupillary distance alignment for the user, wherein the pupillary distance alignment identifies a viewable region of the reflective surface for a known gaze direction of the user, and wherein the displacement between the at least one visible light source and the infrared light source is based at least in part on the gaze direction and the pupillary distance alignment.
18 . The method of claim 16 , wherein the determining the displacement between the at least one visible light source and the infrared light source comprises:
retrieving the displacement between the at least one visible light source and the infrared light source, wherein the displacement between the at least one visible light source and the infrared light source is determined during a manufacturing calibration operation and stored in a memory unit.
19 . The method of claim 14 , wherein the at least one visible light source comprises a plurality of visible light sources, and wherein a visible light source displacement between the plurality of visible light sources is determined during a manufacturing calibration operation and stored in a memory unit.
20 . The method of claim 19 , the method further comprising:
aligning the plurality of visible light sources based at least in part on the visible light source displacement.
21 . The method of claim 14 , wherein the determining a gaze direction of the user based at least in part on the reflected infrared light comprises:
measuring an amount of the reflected infrared light over the field of view of the scanning mirror on the reflective surface; and
mapping the amount of the reflected infrared light over the field of view of the scanning mirror on the reflective surface to generate an infrared reflectivity map of the field of view of the scanning mirror, wherein the infrared reflectivity map identifies the gaze direction.