Camera system for pupil detection and eye tracking
An event camera system for pupil detection may include a camera assembly and a controller, and may also include one or more off-axis light sources. The camera assembly may include one or more infrared (IR) light sources and an event camera. The one or more IR light sources are configured to emit pulses of IR light along an optical path toward an eyebox. The IR light is reflected from an eye in the eyebox, and the reflected light propagates back along the optical path toward the event camera for detection. The controller is configured to determine an orientation of the eye using data output from the event camera.
1. An eye tracking system comprising:
a first light source that emits a first pulse of light over a first time period, the first pulse of light directed along an optical path towards an eye of a user, a surface of the eye reflecting a portion of the first pulse of light back as reflected light along the optical path at a first brightness towards a target area;
a camera located in the target area, the camera and the first light source aligned with the optical path, the camera including a plurality of sensing pixels, each sensing pixel of a subset of the plurality of sensing pixels configured to:
detect an intensity value corresponding to a portion of the reflected light, and
responsive to the detected intensity value above a threshold level, output a data value representing a difference between a previous intensity value detected by each sensing pixel of the subset of sensing pixels and the detected intensity value; and
a controller configured to:
group data values output by the subset of sensing pixels into a two-dimensional matrix,
identify a pupil of the eye based on the data values grouped into the two-dimensional matrix, and
determine a gaze location for the eye based at least in part on the identified pupil.
2. The eye tracking system of claim 1 , further comprising:
a second light source positioned off-axis to the optical path and configured to emit a second pulse of light, and the controller is further configured to:
instruct the second light source to emit the second pulse of light over a second time period that partially overlaps with the first time period.
3. The eye tracking system of claim 2 , wherein a portion of the second pulse of light is reflected from the surface of the eye back along the optical path towards the target area at a second brightness that is less than the first brightness.
4. The eye tracking system of claim 1 , wherein the first light source emits the first pulse of light off-axis to the optical path, and the eye tracking system further comprising:
a beam splitter positioned within the optical path and configured to:
redirect at least a portion of the first pulse of light to propagate substantially along the optical path, and
transmit the portion of the first pulse of light reflected from the surface of the eye back along the optical path towards the camera.
5. The eye tracking system of claim 1 , wherein the threshold level is such that the data value at least partially identifies the pupil.
6. The eye tracking system of claim 1 , wherein the controller is further configured to instruct the first light source to emit the first pulse of light.
7. The eye tracking system of claim 1 , wherein the controller is further configured to instruct the first light source to emit a plurality of pulses of light with a pre-determined pulse rate at a defined duty cycle, the plurality of pulses of light including the first pulse of light.
8. The eye tracking system of claim 1 , wherein the controller is further configured to instruct the first light source to emit a plurality of pulses of light at a plurality of time instances dynamically selected by the controller.
9. The eye tracking system of claim 1 , wherein the controller is further configured to:
instruct the first light source to emit a plurality of pulses of light including the first pulse of light;
generate a respective image frame of a plurality of image frames using data values generated by the camera as a result of emitting each of the plurality of pulses of light and reflection from the surface of the eye; and
track an orientation of the eye based in part on the generated plurality of image frames.
10. The eye tracking system of claim 1 , wherein:
the camera is configured to detect a glint from the first light source reflected off the surface of the eye; and
the controller is further configured to determine the gaze location further based on the detected glint.
11. The eye tracking system of claim 1 , further comprising a second light source positioned off-axis to the optical path and configured to emit a second pulse of light, wherein:
the camera is further configured to detect glints from the first and second light sources reflected off the surface of the eye, and
the controller is further configured to determine the gaze location further based on the detected glints.
12. A method comprising:
emitting, by a first light source, a first pulse of light over a first time period, wherein the first pulse of light is directed along an optical path towards an eye of a user, a surface of the eye reflecting a portion of the first pulse of light back as reflected light along the optical path at a first brightness towards a target area;
detecting, by each sensing pixel of a subset of a plurality of sensing pixels in a camera located in the target area, an intensity value corresponding to a portion of the reflected light, the camera and the first light source aligned with the optical path;
responsive to the detected intensity value above a threshold level, outputting, by each sensing pixel of the subset of sensing pixels, a data value representing a difference between a previous intensity value detected by each sensing pixel of the subset of sensing pixels and the detected intensity value;
grouping data values output by the subset of sensing pixels into a two-dimensional matrix;
identifying a pupil of the eye based on the data values grouped into the two-dimensional matrix; and
determining a gaze location for the eye based at least in part on the identified pupil.
13. The method of claim 12 , further comprising:
instructing a second light source positioned off-axis to the optical path to emit a second pulse of light over a second time period that partially overlaps with the first time period; and
reflecting a portion of the second pulse of light from the surface of the eye back along the optical path towards the target area at a second brightness that is less than the first brightness.
14. The method of claim 12 , further comprising:
emitting, by the first light source, the first pulse of light off-axis to the optical path;
redirecting, by a beam splitter positioned within the optical path, at least a portion of the first pulse of light to propagate substantially along the optical path; and
transmitting, by the beam splitter, the portion of the first pulse of light reflected from the surface of the eye back along the optical path towards the camera.
15. The method of claim 12 , further comprising:
instructing the first light source to emit the first pulse of light having a pulse width of no more than 1 second.
16. The method of claim 12 , further comprising:
instructing the first light source to emit a plurality of pulses of light with a predetermined pulse rate at a defined duty cycle, the plurality of pulses of light including the first pulse of light.
17. The method of claim 12 , further comprising:
instructing the first light source to emit a plurality of pulses of light at a plurality of time instances that are dynamically selected.
18. The method of claim 12 , further comprising:
instructing the first light source to emit a plurality of pulses of light including the first pulse of light;
generating a respective image frame of a plurality of image frames using data values generated by the camera as a result of emitting each of the plurality of pulses of light and reflection from the surface of the eye; and
tracking an orientation of the eye based in part on the generated plurality of image frames.
19. The method of claim 12 , further comprising:
detecting, by the camera, a glint from the first light source reflected off the surface of the eye; and
determining the gaze location further based on the detected glint.
20. The method of claim 12 , further comprising:
instructing a second light source positioned off-axis to the optical path to emit a second pulse of light;
detecting, by the camera, glints from the first and second light sources reflected off the surface of the eye; and
determining the gaze location further based on the detected glints.