Eye gesture tracking
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for eye gesture recognition. In one aspect, a method includes obtaining an electrical signal that represents a measurement, by a photodetector, of an optical signal reflected from an eye and determining a depth map of the eye based on phase differences between the electrical signal generated by the photodetector and a reference signal. Further, the method includes determining gaze information that represents a gaze of the eye based on the depth map and providing output data representing the gaze information.
1. A system comprising:
a wearable device comprising:
a plurality of tunable optical elements; one or more photodetectors configured to receive light reflected from a user of the system and to generate one or more electrical signals based on the received light; and
one or more processors electrically coupled to the plurality of tunable optical elements and the one or more photodetectors, the one or more processors configured to:
obtain time-of-flight information corresponding to phase differences between a reference signal and the one or more electrical signals generated by the one or more photodetectors;
determine a depth map of the user that comprises a depth map of an eye of the user based on the time-of-flight information;
determine, based on the depth map of the user, action information that corresponds to an action of the user; and provide one or more outputs for adjusting one or more of the plurality of tunable optical elements based on the action information; and
a reference device configured to:
determine a location or a movement of the reference device; and
transmit, from the reference device to the wearable device, reference information representing the location or the movement of the reference device,
wherein the one or more processors of the wearable device are configured to determine the action information based on the reference information received by the wearable device, in addition to the depth map of the user.
2. The system of claim 1 , comprising a display device configured to display an image based on the one or more outputs provided by the one or more processors.
3. The system of claim 2 , wherein the image displayed by the display device is controlled by a subset of the plurality of tunable optical elements.
4. The system of claim 2 , wherein the display device and the one or more photodetectors are packaged as two separate hardware devices.
5. The system of claim 2 , wherein the display device and the one or more photodetectors are packaged as one integrated hardware device.
6. The system of claim 1 , wherein the plurality of tunable optical elements comprise tunable lenses or mirrors.
7. The system of claim 1 , wherein the action of the user comprises an eye gaze of the user.
8. The system of claim 1 , wherein the one or more processors are configured to provide the one or more outputs for adjusting one or more focal lengths of the one or more of the plurality of tunable optical elements.
9. The system of claim 1 , wherein the one or more photodetectors are configured to filter the received light to remove a part of wavelengths of the received light.
10. The system of claim 1 , wherein the received light is a modulation signal with 50% duty cycle.
11. The system of claim 1 , wherein each photodetector of the one or more photodetectors comprises a time-of-flight pixel and two sets of transistors.
12. The system of claim 11 , wherein each set of the two sets of transistors comprises a resistor, a source-follower transistor, and a selection transistor.
13. The system of claim 11 , wherein the time-of-flight pixel is a dual-switching time-of-flight pixel.
14. The system of claim 11 , wherein the time-of-flight pixel is configured to implement a demodulation function by dual-switching transistors.
15. The system of claim 11 , wherein the time-of-flight pixel comprises a light-absorption layer including germanium.
16. A tracking method, comprising:
generating, by one or more photodetectors, one or more electrical signals in response to light reflected from a user and received by the one or more photodetectors;
obtaining, by a computing system having one or more processors electrically coupled to a plurality of tunable optical elements and the one or more photodetectors, time-of-flight information corresponding to phase differences between a reference signal and the one or more electrical signals generated by the one or more photodetectors;
determining, by the computing system, a depth map of the user based on the time-of-flight information, the depth map of the user comprising a depth map of an eye of the user;
obtaining, by the computing system, reference information representing a location or a movement of a reference device in communication with the computing system;
determining, by the computing system and based on the depth map and the reference information, action information that corresponds to an action of the user; and
providing, by the computing system, one or more outputs for adjusting one or more of the plurality of tunable optical elements based on the action information.
17. The method of claim 16 , further comprising:
displaying an image using a display device based on the one or more outputs provided by the one or more processors.
18. The method of claim 17 , wherein the image displayed by the display device is controlled by a subset of the plurality of tunable optical elements.
19. The method of claim 17 , wherein the display device and the one or more photodetectors are integrated in a hardware package.
20. A system comprising:
a wearable device comprising:
a plurality of tunable optical elements; one or more photodetectors configured to receive light reflected from a user of the system and to generate one or more electrical signals based on the received light; and
one or more processors electrically coupled to the plurality of tunable optical elements and the one or more photodetectors, the one or more processors configured to:
obtain time-of-flight information corresponding to phase differences between a reference signal and the one or more electrical signals generated by the one or more photodetectors;
determine a depth map of the user that comprises a depth map of an eye of the user based on the time-of-flight information;
determine, based on the depth map of the user, action information that corresponds to an action of the user; and
provide one or more outputs for adjusting one or more of the plurality of tunable optical elements based on the action information; and
a reference device configured to:
determine a location or a movement of the reference device; and
transmit, from the reference device to the wearable device, reference information representing the location or the movement of the reference device,
wherein the one or more processors of the wearable device are configured to determine the action information based on the reference information received by the wearable device, in addition to the depth map of the user.