Eye gesture tracking
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for eye gesture recognition. In one aspect, a method includes obtaining an electrical signal that represents a measurement, by a photodetector, of an optical signal reflected from an eye and determining a depth map of the eye based on phase differences between the electrical signal generated by the photodetector and a reference signal. Further, the method includes determining gaze information that represents a gaze of the eye based on the depth map and providing output data representing the gaze information.
1. A system, comprising:
a machine including a display, the display including a plurality of tunable optical elements;
a device including circuitry configured to:
obtain an electrical signal that represents a measurement, by a photodetector, of an optical signal reflected from an eye,
determine a depth map of the eye based on phase differences between a reference signal and the electrical signal generated by the photodetector, and
determine a gaze information that represents a gaze of the eye based on the depth map; and
one or more processors in communication with the machine and the device, the one or more processors including one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform the operations including:
receiving, from the device, output data representing the gaze information; and
determining the gaze information representing the gaze of the eye in relation to the display of the machine.
2. The system of claim 1 , wherein the operations further comprise:
determining a particular position on the display that the eye is focused on, the particular position being based on the gaze information representing the gaze of the eye in relation to the display; and
providing an indication at the particular position on the display.
3. The system of claim 1 , wherein the operations further comprise:
determining a particular position on the display that the eye is focused on, the particular position being based on the gaze information representing the gaze of the eye in relation to the display; and
providing a foveated image at the particular area on the display.
4. The system of claim 1 , wherein the plurality of tunable optical elements include tunable lenses or tunable mirrors.
5. The system of claim 4 , wherein a tuning of a subset of the plurality of tunable optical elements is activated based on the gaze information representing the gaze of the eye in relation to the display.
6. The system of claim 5 , wherein the tuning of the subset of the plurality of tunable optical elements comprises dynamically refocusing light incident on the subset of the plurality of tunable optical elements.
7. The system of claim 1 , further comprising:
a wearable coupled to the machine, the device, and the one or more processors to form an integrated hardware package, the display of the machine being opaque in which visual images are shown on the display by one or more of an array of light sources.
8. The system of claim 1 , further comprising:
a wearable coupled to the machine and the device to form an integrated hardware package, the display of the machine being opaque in which visual images are shown on the display by one or more of an array of light sources; and
the one or more processors located at a remote location and in communication with the integrated hardware package via a wireless or wired connection.
9. The system of claim 1 , further comprising:
a wearable coupled to the machine, the device, and the one or more processors to form an integrated hardware package, the display of the machine being at least partly transparent to images projected towards the display, whereby a property of the images projected towards the display is modified by one or more of the plurality of tunable optical elements of the display.
10. The system of claim 1 , further comprising:
a wearable coupled to the machine and the device to form an integrated hardware package, the display of the machine being at least partly transparent to images projected towards the display, whereby a property of the images projected towards the display is modified by one or more of the plurality of tunable optical elements of the display; and
the one or more processors located at a remote location and in communication with the integrated hardware package via a wireless or wired connection.
11. The system of claim 1 , further comprising:
a pluggable coupled to the device and the one or more processors to form an integrated hardware package; and
the machine located at a remote location and in communication with the integrated hardware package via a wireless or wired connection, the display of the machine being opaque in which visual images are shown on the display by one or more of an array of light sources.
12. The system of claim 1 , further comprising:
a wearable couple to the device and the one or more processors to form an integrated hardware package; and
the machine located at a remote location and in communication with the integrated hardware package via a wireless or wired connection, the display of the machine being opaque in which visual image are shown on the display by one or more of an array of light sources.
13. The system of claim 12 , wherein the operations further comprise:
determining a particular position on the display that the eye is focused on, the particular position being based on the gaze information representing the gaze of the eye in relation to the display; and
providing an indication at the particular position on the display.
14. The system of claim 1 , wherein the optical signal reflected from the eye is generated by an optical source that is biased by a modulated signal, the modulation signal being in sync with the reference signal.
15. A device, comprising:
a plurality of tunable optical elements for adjusting focal lengths;
one or more processors in communication with the plurality of tunable optical elements, the one or more processors including one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising:
obtaining an electrical signal that represents a measurement, by a photodetector, of an optical signal reflected from an eye,
determining a depth map of the eye based on phase differences between a reference signal and the electrical signal generated by the photodetector,
determining gaze information that represents a gaze of the eye based on the depth map, the gaze information representing the gaze of the eye in relation to a display of a remote device, and
activating a tuning of a subset of the plurality of tunable optical elements based on the gaze information.