Eye tracking using alternate sampling
An eye tracking system can include a first camera configured to capture a first plurality of visual data of a right eye at a first sampling rate. The system can include a second camera configured to capture a second plurality of visual data of a left eye at a second sampling rate. The second plurality of visual data can be captured during different sampling times than the first plurality of visual data. The system can estimate, based on at least some visual data of the first and second plurality of visual data, visual data of at least one of the right or left eye at a sampling time during which visual data of an eye for which the visual data is being estimated are not being captured. Eye movements of the eye based on at least some of the estimated visual data and at least some visual data of the first or second plurality of visual data can be determined.
1. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform one or more operations for eye tracking, comprising:
capturing with a first camera a first plurality of visual data of a right eye at a first sampling rate;
capturing with a second camera a second plurality of visual data of a left eye at a second sampling rate, the second plurality of visual data captured during different sampling times than the first plurality of visual data; and
by processing electronics:
based on at least some visual data of the first plurality of visual data and the second plurality of visual data, estimating, as estimated visual data, visual data of at least one of the right eye or the left eye at a sampling time during which visual data of the at least one of the right eye or the left eye for which the visual data is being estimated is not being captured; and
determining eye movement of the at least one of the right eye or the left eye based on at least some of the estimated visual data and at least some visual data of the first plurality of visual data or the second plurality of visual data.
2. The non-transitory, computer-readable medium of claim 1 , further comprising, by the processing electronics, causing a display to render virtual content based at least partly on the eye movement.
3. The non-transitory, computer-readable medium of claim 1 , wherein determining the eye movement comprises:
determining eye movement of the right eye based on at least some of the estimated visual data of the right eye and at least some visual data of the first plurality of visual data of the right eye; and
determining eye movement of the left eye based on at least some of the estimated visual data of the left eye and at least some visual data of the second plurality of visual data of the left eye.
4. The non-transitory, computer-readable medium of claim 1 , wherein a combined sampling rate of the first camera and the second camera comprises an aggregate of the first sampling rate and the second sampling rate.
5. The non-transitory, computer-readable medium of claim 1 , wherein the first sampling rate is equal to the second sampling rate.
6. The non-transitory, computer-readable medium of claim 1 , wherein the first camera and the second camera alternately capture the first plurality of visual data and the second plurality of visual data.
7. The non-transitory, computer-readable medium of claim 6 , wherein a combined sampling rate of the first camera and the second camera comprises 60 Hz.
8. The non-transitory, computer-readable medium of claim 1 , wherein the first sampling rate and second the sampling rate comprise 30 Hz.
9. The non-transitory, computer-readable medium of claim 1 , wherein estimating the visual data of comprises determining a difference between visual data of the first plurality of visual data of the right eye and visual data of the second plurality of visual data of the left eye.
10. The non-transitory, computer-readable medium of claim 9 , further comprising filtering the difference by a low pass filter.
11. The non-transitory, computer-readable medium of claim 9 , wherein:
estimating the visual data of the right eye comprises determining a first difference between visual data of the second plurality of visual data of the left eye captured during the sampling time of the estimated visual data and visual data of the first plurality of visual data of the right eye captured during a sampling time prior to the sampling time of the estimated visual data; and
estimating the visual data of the left eye comprises determining a second difference between visual data of the first plurality of visual data of the right eye captured during the sampling time of the estimated visual data and visual data of the second plurality of visual data of the left eye captured during a sampling time prior to the sampling time of the estimated visual data.
12. The non-transitory, computer-readable medium of claim 11 , wherein:
the sampling time of the visual data of the first plurality of visual data of the right eye is immediately prior to the sampling time of the estimated visual data; and
the sampling time of the visual data of the second plurality of visual data of the left eye is immediately prior to the sampling time of the estimated visual data.
13. The non-transitory, computer-readable medium of claim 9 , further comprising, by the processing electronics:
estimating the visual data of the right eye based on the difference and visual data of the first plurality of visual data of the right eye captured during a sampling time prior to the sampling time of the estimated visual data; and
estimating the visual data of the left eye based on the difference and visual data of the second plurality of visual data of the left eye captured during a sampling time prior to the sampling time of the estimated visual data.
14. The non-transitory, computer-readable medium of claim 13 , wherein:
the sampling time of the visual data of the first plurality of visual data of the right eye is immediately prior to the sampling time of the estimated visual data; and
the sampling time of the visual data of the second plurality of visual data of the left eye is immediately prior to the sampling time of the estimated visual data.
15. A computer-implemented system, comprising:
one or more computers; and
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations for eye tracking, comprising:
capturing with a first camera a first plurality of visual data of a right eye at a first sampling rate;
capturing with a second camera a second plurality of visual data of a left eye at a second sampling rate, the second plurality of visual data captured during different sampling times than the first plurality of visual data; and
by processing electronics of the one or more computers:
based on at least some visual data of the first plurality of visual data and the second plurality of visual data, estimating, as estimated visual data, visual data of at least one of the right eye or the left eye at a sampling time during which visual data of the at least one of the right eye or the left eye for which the visual data is being estimated is not being captured; and
determining eye movement of the at least one of the right eye or the left eye based on at least some of the estimated visual data and at least some visual data of the first plurality of visual data or the second plurality of visual data.
16. The computer-implemented system of claim 15 , further comprising, by the processing electronics, causing a display to render virtual content based at least partly on the eye movement.
17. The computer-implemented system of claim 15 , wherein determining the eye movement comprises:
determining eye movement of the right eye based on at least some of the estimated visual data of the right eye and at least some visual data of the first plurality of visual data of the right eye; and
determining eye movement of the left eye based on at least some of the estimated visual data of the left eye and at least some visual data of the second plurality of visual data of the left eye.
18. The computer-implemented system of claim 15 , wherein a combined sampling rate of the first camera and the second camera comprises an aggregate of the first sampling rate and the second sampling rate.
19. The computer-implemented system of claim 15 , wherein the first sampling rate is equal to the second sampling rate.
20. The computer-implemented system of claim 15 , wherein the first camera and the second camera alternately capture the first plurality of visual data and the second plurality of visual data.
21. The computer-implemented system of claim 20 , wherein a combined sampling rate of the first camera and the second camera comprises 60 Hz.
22. The computer-implemented system of claim 15 , wherein the first sampling rate and second the sampling rate comprise 30 Hz.
23. The computer-implemented system of claim 15 , wherein estimating the visual data of comprises determining a difference between visual data of the first plurality of visual data of the right eye and visual data of the second plurality of visual data of the left eye.
24. The computer-implemented system of claim 23 , further comprising filtering the difference by a low pass filter.
25. The computer-implemented system of claim 23 , wherein:
estimating the visual data of the right eye comprises determining a first difference between visual data of the second plurality of visual data of the left eye captured during the sampling time of the estimated visual data and visual data of the first plurality of visual data of the right eye captured during a sampling time prior to the sampling time of the estimated visual data; and
estimating the visual data of the left eye comprises determining a second difference between visual data of the first plurality of visual data of the right eye captured during the sampling time of the estimated visual data and visual data of the second plurality of visual data of the left eye captured during a sampling time prior to the sampling time of the estimated visual data.
26. The computer-implemented system of claim 25 , wherein:
the sampling time of the visual data of the first plurality of visual data of the right eye is immediately prior to the sampling time of the estimated visual data; and
the sampling time of the visual data of the second plurality of visual data of the left eye is immediately prior to the sampling time of the estimated visual data.
27. The computer-implemented system of claim 23 , further comprising, by the processing electronics:
estimating the visual data of the right eye based on the difference and visual data of the first plurality of visual data of the right eye captured during a sampling time prior to the sampling time of the estimated visual data; and
estimating the visual data of the left eye based on the difference and visual data of the second plurality of visual data of the left eye captured during a sampling time prior to the sampling time of the estimated visual data.
28. The computer-implemented system of claim 27 , wherein:
the sampling time of the visual data of the first plurality of visual data of the right eye is immediately prior to the sampling time of the estimated visual data; and
the sampling time of the visual data of the second plurality of visual data of the left eye is immediately prior to the sampling time of the estimated visual data.