Head mounted eye tracking device and method for providing drift free eye tracking through a lens system
The invention relates to a method and a head mounted eye tracking device ( 10 a, 10 b ) for determining at least a feature of an eye ( 12 ), wherein the eye tracking device ( 10 a, 10 b ) comprises a capturing device (C) for capturing light reflected by the eye ( 12 ) and an optical component ( 14 ; E, E 1 , En, E 2 ) capable of altering a propagation direction of the light passing through the optical component ( 14 ; E, E 1 , En, E 2 ). The eye tracking device ( 10 a, 10 b ) is configured such that when fixed to the head of the user the light captured by the capturing device (C) has passed through the optical component ( 14 ; E, E 1 , En, E 2 ) and constitutes at least part of an image and the feature is determined on the basis of the image in dependency of an information about a relative position between the eye ( 12 ) and the head mounted eye tracking device ( 10 a, 10 b ).
1. A method comprising:
at an electronic device including a processor, non-transitory memory, and a camera:
capturing, with the camera, a first image of an eye of a user at a first time;
determining, using the processor based on the first image, a position of a gaze-variant feature of the eye of the user at the first time and a position of a gaze-invariant feature of the eye of the user at the first time, wherein the position of the gaze-variant feature changes if the user changes a gaze direction of the eye and the position of the gaze-invariant feature does not change if the user changes the gaze direction of the eye;
determining, using the processor, a first gaze direction of the eye based on the position of the gaze-variant feature at the first time;
capturing, with the camera, a second image of the eye of the user at a second time;
determining, using the processor based on the second image, an initial change in the position of the gaze-variant feature from the first time to the second time;
determining, using the processor, a change in the position of the gaze-invariant feature from the first time to the second time;
determining, using the processor, a corrected change in the position of the gaze-variant feature based on the initial change in the position of the gaze-variant feature and the change in the position of the gaze-invariant feature; and
determining, using the processor, a second gaze direction of the eye based on the corrected change in the position of the gaze-variant feature.
2. The method of claim 1 , wherein determining the change in the position of the gaze-invariant feature includes determining a change in the position of the gaze-invariant feature in the second image as compared to the first image.
3. The method of claim 1 , wherein determining the change in the position of the gaze-invariant feature includes receiving data from a motion sensor.
4. The method of claim 1 , wherein determining the second gaze direction based on the corrected change in the position of the gaze-variant feature includes modeling the refractive properties of at least one refractive optical element.
5. The method of claim 1 , wherein the gaze-variant feature includes at least one of a pupil center or an iris center.
6. The method of claim 1 , wherein the gaze-invariant feature includes at least one of an eye corner, and eyebrow, or a nose.
7. The method of claim 1 , further comprising:
capturing, with the camera, a third image of a second eye of the user at the first time;
determining, using the processor, a third gaze direction of the second eye based on the third image;
capturing, with the camera, a fourth image of a second eye of the user at the second time; and
determining, using the processor, a fourth gaze direction of the second eye based on the fourth image and the change in the position of the gaze-invariant feature.
8. The method of claim 7 , further comprising determining a point of gaze based on the second gaze direction of the eye and the fourth gaze direction of the second eye.
9. An apparatus comprising:
a camera to capture a first image of an eye of a user at a first time and capture a second image of the eye of the user at a second time; and
a processor to:
determine, based on the first image, a position of a gaze-variant feature of the eye of the user at the first time and a position of a gaze-invariant feature of the eye of the user at the first time, wherein the position of the gaze-variant feature changes if the user changes a gaze direction of the eye and the position of the gaze-invariant feature does not change if the user changes the gaze direction of the eye;
determine a first gaze direction of the eye based on the position of the gaze-variant feature at the first time;
determine, based on the second image, an initial change in the position of the gaze-variant feature from the first time to the second time;
determine a change in the position of the gaze-invariant feature from the first time to the second time;
determine, a corrected change in the position of the gaze-variant feature based on the initial change in the position of the gaze-variant feature and the change in the position of the gaze-invariant feature; and
determine a second gaze direction of the eye based on the corrected change in the position of the gaze-variant feature.
10. The apparatus of claim 9 , wherein the processor is to determine the second gaze direction based on the corrected position by modeling the refractive properties of at least one refractive optical element.
11. The apparatus of claim 9 , wherein the gaze-variant feature includes at least one of a pupil center or an iris center.
12. The apparatus of claim 9 , wherein the gaze-invariant feature includes at least one of an eye corner, and eyebrow, or a nose.
13. The apparatus of claim 9 , wherein the camera is further to capture a third image of a second eye of the user at the first time and capture a fourth image of a second eye of the user at the second time, wherein the processor is to determine a third gaze direction of the second eye based on the third image and determine a fourth gaze direction of the second eye based on the fourth image and the change in the position of the gaze-invariant feature.
14. A non-transitory computer-readable medium encoding instructions which, when executed by a processor, cause a processor to perform operations comprising:
determining, based on a first image of an eye of a user taken at a first time, a position of a gaze-variant feature of the eye of the user at the first time and a position of a gaze-invariant feature of the eye of the user at the first time, wherein the position of the gaze-variant feature changes if the user changes a gaze direction of the eye and the position of the gaze-invariant feature does not change if the user changes the gaze direction of the eye;
determining a first gaze direction of the eye based on the position of gaze-variant feature at the first time;
determining, based on a second image of the eye of the user taken at a second time, an initial change in the position of the gaze-variant feature from the first time to the second time;
determining a change in the position of the gaze-invariant feature from the first time to the second time;
determining a corrected change in the position of the gaze-variant feature based on the initial change in the position of the gaze-variant feature and the change in the position of the gaze-invariant feature; and
determining a second gaze direction of the eye based on the corrected change in the position of the gaze-variant feature.
15. The non-transitory computer-readable medium of claim 14 , wherein determining the change in the position of the gaze-invariant feature includes determining a change in position of the gaze-invariant feature in the second image as compared to the first image.
16. The non-transitory computer-readable medium of claim 14 , wherein determining the second gaze direction based on the corrected change in the position of the gaze-variant feature includes modeling the refractive properties of at least one refractive optical element.