Under-display image sensor for eye tracking

1. A method comprising:

receiving a first image captured by a first camera based at least in part on reflections of light emitted by a light source, wherein the reflections are partially occluded by an electronic display disposed in front of the first camera;

filtering the first image using a neural network model trained to recognize occlusions from the electronic display;

detecting a first eye of a user in the filtered first image; and

determining a position or orientation of the first eye relative to the electronic display.

2. The method of claim 1 , further comprising:

emitting infrared (IR) light from the light source, wherein the first camera is configured to detect the IR light.

3. The method of claim 1 , further comprising:

emitting visible light from the light source, wherein the first camera is configured to detect the visible light.

4. The method of claim 1 , further comprising:

determining a gaze direction of the first eye, wherein the gaze direction corresponds to the position or the orientation of the first eye.

5. The method of claim 4 , further comprising:

identifying a display pixel of the electronic display, wherein the display pixel is aligned with the gaze direction or the orientation of the first eye.

6. The method of claim 1 , further comprising:

focusing first light emitted by the electronic display via a first lens; and

focusing second light emitted by the electronic display via a second lens.

7. The method of claim 1 , further comprising:

receiving a second image captured by a second camera proximate to the electronic display.

8. The method of claim 7 , further comprising:

filtering the second image using the neural network model.

9. The method of claim 1 , further comprising:

detecting a second eye of the user in the filtered first image; and

determining a position or orientation of the second eye relative to the electronic display.

10. The method of claim 9 , further comprising:

identifying a first display pixel, of the electronic display, that is aligned with the orientation of the first eye; and

verifying that the first display pixel is aligned with the orientation of the first eye based at least in part on the orientation of the second eye.

11. A device comprising:

an electronic display;

a first camera disposed behind the electronic display, the first camera configured to detect reflections of light emitted by a light source and to capture a first image based at least in part on the reflections, wherein the reflections are partially occluded by the electronic display;

an image filter configured to filter the first image using a neural network model trained to recognize occlusions from the electronic display; and

an eye tracking module configured to detect a first eye of a user in the filtered first image, and to determine a position or orientation of the first eye relative to the electronic display.

12. The device of claim 11 , wherein the light source is configured to emit infrared (IR) light, and the first camera is configured to detect the IR light.

13. The device of claim 11 , wherein the light source is configured to emit visible light, and the first camera is configured to detect the visible light.

14. The device of claim 11 , wherein the eye tracking module is further configured to determine a gaze direction of the first eye, the gaze direction corresponding to the position or the orientation of the first eye.

15. The device of claim 14 , wherein the eye tracking module is further configured to identify a display pixel of the electronic display, the display pixel aligned with the gaze direction or the orientation of the first eye.

16. The device of claim 11 , further comprising:

a first lens configured to focus first light emitted by the electronic display;

a second lens configured to focus second light emitted by the electronic display; and

a housing that couples the first lens to the second lens and the electronic display.

17. The device of claim 11 , further comprising:

a second camera proximate to the electronic display, the second camera configured to capture a second image.

18. The device of claim 17 , wherein the image filter is further configured to filter the second image using the neural network model.

19. The device of claim 11 , wherein the eye tracking module is further configured to:

detect a second eye of the user in the filtered first image; and

determine a position or orientation of the second eye relative to the electronic display.

20. The device of claim 19 , wherein the eye tracking module is further configured to:

identify a first display pixel, of the electronic display, that is aligned with the orientation of the first eye; and

verify that the first display pixel is aligned with the orientation of the first eye based at least in part on the orientation of the second eye.

21. A processing system comprising:

a processor; and

a memory storing instructions that, when executed by the processor, cause the processing system to:

receive an image captured by a camera based at least in part on reflections of light emitted by a light source, wherein the reflections are partially occluded by an electronic display disposed in front of the camera;

filter the image using a neural network model trained to recognize occlusions from the electronic display;

detect an eye of a user in the filtered image; and

determine a position or orientation of the eye relative to the electronic display.

22. The processing system of claim 21 , wherein execution of the instructions causes the processing system to further:

emit infrared (IR) light from the light source, wherein the camera is configured to detect the IR light.