IP Library › Granted Patent US 12672775
Granted Patent B2
US 12672775 · App. 18/118,936 · Granted Jul 7, 2026

Accommodation tracking based on retinal-imaging

Inventors: Itai Afek (Kfar Sava, IL); Ariel Lipson (Tel Aviv, IL); Roei Remez (Tel Aviv, IL)
Assignee: Apple Inc.
A61B3/113G06T7/73A61B3/0025
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12672775
App. No.
18/118,936
Granted
Jul 7, 2026
Kind
B2
Abstract

Various implementations disclosed herein include devices, systems, and methods that track an eye characteristic (e.g., gaze direction, eye orientation, etc.). For example, an example process may include producing light that reflects off a retina of an eye, receiving an image of a portion of the retina from an image sensor, the image corresponding to a plurality of reflections of the light scattered from the retina of the eye, obtaining a representation of the eye corresponding to a first accommodative state, the representation representing at least some of the portion of the retina, and tracking an eye characteristic based on a comparison of the image of the portion of the retina with the representation of the eye.

Claims (56)

1 . A method comprising:

at an electronic device having a processor:

producing light that reflects off a retina of an eye;

receiving an image of a portion of the retina from an image sensor, the image corresponding to a plurality of reflections of the light scattered from the retina of the eye;

obtaining a representation of the eye corresponding to a first accommodative state, the representation representing at least some of the portion of the retina;

tracking an eye characteristic based on a comparison of the image of the portion of the retina with the representation of the eye; and

determining an accommodation of the eye based on estimating a blurring measure and estimating a geometric scaling factor between the image of the portion of the retina and the representation of the eye.

2 . The method of claim 1 , wherein the representation of the eye comprises a map of the at least some of the portion of the retina.

3 . The method of claim 2 , wherein generating the map of the at least some of the portion of the retina comprises:

obtaining enrollment images of the eye of a user while the user (i) accommodates the eye to an enrollment depth, and (ii) scans through a gaze angle space representative of a defined field of view; and

generating the map of the at least some of the portion of the retina based on combining at least a portion of two or more of the enrollment images of the eye.

4 . The method of claim 1 , wherein tracking the eye characteristic based on the comparison of the image of the portion of the retina with the representation of the eye comprises estimating a degree of defocus of a feature.

5 . The method of claim 4 , wherein estimating the degree of defocus of the feature is based on using focus pixels configured to measure focus for defocus estimation.

6 . The method of claim 1 , wherein tracking the eye characteristic based on the comparison of the image of the portion of the retina with the representation of the eye comprises:

adjusting a focus position of a lens to increase an image sharpness metric; and

determining an amount of defocus based on an amount of lens movement required to increase the image sharpness metric.

7 . The method of claim 1 , wherein tracking the eye characteristic based on the comparison of the image of the portion of the retina with the representation of the eye comprises:

comparing a difference of a first feature width of the image of the portion of the retina with a second feature width of the representation of the eye to a threshold.

8 . The method of claim 1 , wherein tracking the eye characteristic based on the comparison of the image of the portion of the retina with the representation of the eye is tracked at a same time the image of the portion of the retina is received.

9 . The method of claim 1 , wherein the light is infrared (IR) light.

10 . The method of claim 1 , wherein the electronic device is a head-mounted device (HMD).

11 . The method of claim 10 , wherein the HMD comprises a waveguide.

12 . A device comprising:

a non-transitory computer-readable storage medium; and

one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising:

producing light that reflects off a retina of an eye;

receiving an image of a portion of the retina from an image sensor, the image corresponding to a plurality of reflections of the light scattered from the retina of the eye;

obtaining a representation of the eye corresponding to a first accommodative state, the representation representing at least some of the portion of the retina;

tracking an eye characteristic based on a comparison of the image of the portion of the retina with the representation of the eye; and

determining an accommodation of the eye based on estimating a blurring measure and estimating a geometric scaling factor between the image of the portion of the retina and the representation of the eye.

13 . The device of claim 12 , wherein the representation of the eye comprises a map of the at least some of the portion of the retina.

14 . The device of claim 13 , wherein generating the map of the at least some of the portion of the retina comprises:

obtaining enrollment images of the eye of a user while the user (i) accommodates the eye to an enrollment depth, and (ii) scans through a gaze angle space representative of a defined field of view; and

generating the map of the at least some of the portion of the retina based on combining at least a portion of two or more of the enrollment images of the eye.

15 . The device of claim 12 , wherein tracking the eye characteristic based on the comparison of the image of the portion of the retina with the representation of the eye comprises estimating a degree of defocus of a feature.

16 . The device of claim 15 , wherein estimating the degree of defocus of the feature is based on using focus pixels configured to measure focus for defocus estimation.

17 . The device of claim 12 , wherein tracking the eye characteristic based on the comparison of the image of the portion of the retina with the representation of the eye comprises:

adjusting a focus position of a lens to increase an image sharpness metric; and

determining an amount of defocus based on an amount of lens movement required to increase the image sharpness metric.

18 . The device of claim 12 , wherein tracking the eye characteristic based on the comparison of the image of the portion of the retina with the representation of the eye comprises:

comparing a difference of a first feature width of the image of the portion of the retina with a second feature width of the representation of the eye to a threshold.

19 . The device of claim 12 , wherein tracking the eye characteristic based on the comparison of the image of the portion of the retina with the representation of the eye is tracked at a same time the image of the portion of the retina is received.

20 . The device of claim 12 , wherein the light is infrared (IR) light.

21 . The device of claim 12 , wherein the device is a head-mounted device (HMD).

22 . The device of claim 21 , wherein the HMD comprises a waveguide.

23 . A non-transitory computer-readable storage medium, storing program instructions executable on a device to perform operations comprising:

producing light that reflects off a retina of an eye;

receiving an image of a portion of the retina from an image sensor, the image corresponding to a plurality of reflections of the light scattered from the retina of the eye;

obtaining a representation of the eye corresponding to a first accommodative state, the representation representing at least some of the portion of the retina;

tracking an eye characteristic based on a comparison of the image of the portion of the retina with the representation of the eye; and

determining an accommodation of the eye based on estimating a blurring measure and estimating a geometric scaling factor between the image of the portion of the retina and the representation of the eye.

24 . The non-transitory computer-readable storage medium of claim 23 , wherein the representation of the eye comprises a map of the at least some of the portion of the retina.

25 . The non-transitory computer-readable storage medium of claim 24 , wherein generating the map of the at least some of the portion of the retina comprises:

obtaining enrollment images of the eye of a user while the user (i) accommodates the eye to an enrollment depth, and (ii) scans through a gaze angle space representative of a defined field of view; and

generating the map of the at least some of the portion of the retina based on combining at least a portion of two or more of the enrollment images of the eye.

26 . The method of claim 1 , wherein determining the accommodation of the eye is further based on mapping at least the estimated blurring measure and the estimated geometric scaling factor to an accommodation distance.