Eye center of rotation determination with one or more eye tracking cameras
A display system can include a head-mounted display configured to project light to an eye of a user to display virtual image content at different amounts of divergence and collimation. The display system can include an inward-facing imaging system possibly comprising a plurality of cameras that image the user's eye and glints for thereon and processing electronics that are in communication with the inward-facing imaging system and that are configured to obtain an estimate of a center of rotation of the user's eye using cornea data derived from the glint images. The display system may render virtual image content with a render camera positioned at the determined position of the center of rotation of said eye.
1. A display system configured to project light to an eye of a user to display virtual image content in a vision field of said user, said display system comprising:
a frame configured to be supported on a head of the user;
a head-mounted display disposed on the frame, said display configured to project light into said user's eye to display virtual image content to the user's vision field;
first and second eye tracking cameras configured to image the user's eye;
a plurality of light emitters; and
processing electronics in communication with the display and the first and second eye tracking cameras, the processing electronics configured to:
receive images of the user's eye captured by the first and second eye tracking cameras, glint reflections of the different light emitters observable in said images of the eye captured by the first and second eye tracking cameras; and
estimate a location of a center of corneal curvature of the user's eye based on the location of the glint reflections in said images produced by both said first and second eye tracking camera and based on the location of both the first and second eye tracking cameras and the locations of the emitters that produced said respective glint reflections, and
wherein said processing electronics is configured to obtain an estimate of a center of rotation of the user's eye based on multiple determinations of the center of corneal curvature of the user's eye for different eye poses.
2. The display system of claim 1 , wherein said processing electronics is configured to:
based on the location of the glint reflections in one or more images produced by said first eye tracking camera and based on the location of the first eye tracking camera and the location of the emitters that produced said glint reflections, determine a first direction toward the center of corneal curvature of the user's eye; and
based on the location of the glint reflections in one or more images produced by said second eye tracking camera and based on the location of the second eye tracking camera and the location of the emitters that produced said glint reflections, determine a second direction toward the center of corneal curvature of the user's eye.
3. The display system of claim 2 , wherein said processing electronics is configured to determine the first direction by:
defining a first plane that includes the first eye tracking camera, a location of a first glint reflection and a location of the light emitter corresponding to said first glint reflection;
defining a second plane that includes the first eye tracking camera, a location of a second glint reflection and a location of the light emitter corresponding to said second glint reflection; and
determining a region of convergence of the first plane and the second plane, the region of convergence extending along the first direction.
4. The display system of claim 3 , said processing electronics are configured to determine the second direction by:
defining a third plane that includes the second eye tracking camera, the location of a third glint reflection, and a location of the light emitter corresponding to said third glint reflection;
defining a fourth plane that includes the second eye tracking camera, the location of a fourth glint reflection, and a location of the light emitter corresponding to said fourth glint reflection; and
determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction.
5. The display system of claim 2 , wherein said processing electronics is configured to estimate a location of said center of corneal curvature of the user's eye based on said first and second directions toward the center of the corneal curvature of the user's eye.
6. The display system of claim 2 , wherein said processing electronics is configured to:
determine said first direction along which the center of corneal curvature of the user's eye is estimated to be located based on at least one first image received from the first eye tracking camera; and
determine said second direction along which the center of corneal curvature of the user's eye is estimated to be located based on at least one second image received from the second eye tracking camera, said first and second directions converging toward a region.
7. The display system of claim 2 , wherein said processing electronics is configured to:
obtain an estimate of a center of corneal curvature of the user's eye based on the convergence of the first and second directions.
8. The display system of claim 2 , wherein said processing electronics is configured to estimate a location of said center of corneal curvature of the user's eye by identifying a region of convergence of said first and second directions toward the center of the corneal curvature of the user's eye.
9. A display system configured to project light to an eye of a user to display virtual image content in a vision field of said user, said display system comprising:
a frame configured to be supported on a head of the user;
a head-mounted display disposed on the frame, said display configured to project light into said user's eye to display virtual image content to the user's vision field;
first and second eye tracking cameras configured to image the user's eye;
a plurality of light emitters; and
processing electronics in communication with the display and the first and second eye tracking cameras, the processing electronics configured to:
receive images of the user's eye captured by the first and second eye tracking cameras, glint reflections of the different light emitters observable in said images of the eye captured by the first and second eye tracking cameras; and
estimate a location of a center of corneal curvature of the user's eye based on the location of the glint reflections in said images produced by both said first and second eye tracking camera and based on the location of both the first and second eye tracking cameras and the locations of the emitters that produced said respective glint reflections,
wherein said processing electronics is configured to determine a locus of points corresponding to estimates of the center of corneal curvature of the user's eye for different eye poses, and
wherein said processing electronics is configured to obtain an estimate of a center of rotation of the user's eye based on said locus of points corresponding to estimates of the center of corneal curvature of the user's eye for different eye poses.
10. The display system of claim 9 , wherein said processing electronics is configured to determine a surface based on said locus of points and to obtain the estimate of the center of rotation of the user's eye by estimating a center of curvature of said surface.
11. The display system of claim 10 , wherein said processing electronics is configured to fit said surface to said locus of points to obtain said surface.
12. The display system of claim 9 , wherein said processing electronics is configured to:
based on the location of the glint reflections in one or more images produced by said first eye tracking camera and based on the location of the first eye tracking camera and the location of the emitters that produced said glint reflections, determine a first direction toward the center of corneal curvature of the user's eye; and
based on the location of the glint reflections in one or more images produced by said second eye tracking camera and based on the location of the second eye tracking camera and the location of the emitters that produced said glint reflections, determine a second direction toward the center of corneal curvature of the user's eye.
13. The display system of claim 12 , wherein said processing electronics is configured to estimate a location of said center of corneal curvature of the user's eye based on said first and second directions toward the center of the corneal curvature of the user's eye.
14. The display system of claim 12 , wherein said processing electronics is configured to:
determine said first direction along which the center of corneal curvature of the user's eye is estimated to be located based on at least one first image received from the first eye tracking camera; and
determine said second direction along which the center of corneal curvature of the user's eye is estimated to be located based on at least one second image received from the second eye tracking camera, said first and second directions converging toward a region.
15. The display system of claim 12 , wherein said processing electronics is configured to:
obtain an estimate of a center of corneal curvature of the user's eye based on the convergence of the first and second directions.
16. A display system configured to project light to an eye of a user to display virtual image content in a vision field of said user, said display system comprising:
a frame configured to be supported on a head of the user;
a head-mounted display disposed on the frame, said display configured to project light into said user's eye to display virtual image content to the user's vision field;
first and second eye tracking cameras configured to image the user's eye;
a plurality of light emitters; and
processing electronics in communication with the display and the first and second eye tracking cameras, the processing electronics configured to:
receive images of the user's eye captured by the first and second eye tracking cameras, glint reflections of the different light emitters observable in said images of the eye captured by the first and second eye tracking cameras; and
estimate a location of a center of corneal curvature of the user's eye based on the location of the glint reflections in said images produced by both said first and second eye tracking camera and based on the location of both the first and second eye tracking cameras and the locations of the emitters that produced said respective glint reflections,
wherein said processing electronics is configured to determine a locus of points corresponding to estimates of the center of corneal curvature of the user's eye for different eye poses, and
wherein said processing electronics is configured to determine a surface based on said locus of points and to obtain an estimate of a center of rotation of the user's eye by estimating a center of curvature of said surface.
17. The display system of claim 16 , wherein said processing electronics is configured to:
based on the location of the glint reflections in one or more images produced by said first eye tracking camera and based on the location of the first eye tracking camera and the location of the emitters that produced said glint reflections, determine a first direction toward the center of corneal curvature of the user's eye; and
based on the location of the glint reflections in one or more images produced by said second eye tracking camera and based on the location of the second eye tracking camera and the location of the emitters that produced said glint reflections, determine a second direction toward the center of corneal curvature of the user's eye.
18. The display system of claim 17 , wherein said processing electronics is configured to estimate a location of said center of corneal curvature of the user's eye based on said first and second directions toward the center of the corneal curvature of the user's eye.
19. The display system of claim 17 , wherein said processing electronics is configured to:
determine said first direction along which the center of corneal curvature of the user's eye is estimated to be located based on at least one first image received from the first eye tracking camera; and
determine said second direction along which the center of corneal curvature of the user's eye is estimated to be located based on at least one second image received from the second eye tracking camera, said first and second directions converging toward a region.
20. The display system of claim 17 , wherein said processing electronics is configured to:
obtain an estimate of a center of corneal curvature of the user's eye based on the convergence of the first and second directions.
21. A display system configured to project light to an eye of a user to display virtual image content in a vision field of said user, said display system comprising:
a frame configured to be supported on a head of the user;
a head-mounted display disposed on the frame, said display configured to project light into said user's eye to display virtual image content to the user's vision field;
first and second eye tracking cameras configured to image the user's eye;
a plurality of light emitters; and
processing electronics in communication with the display and the first and second eye tracking cameras, the processing electronics configured to:
receive images of the user's eye captured by the first and second eye tracking cameras, glint reflections of the different light emitters observable in said images of the eye captured by the first and second eye tracking cameras; and
estimate a location of a center of corneal curvature of the user's eye based on the location of the glint reflections in said images produced by both said first and second eye tracking camera and based on the location of both the first and second eye tracking cameras and the locations of the emitters that produced said respective glint reflections,
wherein said processing electronics is configured to determine a locus of points corresponding to estimates of the center of corneal curvature of the user's eye for different eye poses, and
wherein said processing electronics is configured to determine a surface based on said locus of points and to obtain an estimate of a center of rotation of the user's eye by determining a region where a plurality of normals to said surface converge.
22. The display system of claim 21 , wherein said processing electronics is configured to:
based on the location of the glint reflections in one or more images produced by said first eye tracking camera and based on the location of the first eye tracking camera and the location of the emitters that produced said glint reflections, determine a first direction toward the center of corneal curvature of the user's eye; and
based on the location of the glint reflections in one or more images produced by said second eye tracking camera and based on the location of the second eye tracking camera and the location of the emitters that produced said glint reflections, determine a second direction toward the center of corneal curvature of the user's eye.
23. The display system of claim 22 , wherein said processing electronics is configured to estimate a location of said center of corneal curvature of the user's eye based on said first and second directions toward the center of the corneal curvature of the user's eye.
24. The display system of claim 22 , wherein said processing electronics is configured to:
determine said first direction along which the center of corneal curvature of the user's eye is estimated to be located based on at least one first image received from the first eye tracking camera; and
determine said second direction along which the center of corneal curvature of the user's eye is estimated to be located based on at least one second image received from the second eye tracking camera, said first and second directions converging toward a region.
25. The display system of claim 22 , wherein said processing electronics is configured to:
obtain an estimate of a center of corneal curvature of the user's eye based on the convergence of the first and second directions.