Display systems and methods for determining registration between display and eyes of user
A wearable device may include a head-mounted display (HMD) for rendering a three-dimensional (3D) virtual object which appears to be located in an ambient environment of a user of the display. The relative positions of the HMD and one or more eyes of the user may not be in desired positions to receive, or register, image information outputted by the HMD. For example, the HMD-to-eye alignment vary for different users and may change over time (e.g., as a given user moves around or as the HMD slips or otherwise becomes displaced). The wearable device may determine a relative position or alignment between the HMD and the user's eyes by determining whether features of the eye are at certain vertical positions relative to the HMD. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, and provides feedback on the quality of the fit to the user, and may take actions to reduce or minimize effects of any misalignment.
1. A display system configured to project light to an eye of a user to display virtual image content, the display system comprising:
a frame configured to be supported on a head of the user;
a head-mounted display disposed on the frame, the head-mounted display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time;
one or more eye-tracking cameras configured to image a left eye of the user and a right eye of the user; and
processing electronics in communication with the head-mounted display and the one or more eye-tracking cameras, the processing electronics configured to:
determine a left eye tracking confidence score that indicates a confidence level in a position of the left eye of the user and a right eye tracking confidence score that indicates a confidence level in a position of the right eye of the user;
determine whether the head-mounted display is properly registered to the eye of the user by determining whether imaged features of the eye are within a predetermined range of vertical positions relative to the head-mounted display, wherein, when one of the confidence scores is greater than the other, the processing electronics are configured to determine whether the head-mounted display is properly registered based on the left eye or right eye of the user associated with the greater confidence score; and
provide feedback to the user if the head-mounted display is not properly adjusted to fit the user,
wherein the left eye tracking confidence score is based on one or more of detecting a number of glints in the left eye of the user, detecting that the user is blinking, detecting that the left eye of the user is moving, and detecting that the pupil of the left eye of the user is on a boundary of an eye tracking image, and
wherein the right eye tracking confidence score is based on one or more of detecting a number of glints in the right eye of the user, detecting that the user is blinking, detecting that the right eye of the user is moving, and detecting that the pupil of the right eye of the user is on a boundary of an eye tracking image.
2. The display system of claim 1 , wherein the one or more eye-tracking cameras are configured to image a left eye of the user and a right eye of the user, and
wherein, when the left eye and right eye of the user are vertically offset from each other by less than a first predetermined threshold, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on a position of the left eye and right eye of the user that is furthest from a desired vertical position.
3. The display system of claim 2 , wherein, when the left eye and right eye of the user are vertically offset from each other by less than a second predetermined threshold that is less than the first predetermined threshold, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on an average position of the left eye and right eye of the user.
4. The display system of claim 3 , wherein, when the left eye and right eye of the user are vertically offset from each other by more than the first predetermined threshold, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on the average position of the left eye and right eye of the user.
5. The display system of claim 1 , further comprising at least one interchangeable fit piece removably mounted to the frame and configured to adjust a fit of the frame.
6. The display system of claim 5 , wherein the at least one interchangeable fit piece comprises an interchangeable nose bridge configured to adjust the fit of the frame between the frame and a nose bridge of the user.
7. The display system of claim 5 , wherein the at least one interchangeable fit piece comprises an interchangeable forehead pad configured to adjust the fit of the frame between the frame and a forehead of the user.
8. The display system of claim 5 , wherein the at least one interchangeable fit piece comprises an interchangeable back pad configured to adjust the fit of the frame between the frame and a back of the head of the user.
9. The display system of claim 1 , wherein the processing electronics is further configured such that providing feedback to the user that the head-mounted display is not properly adjusted to fit the user comprises providing a suggestion to the user to swap out a currently-installed interchangeable fit piece for another interchangeable fit piece.
10. A method for evaluating registration of virtual image content from a head-mounted display system by a user's eye, the method comprising:
imaging the eye;
determining a left eye tracking confidence score that indicates a confidence level in a position of the left eye of the user and a right eye tracking confidence score that indicates a confidence level in a position of the right eye of the user;
determining whether imaged features of the eye are within a predetermined range of vertical positions relative to a head-mounted display, wherein, when one of the confidence scores is greater than the other, processing electronics are configured to determine whether the head-mounted display is properly registered based on the left eye or right eye of the user associated with the greater confidence score; and
providing a notification based on a determined position of the imaged features, wherein the notification indicates at least that the head-mounted display and the eye are not properly registered,
wherein the left eye tracking confidence score is based on one or more of detecting a number of glints in the left eye of the user, detecting that the user is blinking, detecting that the left eye of the user is moving, and detecting that the pupil of the left eye of the user is on a boundary of an eye tracking image, and
wherein the right eye tracking confidence score is based on one or more of detecting a number of glints in the right eye of the user, detecting that the user is blinking, detecting that the right eye of the user is moving, and detecting that the pupil of the right eye of the user is on a boundary of an eye tracking image.
11. The method of claim 10 , wherein determining whether the imaged features of the eye are within a predetermined range of vertical positions comprises determining positions of glints of the eye.
12. The method of claim 11 , further comprising determining a position of a pupil of the eye based upon the glints of the eye.
13. The method of claim 10 , wherein the head-mounted display system is configured to project light into the eye to display virtual image content in a field of view of the user, and wherein providing the notification comprises displaying the notification as virtual image content.
14. The method of claim 10 , further comprising automatically tracking a pupil of the eye over time and notifying the user when a center of rotation of the eye moves outside of the predetermined range of vertical positions.
15. The method of claim 10 , further comprising changing from a first field of view of the head-mounted display system to a second field of view of the head-mounted display system when the position of the eye is outside a display registration volume,
wherein the head-mounted display system comprises at least one display having the first field of view when the position of the eye is inside the display registration volume, wherein the at least one display has the second field of view when the position of the eye is outside the display registration volume, and wherein the second field of view is smaller than the first field of view.
16. The method of claim 10 , wherein the head-mounted display system comprises at least one interchangeable fit piece, and wherein providing the notification comprises indicating that the head-mounted display system is not properly fitted to the user and suggesting or instructing the user to replace a currently-installed interchangeable fit piece with an alternative interchangeable fit piece.
17. The method of claim 16 , wherein the at least one interchangeable fit piece comprises at least one fit piece selected from the group consisting of: a nose bridge pad, a forehead pad, and a back pad that goes between the head-mounted display system and a back of a user's head.
18. The method of claim 17 , wherein the at least one interchangeable fit piece comprises at least one interchangeable nose bridge pad, and further comprising determining that the head-mounted display is too low with respect to the eye, and wherein providing the notification to the user further comprises prompting the user to install a larger nose bridge pad.
19. The method of claim 10 , further comprising:
identifying a plurality of pixels of a display of the head-mounted display system that the user is expected to perceive as dimmed as a result of a first position of the eye being outside a display registration volume; and
boosting brightness of the plurality of pixels of the display relative to other pixels in the display to mitigate the expected dimming.