IP Library Granted Patent US 10,991,343
Granted Patent B2
US 10,991,343 · App. 16/532,308 · Granted Apr 27, 2021

Automatic image alignment with head mounted display optics

Inventors: Wei Rong (San Jose, CA); Nirav Rajendra Patel (San Francisco, CA)
Assignee: Facebook Technologies, LLC
G09G5/38G09G2354/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,991,343
App. No.
16/532,308
Granted
Apr 27, 2021
Kind
B2
Abstract

A method of automatic image alignment with head mounted display (HMD) optics includes generating a pixel pattern of display light on an electronic display of the HMD. A light sensor of the HMD then generates light measurements in response to the pixel pattern, where the light measurements are representative of a lateral position of an optical element of the HMD. The method also includes adjusting a rendering center of the electronic display for presentation of one or more images with respect to the optical element based on the light measurements.

Claims (49)

1. A computer-implemented method for use with a head mounted display (HMD), the method comprising:

generating a pixel pattern of display light on an electronic display of the HMD;

varying a location of the pixel pattern on the electronic display;

generating light measurements of the display light with a light sensor of the HMD while the location of the pixel pattern is varied, wherein the light measurements are representative of a lateral position of an optical element of the HMD; and

adjusting a rendering center of the electronic display for presentation of one or more images with respect to the optical element based on the light measurements wherein adjusting the rendering center of the electronic display includes adjusting a location of where the one or more images are to be presented on the electronic display.

2. The computer-implemented method of claim 1 , wherein adjusting the rendering center of the electronic display comprises:

determining a position of a center of the optical element based on the light measurements; and

aligning the rendering center of the electronic display with the center of the optical element.

3. The computer-implemented method of claim 2 , wherein determining the position of the center of the optical element comprises:

determining a first display coordinate of the electronic display that is aligned with the light sensor based on corresponding intensity values of the light measurements; and

determining a second display coordinate of the electronic display that is aligned with the center of the optical element based on the first display coordinate and a calibrated distance between the light sensor and the center of the optical element.

4. The computer-implemented method of claim 1 , wherein the pixel pattern comprises a single column of pixels, wherein each of the light measurements includes an intensity value measured by the light sensor while the single column of pixels is illuminated at a respective display coordinate of the electronic display.

5. The computer-implemented method of claim 1 , wherein the pixel pattern comprises multiple columns of pixels, wherein each of the light measurements includes an intensity value measured by the light sensor while the multiple columns of pixels are simultaneously illuminated at a respective display coordinate of the electronic display.

6. The computer-implemented method of claim 1 , wherein the pixel pattern comprises at least one column of pixels segmented into several groups of pixels, wherein each of the light measurements includes an intensity value measured by the light sensor while the at least one column of pixels is illuminated at a respective display coordinate of the electronic display.

7. The computer-implemented method of claim 1 , wherein the pixel pattern comprises a single pixel, wherein each of the light measurements includes an intensity value measured by the light sensor while the single pixel is illuminated at a respective display coordinate of the electronic display.

8. A head mounted display (HMD), comprising:

an electronic display configured to generate display light;

an optical element configured to pass the display light to a user of the HMD;

a light sensor coupled to the optical element;

at least one processor; and

at least one memory coupled to the at least one processor, the at least one memory having instructions stored therein, which when executed by the at least one processor, direct the HMD to:

generate a pixel pattern of the display light on the electronic display;

vary a location of the pixel pattern on the electronic display;

generate light measurements with the light sensor while the location of the pixel pattern is varied, wherein the light measurements are representative of a lateral position of the optical element; and

adjust a rendering center of the electronic display for presentation of one or more images with respect to the optical element based on the light measurements wherein the instructions to adjust the rendering center of the electronic display includes instructions to adjust a location of where the one or more images are to be presented on the electronic display.

9. The HMD of claim 8 , wherein the instructions to adjust the rendering center of the electronic display comprises instructions to direct the HMD to:

determine a position of a center of the optical element based on the light measurements; and

align the rendering center of the electronic display with the center of the optical element.

10. The HMD of claim 9 , wherein the instructions to determine the position of the center of the optical element comprises instructions to direct the HMD to:

determine a first display coordinate of the electronic display that is aligned with the light sensor based on corresponding intensity values of the light measurements; and

determine a second display coordinate of the electronic display that is aligned with the center of the optical element based on the first display coordinate and a calibrated distance between the light sensor and the center of the optical element.

11. The HMD of claim 8 , wherein the pixel pattern comprises a single column of pixels, wherein each of the light measurements includes an intensity value measured by the light sensor while the single column of pixels is illuminated at a respective display coordinate of the electronic display.

12. The HMD of claim 8 , wherein the pixel pattern comprises multiple columns of pixels, wherein each of the light measurements includes an intensity value measured by the light sensor while the multiple columns of pixels are simultaneously illuminated at a respective display coordinate of the electronic display.

13. The HMD of claim 8 , wherein the pixel pattern comprises at least one column of pixels segmented into several groups of pixels, wherein each of the light measurements includes an intensity value measured by the light sensor while the at least one column of pixels is illuminated at a respective display coordinate of the electronic display.

14. One or more non-transitory computer-readable media storing computer-executable instructions, which when executed by at least one processor, direct the processor to:

generate a pixel pattern of display light by an electronic display;

vary a location of the pixel pattern on the electronic display;

generate light measurements with a light sensor while the location of the pixel pattern is varied, wherein the light measurements are representative of a lateral position of an optical element; and

adjust a rendering center of the electronic display for presentation of one or more images with respect to the optical element based on the light measurements, wherein the instructions to adjust the rendering center of the electronic display includes instructions to adjust a location of where the one or more images are to be presented on the electronic display.

15. The one or more non-transitory computer-readable media of claim 14 , wherein the instructions to adjust the rendering center of the electronic display comprises instructions to direct the processor to:

determine a position of a center of the optical element based on the light measurements; and

align the rendering center of the electronic display with the center of the optical element.

16. The one or more non-transitory computer-readable media of claim 15 , wherein the instructions to determine the position of the center of the optical element comprises instructions to direct the processor to:

determine a first display coordinate of the electronic display that is aligned with the light sensor based on corresponding intensity values of the light measurements; and

determine a second display coordinate of the electronic display that is aligned with the center of the optical element based on the first display coordinate and a calibrated distance between the light sensor and the center of the optical element.

17. The one or more non-transitory computer-readable media of claim 14 , wherein the pixel pattern comprises a single column of pixels, wherein each of the light measurements includes an intensity value measured by the light sensor while the single column of pixels is illuminated at a respective display coordinate of the electronic display.

18. The one or more non-transitory computer-readable media of claim 14 , wherein the pixel pattern comprises multiple columns of pixels, wherein each of the light measurements includes an intensity value measured by the light sensor while the multiple columns of pixels are simultaneously illuminated at a respective display coordinate of the electronic display.

19. The one or more non-transitory computer-readable media of claim 14 , wherein the pixel pattern comprises at least one column of pixels segmented into several groups of pixels, wherein each of the light measurements includes an intensity value measured by the light sensor while the at least one column of pixels is illuminated at a respective display coordinate of the electronic display.

20. The HMD of claim 8 , wherein the optical element comprises one or more of a lens, a mirror, a diffuser, a filter, a polarizer, a prism, a window, a beam splitter, or a diffractive element.

Assignments (2)
CHANGE OF NAME Recorded Jun 1, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060246/0845 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 27, 2019
From: RONG, WEI; PATEL, NIRAV RAJENDRA
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 050186/0800 →
Continuity (1)
Related Publication 20210043170A1 · Feb 11, 2021
Cited By (1)
US 12,304,471