IP Library Granted Patent US 10,714,055
Granted Patent B1
US 10,714,055 · App. 16/008,623 · Granted Jul 14, 2020

Systems and methods for display synchronization in head-mounted display devices

Inventors: Hauke Malte Strasdat (Kirkland, WA); Richard Andrew Newcombe (Seattle, WA); Jeffrey Scott Campeau (Kirkland, WA)
Assignee: Facebook Technologies, LLC
G09G5/12G06K19/06037G09G5/006H04N5/247G06F3/012G09G2320/0261G09G2320/0693G09G2370/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,714,055
App. No.
16/008,623
Granted
Jul 14, 2020
Kind
B1
Abstract

The disclosed computer-implemented method may include directing a head-mounted display system to render, within a frame generated by the head-mounted display system, at least one performance metric for the frame. The method may also include recording when the frame containing the performance metric is displayed by a display of the head-mounted display system, measuring, based at least in part on when the frame containing the performance metric was displayed, at least one aspect of the head-mounted display system's performance, and calibrating the head-mounted display system based on the measurement. Various other methods, calibration systems, and computer-readable media are also disclosed.

Claims (57)

1. A method for calibrating a head-mounted display system, the method comprising:

directing the head-mounted display system to render, within a frame generated by the head-mounted display system, at least one performance metric for the frame;

recording when the frame containing the performance metric is displayed by a display of the head-mounted display system, wherein recording when the frame containing the performance metric is displayed comprises:

capturing an image of the displayed frame using a camera that is positioned approximately where a user's eyes would be positioned with respect to the display when the head-mounted display system is in use by the user; and

recording when the image of the displayed frame was captured by the camera;

measuring, based at least in part on when the frame containing the performance metric was displayed, at least one aspect of the head-mounted display system's performance; and

calibrating the head-mounted display system based on the measurement.

2. The method of claim 1 , wherein directing the head-mounted display system to render the performance metric within the frame comprises directing the head-mounted display system to render a machine-readable representation of the performance metric within the frame.

3. The method of claim 2 , wherein the machine-readable representation comprises at least one of:

a numeral; or

a binary code.

4. The method of claim 3 , wherein the binary code comprises a two-dimensional barcode.

5. The method of claim 1 , wherein the image of the displayed frame comprises a computer-generated image.

6. The method of claim 1 , wherein the performance metric comprises at least one of:

a predicted display time that identifies a predicted point in time when the head-mounted display system will display the frame on the display of the head-mounted display system;

a frame identifier that identifies the frame; or

a synchronization value that identifies synchronization errors between the display and a graphics processing unit of the head-mounted display system.

7. The method of claim 6 , wherein measuring at least one aspect of the head-mounted display system's performance comprises:

determining, using a timestamp obtained from the camera that captured the displayed frame, an actual display time for the frame; and

comparing the actual display time with the predicted display time.

8. The method of claim 7 , wherein measuring at least one aspect of the head-mounted display system's performance comprises using the frame identifier to identify the frame within a sequence of frames reported by a subsystem of the head-mounted display system.

9. The method of claim 1 , further comprising:

directing the head-mounted display system to render the performance metric for each of a plurality of additional frames generated by the head-mounted display system; and

recording when the additional frames containing the performance metric are displayed by the display of the head-mounted display system; and

wherein the measurement of at least one aspect of the head-mounted display system's performance is based at least in part on an aggregation of when the additional frames containing the performance metric were displayed.

10. The method of claim 1 , wherein directing the head-mounted display system to render the performance metric within the frame comprises directing the head-mounted display system to render frames at a frequency that is higher than a frame rate of an image sensor used by the head-mounted display system to capture the frame.

11. The method of claim 1 , wherein directing the head-mounted display system to render the performance metric within the frame comprises directing the head-mounted display system to display the performance metric for an amount of time that is approximately equal to an exposure length of an image sensor used by the head-mounted display system to capture the frame.

12. The method of claim 1 , wherein calibrating the head-mounted display system comprises calibrating at least one of:

the display of the head-mounted display system;

a sensor of the head-mounted display system;

a display-rendering property of the head-mounted display system; or

a display-timing property of the head-mounted display system.

13. A calibration system for a head-mounted display system, the calibration system comprising:

at least one memory device; and

a physical processor that, when executing instructions contained within the memory device, causes the calibration system to:

direct the head-mounted display system to render, within a frame generated by the head-mounted display system, at least one performance metric for the frame;

record when the frame containing the performance metric is displayed by a display of the head-mounted display system, wherein recording when the frame containing the performance metric is displayed comprises:

capturing an image of the displayed frame using a camera subsystem that is positioned approximately where a user's eyes would be positioned with respect to the display when the head-mounted display system is in use by the user; and

recording when the image of the displayed frame was captured by the camera;

measure, based at least in part on when the frame containing the performance metric was displayed, at least one aspect of the head-mounted display system's performance; and

calibrate the head-mounted display system based on the measurement.

14. The calibration system of claim 13 , wherein the camera subsystem is configured to capture images of frames displayed by the display of the head-mounted display system.

15. The calibration system of claim 13 , wherein the camera subsystem comprises a pair of cameras that are positioned to approximate positions of the user's eyes with respect to the display when the head-mounted display system is in use by the user.

16. The calibration system of claim 15 , wherein:

the display comprises a first display and a second display;

a first camera within the pair of cameras is configured to capture images of frames displayed by the first display; and

a second camera within the pair of cameras is configured to capture images of frames displayed by the second display.

17. The calibration system of claim 16 , wherein the calibration system is configured to calibrate the first display independent of the second display.

18. A tangible, non-transitory computer-readable medium having instructions stored thereon, that when executed by a physical processor, cause the physical processor to:

direct a head-mounted display system to render, within a frame generated by the head-mounted display system, at least one performance metric for the frame;

record when the frame containing the performance metric is displayed by a display of the head-mounted display system, wherein recording when the frame containing the performance metric is displayed comprises:

capturing an image of the displayed frame using a camera that is positioned approximately where a user's eyes would be positioned with respect to the display when the head-mounted display system is in use by the user; and

recording when the image of the displayed frame was captured by the camera;

measure, based at least in part on when the frame containing the performance metric was displayed, at least one aspect of the head-mounted display system's performance; and

calibrate the head-mounted display system based on the measurement.

19. The tangible, non-transitory computer-readable medium of claim 18 , wherein directing the head-mounted display system to render the performance metric within the frame comprises directing the head-mounted display system to render a machine-readable representation of the performance metric within the frame.

20. The tangible, non-transitory computer-readable medium of claim 19 , wherein the machine-readable representation comprises a two-dimensional barcode.

Assignments (3)
CHANGE OF NAME Recorded May 26, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060199/0876 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 5, 2021
From: STRASDAT, HAUKE M; NEWCOMBE, RICHARD A; CAMPEAU, JEFFREY S
To: OCULUS VR, LLC
Reel/Frame 056143/0493 →
CHANGE OF NAME Recorded Sep 20, 2018
From: OCULUS VR, LLC
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 047112/0836 →
Cited By (1)
US 12,504,325