IP Library Granted Patent US 11,042,034
Granted Patent B2
US 11,042,034 · App. 16/557,419 · Granted Jun 22, 2021

Head mounted display calibration using portable docking station with calibration target

Inventors: Sebastian Sztuk (Menlo Park, CA); Javier San Agustin Lopez (Menlo Park, CA); Sapna Shroff (Sunnyvale, CA); Robert Dale Cavin (Seattle, WA); Alexander Jobe Fix (Seattle, WA)
Assignee: Facebook Technologies, LLC
G02B27/0172G02B27/0176G06F3/013H02J7/0044G02B2027/014G02B2027/0138G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,042,034
App. No.
16/557,419
Granted
Jun 22, 2021
Kind
B2
Abstract

A system is describes that includes a head mounted display (HMD) and a portable docking station configured to receive the HMD for calibration of one or more components of the HMD. The portable docking station includes at least one calibration target, e.g., a checkerboard pattern and/or a convex reflector. Techniques of this disclosure include calibrating an image capture device of the HMD based on one or more images of the calibration target captured by the image capture device when the HMD is placed in the portable docking station. The disclosed techniques may be applied to calibrate multiple different components of the HMD, including image capture devices such as eye-tracking cameras and inside-out cameras, displays, illuminators, sensors, and the like. In some examples, a rechargeable battery of the HMD may be charged when the HMD is placed in the portable docking station.

Claims (50)

1. A system comprising:

a head mounted display (HMD) comprising at least one image capture device;

a portable docking station configured to receive the HMD, the portable docking station including at least one calibration target that is within a field of view of the at least one image capture device when the HMD is placed in the portable docking station; and

a processor executing a calibration engine configured to calibrate the at least one image capture device of the HMD based on one or more images of the at least one calibration target captured by the at least one image capture device when the HMD is placed in the portable docking station.

2. The system of claim 1 , wherein the HMD further comprises one or more of a display, at least one illuminator, or at least one sensor, and wherein the calibration engine is further configured to calibrate at least one of the display, the at least one illuminator, or the at least one sensor with respect to the at least one image capture device.

3. The system of claim 1 , wherein to calibrate the at least one image capture device, the calibration engine is configured to:

determine one or more of intrinsic parameters or extrinsic parameters of the at least one image capture device based on the one or more images of the at least one calibration target captured by the at least one image capture device when the HMD is placed in the portable docking station; and

configure the at least one image capture device of the HMD to operate according to the one or more of the intrinsic parameters or the extrinsic parameters.

4. The system of claim 1 ,

wherein the HMD further comprises at least one illuminator;

wherein the at least one image capture device comprises an eye-tracking camera;

wherein the at least one calibration target included in the portable docking station comprises a convex reflector for reflecting light emitted by the at least one illuminator towards the eye-tracking camera; and

wherein the calibration engine is configured to determine extrinsic parameters of the eye-tracking camera based on one or more images of the reflected light captured by the eye-tracking camera.

5. The system of claim 1 ,

wherein the at least one image capture device comprises an eye-tracking camera;

wherein the at least one calibration target included in the portable docking station comprises a checkerboard pattern within the field of view of the eye-tracking camera; and

wherein the calibration engine is configured to determine intrinsic parameters of the eye-tracking camera based on one or more images of the checkerboard pattern captured by the eye-tracking camera.

6. The system of claim 1 ,

wherein the at least one calibration target included in the portable docking station includes one or more fiducial marks either adjacent to the at least one calibration target or embedded within the at least one calibration target; and

wherein the calibration engine is configured to:

determine a spatial relationship between a position of the HMD when the HMD is placed in the portable docking station and a position of the at least one calibration target within the portable docking station based on the one or more fiducial marks, and

calibrate the at least one image capture device further based on the determined spatial relationship between the position of the HMD and the position of the at least one calibration target.

7. The system of claim 1 ,

wherein the portable docking station includes one or more fixtures configured to receive and hold the HMD in a fixed position when the HMD is placed in the portable docking station; and

wherein the calibration engine is configured to calibrate the at least one image capture device further based on a known spatial relationship between the fixed position of the HMD and a position of the at least one calibration target within the portable docking station.

8. The system of claim 1 , wherein the calibration engine is configured to calibrate the at least one image capture device in response to at least one of determining that the HMD has been received within the portable docking station or determining that a rechargeable battery of the HMD is charged to at least a threshold charge level while the HMD is within the portable docking station.

9. The system of claim 1 , further comprising a charging circuit configured to charge a rechargeable battery of the HMD when the HMD is placed in the portable docking station.

10. The system of claim 1 ,

wherein the at least one image capture device of the HMD comprises at least one first image capture device, the system further comprising a peripheral device comprising at least one second image capture device; and

wherein the portable docking station is further configured to receive the peripheral device adjacent to the HMD and position the peripheral device such that a display of the HMD is within a field of view of the at least one second image capture device of the peripheral device when the HMD is placed in the portable docking station.

11. The system of claim 10 , wherein the calibration engine is further configured to calibrate the display of the HMD and the at least one second image capture device of the peripheral device based on one or more images produced on the display of the HMD that are captured by the at least one second image capture device of the peripheral device when both the HMD and the peripheral device are placed in the portable docking station.

12. The system of claim 11 ,

wherein the HMD includes one or more fiducial marks that are within the field of view of the at least one second image capture device of the peripheral device when both the HMD and the peripheral device are placed in the portable docking station; and

wherein the calibration engine is configured to:

determine a spatial relationship between a position of the HMD and a position of the peripheral device when both the HMD and the peripheral device are placed in the portable docking station based on the one or more fiducial marks, and

calibrate the display of the HMD and the at least one second image capture device of the peripheral device further based on the determined spatial relationship between the position of the HMD and the position of the peripheral device.

13. The system of claim 10 , wherein the processor is integrated within one of the HMD or the peripheral device.

14. The system of claim 1 , wherein the portable docking station comprises one of a carrying case or a cover for the HMD.

15. The system of claim 1 , wherein the HMD comprises artificial reality glasses, and wherein the at least one calibration target is positioned at a location along an arm of the artificial reality glasses so that the at least one calibration target is within the field of view of the at least one image capture device when the arm of the artificial reality glasses is folded for placement of the artificial reality glasses in the portable docking station.

16. The system of claim 1 , wherein the processor is integrated within the HMD.

17. A method comprising:

receiving, by a portable docking station, a head mounted display (HMD) comprising at least one image capture device, wherein the portable docking station includes at least one calibration target that is within a field of view of the at least one image capture device when the HMD is placed in the portable docking station;

determining that the at least one image capture device of the HMD is to be calibrated; and

calibrating the at least one image capture device of the HMD based on one or more images of the at least one calibration target captured by the at least one image capture device when the HMD is placed in the portable docking station.

18. The method of claim 17 , wherein determining that the at least one image capture device of the HDM is to be calibrated comprises determining that the HMD has been received by the portable docking station based on one of a proximity sensor or a magnetic sensor.

19. The method of claim 17 , wherein determining that the at least one image capture device of the HDM is to be calibrated comprises determining that a rechargeable battery of the HMD is charged to at least a threshold charge level while the HMD is within the portable docking station.

20. A non-transitory computer-readable medium comprising instruction that, when executed, cause on or more processors to:

determine that a head mounted display (HMD) has been received by a portable docking station, wherein the portable docking station includes at least one calibration target that is within a field of view of at least one image capture device of the HMD when the HMD is placed in the portable docking station;

determine that the at least one image capture device of the HMD is to be calibrated; and

calibrate the at least one image capture device of the HMD based on one or more images of the at least one calibration target captured by the at least one image capture device when the HMD is placed in the portable docking station.

Assignments (2)
CHANGE OF NAME Recorded Jul 21, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060802/0799 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 30, 2019
From: SZTUK, SEBASTIAN; SAN AGUSTIN LOPEZ, JAVIER; SHROFF, SAPNA; CAVIN, ROBERT DALE; FIX, ALEXANDER JOBE
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 050556/0238 →
Continuity (2)
Provisional Application 62785595 · Dec 27, 2018
Related Publication 20200209628A1 · Jul 2, 2020
Cited By (1)
US 12,259,560