IP Library Granted Patent US 11,244,649
Granted Patent B2
US 11,244,649 · App. 17/088,498 · Granted Feb 8, 2022

Calibration of augmented reality device

Inventors: Yu-Tseh Chi (Cupertino, CA); Jean-Yves Bouguet (Palo Alto, CA); Divya Sharma (San Jose, CA); Lei Huang (Fremont, CA); Dennis William Strelow (San Jose, CA); Etienne Gregoire Grossmann (Menlo Park, CA); Evan Gregory Levine (Mountain View, CA); Adam Harmat (Sunnyvale, CA); Ashwin Swaminathan (Dublin, CA)
Assignee: Magic Leap, Inc.
G09G5/003G06T7/0002G06T7/85G06T2207/10012
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,244,649
App. No.
17/088,498
Granted
Feb 8, 2022
Kind
B2
Abstract

Techniques are described for calibrating a device having a first sensor and a second sensor. Techniques include capturing sensor data using the first sensor and the second sensor. The device maintains a calibration profile including a translation parameter and a rotation parameter to model a spatial relationship between the first sensor and the second sensor. Techniques include determining a calibration level associated with the calibration profile at a first time. Techniques include determining, based on the calibration level, to perform a calibration process. Techniques include performing the calibration process at the first time by generating one or both of a calibrated translation parameter and a calibrated rotation parameter and replacing one or both of the translation parameter and the rotation parameter with one or both of the calibrated translation parameter and the calibrated rotation parameter.

Claims (63)

1. A method of updating a calibration profile that models a spatial relationship between a first sensor and a second sensor of a device, the method comprising:

capturing sensor data using one or more sensors of the device;

determining a calibration level associated with the calibration profile based on the sensor data, wherein the calibration profile includes a translation parameter and a rotation parameter; and

determining, based on the calibration level, to perform either a first calibration process or a second calibration process;

wherein performing the first calibration process includes:

generating a calibrated rotation parameter; and

replacing the rotation parameter with the calibrated rotation parameter;

wherein performing the second calibration process includes:

generating both a calibrated translation parameter and the calibrated rotation parameter; and

replacing both the translation parameter and the rotation parameter with both the calibrated translation parameter and the calibrated rotation parameter.

2. The method of claim 1 , wherein performing the first calibration process includes:

replacing only the rotation parameter with the calibrated rotation parameter.

3. The method of claim 1 , wherein the one or more sensors include the first sensor and the second sensor.

4. The method of claim 3 , wherein the sensor data includes:

one or more first images captured using the first sensor; and

one or more second images captured using the second sensor.

5. The method of claim 4 , wherein one or both of the calibrated translation parameter and the calibrated rotation parameter are generated using the sensor data.

6. The method of claim 1 , wherein the one or more sensors include an additional sensor and does not include the first sensor and the second sensor.

7. The method of claim 1 , wherein:

it is determined to perform the first calibration process when the calibration level is less than a calibration threshold; and

it is determined to perform the second calibration process when the calibration level is greater than the calibration threshold.

8. A device comprising:

a first sensor and a second sensor;

a memory device configured to store a calibration profile modeling a spatial relationship between the first sensor and the second sensor, the calibration profile including a translation parameter and a rotation parameter;

a processor coupled to the memory device, wherein the processor is configured to perform operations comprising:

capturing sensor data using one or more sensors of the device;

determining a calibration level associated with the calibration profile based on the sensor data; and

determining, based on the calibration level, to perform either a first calibration process or a second calibration process;

wherein performing the first calibration process includes:

generating a calibrated rotation parameter; and

replacing the rotation parameter with the calibrated rotation parameter;

wherein performing the second calibration process includes:

generating both a calibrated translation parameter and the calibrated rotation parameter; and

replacing both the translation parameter and the rotation parameter with both the calibrated translation parameter and the calibrated rotation parameter.

9. The device of claim 8 , wherein performing the first calibration process includes:

replacing only the rotation parameter with the calibrated rotation parameter.

10. The device of claim 8 , wherein the one or more sensors include the first sensor and the second sensor.

11. The device of claim 10 , wherein the sensor data includes:

one or more first images captured using the first sensor; and

one or more second images captured using the second sensor.

12. The device of claim 11 , wherein one or both of the calibrated translation parameter and the calibrated rotation parameter are generated using the sensor data.

13. The device of claim 8 , wherein the one or more sensors include an additional sensor and does not include the first sensor and the second sensor.

14. A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform operations for updating a calibration profile that models a spatial relationship between a first sensor and a second sensor of a device, the operations comprising:

capturing sensor data using one or more sensors of the device;

determining a calibration level associated with the calibration profile based on the sensor data, wherein the calibration profile includes a translation parameter and a rotation parameter; and

determining, based on the calibration level, to perform either a first calibration process or a second calibration process;

wherein performing the first calibration process includes:

generating a calibrated rotation parameter; and

replacing the rotation parameter with the calibrated rotation parameter;

wherein performing the second calibration process includes:

generating both a calibrated translation parameter and the calibrated rotation parameter; and

replacing both the translation parameter and the rotation parameter with both the calibrated translation parameter and the calibrated rotation parameter.

15. The non-transitory computer-readable medium of claim 14 , wherein performing the first calibration process includes:

replacing only the rotation parameter with the calibrated rotation parameter.

16. The non-transitory computer-readable medium of claim 14 , wherein the one or more sensors include the first sensor and the second sensor.

17. The non-transitory computer-readable medium of claim 16 , wherein the sensor data includes:

one or more first images captured using the first sensor; and

one or more second images captured using the second sensor.

18. The non-transitory computer-readable medium of claim 17 , wherein one or both of the calibrated translation parameter and the calibrated rotation parameter are generated using the sensor data.

19. The non-transitory computer-readable medium of claim 14 , wherein the one or more sensors include an additional sensor and does not include the first sensor and the second sensor.

20. The non-transitory computer-readable medium of claim 14 , wherein:

it is determined to perform the first calibration process when the calibration level is less than a calibration threshold; and

it is determined to perform the second calibration process when the calibration level is greater than the calibration threshold.

Assignments (2)
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 4, 2020
From: CHI, YU-TSEH; BOUGUET, JEAN-YVES; SHARMA, DIVYA; HUANG, LEI; STRELOW, DENNIS WILLIAM; GROSSMANN, ETIENNE GREGOIRE; LEVINE, EVAN GREGORY; HARMAT, ADAM; SWAMINATHAN, ASHWIN
To: MAGIC LEAP, INC.
Reel/Frame 054273/0917 →
Cited By (3)
US 12,333,762 US 12,435,996 US 12,456,225