IP Library Granted Patent US 10,908,680
Granted Patent B1
US 10,908,680 · App. 16/032,927 · Granted Feb 2, 2021

Pose estimation using electromagnetic tracking

Inventor: Michael Janusz Woods (Mountain View, CA)
Assignee: Magic Leap, Inc.
G06F3/012G01B7/003G01R33/0023G02B27/0172G06F3/0346G06T19/006G02B2027/0138G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,908,680
App. No.
16/032,927
Granted
Feb 2, 2021
Kind
B1
Abstract

Head-mounted augmented reality (AR) devices can track pose of a wearer's head to provide a three-dimensional virtual representation of objects in the wearer's environment. An electromagnetic (EM) tracking system can track head or body pose. A handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field. EM information from the sensor can be analyzed to determine location and/or orientation of the sensor and thereby the wearer's pose. An improved or optimized pose can be provided by reverse-estimating a reverse EM measurement matrix and optimizing the pose based on a comparison between the reverse EM measurement matrix and an EM measurement matrix measured by the EM sensor.

Claims (54)

1. An augmented reality display system comprising:

a display configured to project virtual images to eyes of a wearer;

a frame configured to mount the display in front of the eyes of the wearer;

an electromagnetic (EM) field emitter configured to generate a magnetic field;

an EM sensor configured to sense the magnetic field and provide EM sensor data, wherein one of the EM field emitter or the EM sensor is mechanically coupled to the frame and the other of the EM field emitter or the EM sensor is mechanically coupled to a component of the augmented reality display system that is independently movable relative to the frame;

a hardware processor programmed to:

receive EM sensor data from the EM sensor, the EM sensor data comprising a measured EM measurement matrix;

estimate a six degrees-of-freedom (6-DOF) pose based at least in part on the measured EM measurement matrix;

generate a reverse EM measurement matrix based at least in part on the six-DOF pose;

perform an optimization to reduce a difference between the reverse EM measurement matrix and the measured EM measurement matrix; and

generate an optimized 6-DOF pose based in part on the optimization.

2. The augmented reality display system of claim 1 , wherein the measured EM measurement matrix and the reverse EM measurement matrix comprise 3×3 matrices.

3. The augmented reality display system of claim 1 , wherein the 6-DOF pose comprises three spatial coordinates and three angular coordinates.

4. The augmented reality display system of claim 1 , wherein to perform the optimization, the hardware processor is programmed to apply a least squares algorithm, a gradient descent algorithm, or a regression algorithm.

5. The augmented reality display system of claim 1 , wherein the hardware processor is further programmed to:

receive second EM sensor data from the EM sensor; and

utilize the second EM sensor data and the optimized 6-DOF pose to generate a second optimized 6-DOF pose.

6. The augmented reality display system of claim 5 , wherein the hardware processor is further programmed to:

receive third EM sensor data from the EM sensor; and

utilize the third EM sensor data and the second optimized 6-DOF pose to generate a third optimized 6-DOF pose.

7. The augmented reality display system of claim 1 , wherein:

the component of the augmented reality display system comprises a user-input totem or a belt pack,

the EM sensor is mechanically coupled to the frame, and

the EM field emitter is mechanically coupled to the user-input totem or the belt pack.

8. An augmented reality display system comprising:

an electromagnetic (EM) field emitter configured to generate a magnetic field;

an EM sensor configured to sense the magnetic field and provide EM sensor data, a hardware processor programmed to:

receive the EM sensor data from the EM sensor, the EM sensor data comprising a measured EM measurement matrix;

calculate a pose;

generate a reverse EM measurement matrix based at least in part on the pose;

perform an optimization to reduce a difference between the reverse EM measurement matrix and the measured EM measurement matrix; and

generate an optimized pose based in part on the optimization.

9. The augmented reality display system of claim 8 , wherein the pose or the optimized pose comprises a six degree-of-freedom (6-DOF) pose.

10. The augmented reality display system of claim 9 , wherein to perform the optimization, the hardware processor is programmed to iterate until the difference is below a threshold.

11. The augmented reality display system of claim 8 , wherein the hardware processor is further programmed to:

receive second EM sensor data from the EM sensor; and

utilize the second EM sensor data and the optimized pose to generate a second optimized pose.

12. The augmented reality display system of claim 8 , wherein the hardware processor is further programmed to:

receive non-EM sensor data from a non-EM sensor; and

fuse the non-EM sensor data and the optimized pose to generate a second optimized pose.

13. The augmented reality display system of claim 12 , where the non-EM sensor data is received from an inertial measurement unit (IMU).

14. A method for estimating pose of an electromagnetic (EM) sensor in an EM tracking system, the method comprising:

receiving a measured EM measurement matrix measured by the EM sensor;

calculating a pose of the EM sensor;

generating, based at least partly on the pose, a reverse EM measurement matrix;

decreasing a difference between the reverse EM measurement matrix and the measured EM measurement matrix; and

generating an improved pose of the EM sensor based, at least in part, on the difference between the reverse EM measurement matrix and the measured EM measurement matrix.

15. The method of claim 14 , wherein the pose or the improved pose comprise a six degree-of-freedom (6-DOF) pose.

16. The method of claim 14 , wherein decreasing the difference between the reverse EM measurement matrix and the measured EM measurement matrix comprises applying an optimization algorithm.

17. The method of claim 16 , wherein the optimization algorithm comprises least squares, regression, or gradient descent.

18. An electromagnetic tracking system comprising:

an EM emitter configured to generate a magnetic field;

an EM sensor configured to sense the magnetic field; and

a hardware processor programmed to perform the method of claim 14 .

Assignments (5)
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 9, 2020
From: WOODS, MICHAEL JANUSZ
To: MAGIC LEAP, INC.
Reel/Frame 052882/0487 →
ASSIGNMENT OF SECURITY INTEREST IN PATENTS Recorded Nov 7, 2019
From: JPMORGAN CHASE BANK, N.A.
To: CITIBANK, N.A.
Reel/Frame 050967/0138 →
PATENT SECURITY AGREEMENT Recorded Aug 22, 2019
From: MAGIC LEAP, INC.; MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC
To: JP MORGAN CHASE BANK, N.A.
Reel/Frame 050138/0287 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 12, 2018
From: WOODS, MICHAEL JANUSZ
To: MAGIC LEAP, INC.
Reel/Frame 046338/0183 →
Cited By (3)
US 12,373,024 US 12,440,416 US 12,529,895