Pose estimation using electromagnetic tracking
Head-mounted augmented reality (AR) devices can track pose of a wearer's head to provide a three-dimensional virtual representation of objects in the wearer's environment. An electromagnetic (EM) tracking system can track head or body pose. A handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field. EM information from the sensor can be analyzed to determine location and/or orientation of the sensor and thereby the wearer's pose. An improved or optimized pose can be provided by reverse-estimating a reverse EM measurement matrix and optimizing the pose based on a comparison between the reverse EM measurement matrix and an EM measurement matrix measured by the EM sensor.
1. An augmented reality display system comprising:
a display configured to project virtual images to eyes of a wearer;
a frame configured to mount the display in front of the eyes of the wearer; and
a hardware processor programmed to:
receive EM sensor data from an EM sensor configured to sense a magnetic field generated by an EM field emitter, wherein one of the EM field emitter or the EM sensor is mechanically coupled to the frame and the other of the EM field emitter or the EM sensor is mechanically coupled to a component of the augmented reality display system that is independently movable relative to the frame, the EM sensor data comprising a measured EM measurement matrix;
estimate a six degrees-of-freedom (6-DOF) pose based at least in part on the measured EM measurement matrix;
generate a reverse EM measurement matrix based at least in part on the six-DOF pose;
perform an optimization to reduce a difference between the reverse EM measurement matrix and the measured EM measurement matrix; and
generate an optimized 6-DOF pose based in part on the optimization.
2. The augmented reality display system of claim 1 , wherein the measured EM measurement matrix and the reverse EM measurement matrix comprise 3×3 matrices.
3. The augmented reality display system of claim 1 , wherein the 6-DOF pose comprises three spatial coordinates and three angular coordinates.
4. The augmented reality display system of claim 1 , wherein to perform the optimization, the hardware processor is programmed to apply a least squares algorithm, a gradient descent algorithm, or a regression algorithm.
5. The augmented reality display system of claim 1 , wherein the hardware processor is further programmed to:
receive second EM sensor data from the EM sensor; and
utilize the second EM sensor data and the optimized 6-DOF pose to generate a second optimized 6-DOF pose.
6. The augmented reality display system of claim 5 , wherein the hardware processor is further programmed to:
receive third EM sensor data from the EM sensor; and
utilize the third EM sensor data and the second optimized 6-DOF pose to generate a third optimized 6-DOF pose.
7. The augmented reality display system of claim 1 , wherein the component of the augmented reality display system comprises a user-input totem or a belt pack.
8. A hardware processor for an augmented reality display system, programmed to:
receive EM sensor data from an EM sensor, the EM sensor data comprising a measured EM measurement matrix;
calculate a pose;
generate a reverse EM measurement matrix based at least in part on the pose;
perform an optimization to reduce a difference between the reverse EM measurement matrix and the measured EM measurement matrix; and
generate an optimized pose based in part on the optimization.
9. The hardware processor for the augmented reality display system of claim 8 , wherein the pose or the optimized pose comprises a six degree-of-freedom (6-DOF) pose.
10. The hardware processor for the augmented reality display system of claim 9 , wherein to perform the optimization, the hardware processor is programmed to iterate until the difference is below a threshold.
11. The hardware processor for the augmented reality display system of claim 8 , wherein the hardware processor is further programmed to:
receive second EM sensor data from the EM sensor; and
utilize the second EM sensor data and the optimized pose to generate a second optimized pose.
12. The hardware processor for the augmented reality display system of claim 8 , wherein the hardware processor is further programmed to:
receive non-EM sensor data from a non-EM sensor; and
fuse the non-EM sensor data and the optimized pose to generate a second optimized pose.
13. The hardware processor for the augmented reality display system of claim 12 , where the non-EM sensor data is received from an inertial measurement unit (IMU).
14. A method for estimating pose of an electromagnetic (EM) sensor in an EM tracking system, the method comprising:
calculating a pose of the EM sensor based on an EM measurement matrix measured by the EM sensor;
generating, based at least partly on the pose, a reverse EM measurement matrix; and
generating an improved pose of the EM sensor based, at least in part, on a difference between the reverse EM measurement matrix and the measured EM measurement matrix.
15. The method of claim 14 , wherein the pose or the improved pose comprise a six degree-of-freedom (6-DOF) pose.
16. The method of claim 14 , wherein decreasing the difference between the reverse EM measurement matrix and the measured EM measurement matrix comprises applying an optimization algorithm.
17. The method of claim 16 , wherein the optimization algorithm comprises least squares, regression, or gradient descent.
18. An electromagnetic tracking system comprising:
an EM emitter configured to generate a magnetic field;
an EM sensor configured to sense the magnetic field; and
a hardware processor programmed to perform the method of claim 14 .