Ambient electromagnetic distortion correction for electromagnetic tracking
Head-mounted augmented reality (AR) devices can track pose of a wearer's head to provide a three-dimensional virtual representation of objects in the wearer's environment. An electromagnetic (EM) tracking system can track head or body pose. A handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field (e.g., for determining head pose). The generated EM field may be distorted due to nearby electrical conductors or ferromagnetic materials, which may lead to error in the determined pose. Systems and methods are disclosed that measure the degree of EM distortion, as well as correct for the EM distortion. The EM distortion correction may be performed in real time by the EM tracking system without the need for additional data from imaging cameras or other sensors.
1. A system comprising:
a head mounted wearable component including an outward-facing imaging system;
an electromagnetic (EM) emitter configured to generate a magnetic field;
an EM sensor configured to sense the magnetic field and provide EM sensor data;
a hand-held user input device, wherein the EM emitter or the EM sensor is included in the hand-held user input device; and
a hardware processor programmed to:
based on the hand-held user input device with the EM emitter or EM sensor not being in a field of view of the outward-facing imaging system of the head mounted wearable component, determine that EM distortion correction is not needed;
based on the hand-held user input device with the EM emitter or EM sensor being in the field of view of the outward-facing imaging system of the head mounted wearable component, determine that EM distortion correction is needed and:
receive EM sensor data from the EM sensor, the EM sensor data comprising an EM field matrix;
detect presence of EM distortion caused by the EM emitter or EM sensor of the hand-held user input device being present in the field of view of the outward-facing imaging system of the head mounted wearable component;
when either the detected EM distortion is above a first threshold or a change in the detected EM distortion resulting from real-time motion of the system is above a second threshold:
calculate, based at least partly on the EM sensor data, distortion compensating matrices such that a determinant of each of the distortion compensating matrices is close to unity;
calculate, based at least partly on the distortion compensating matrices, an undistorted EM field matrix such that eigenvalues of the undistorted EM field matrix are proportional to [1, ½, ½]; and
calculate, based at least partly on the undistorted EM field matrix, a pose of the EM sensor; and
when neither the detected EM distortion is above the first threshold nor a change in the detected EM distortion resulting from real-time motion of the system is above the second threshold:
not calculate, based at least partly on the EM sensor data, distortion compensating matrices.
2. The system of claim 1 ,
wherein the hardware processor is programmed to:
analyze images obtained by the outward-facing imaging system;
determine, based at least partly on the analyzed images, a pose of the user input device; and
calculate the distortion compensating matrices based at least in part on the determined pose of the user input device.
3. The system of claim 1 , wherein the distortion compensating matrices comprise a first matrix CL and a second matrix CR, and to calculate the undistorted EM field matrix, the hardware processor is programmed to evaluate CL Fd CR where Fd is the EM field matrix.
4. The system of claim 1 , wherein to calculate the pose of the EM sensor, the hardware processor is programmed to use direct computational methods that do not utilize numerical iteration.
5. The system of any claim 1 , wherein the hardware processor is further programmed to render a virtual object based at least in part on the calculated pose.
6. The system of claim 1 , wherein the hardware processor is further programmed to calculate a degree of EM distortion in an ambient environment using a metric based on eigenvalues of the EM field matrix.
7. The system of claim 6 , wherein the metric is based on a combination of differences of the eigenvalues from a reference value.
8. The system of claim 7 , wherein the eigenvalues are normalized based on a determinant of the EM field matrix.
9. The system of claim 1 , wherein to calculate the distortion compensating matrices, the hardware processor is further programmed to compare the EM sensor data to ground truth data for a relative pose of the EM sensor to an EM emitter obtained by a non-EM tracking system.
10. The system of claim 9 , wherein the non-EM tracking system comprises an imaging system or an ultrasound system.
11. A method for correcting for electromagnetic (EM) distortion in an EM tracking system, the method comprising:
based on a hand-held user input device including an EM emitter or EM sensor therein not being in a field of view of an outward-facing imaging system of a head mounted wearable component, determining that EM distortion correction is not needed;
based on the hand-held user input device including the EM emitter or EM sensor therein being in the field of view of the outward-facing imaging system of the head mounted wearable component, determining that EM distortion correction is needed and, under control of a hardware processor:
receiving EM sensor data from an EM sensor, the EM sensor data comprising an EM field matrix;
detecting the presence of EM distortion caused by the EM emitter or EM sensor of the hand-held user input device being present in the field of view of the outward-facing imaging system of the head mounted wearable component;
when either the detected EM distortion is above a first threshold or a change in the detected EM distortion resulting from real-time motion of the EM tracking system is above a second threshold:
calculating, based at least partly on the EM sensor data, distortion compensating matrices such that a determinant of each of the distortion compensating matrices is close to unity;
calculating, based at least partly on the distortion compensating matrices, an undistorted EM field matrix such that eigenvalues of the undistorted EM field matrix are proportional to [1, ½, ½]; and
calculating, based at least partly on the undistorted EM field matrix, a pose of the EM sensor; and
when neither the detected EM distortion is above the first threshold nor a change in the detected EM distortion resulting from real-time motion of the EM tracking system is above the second threshold:
not calculating, based at least partly on the EM sensor data, distortion compensating matrices.
12. The method of claim 11 , wherein the pose comprises a six degree-of-freedom (6DOF) pose.
13. The method of claim 11 , further comprising detecting presence of an EM distortion.
14. The method of claim 13 , wherein detecting the presence of the EM distortion further comprises calculating a deviation of eigenvalues of the EM field matrix from the eigenvalues of the undistorted EM field matrix.
15. The method of claim 14 , wherein detecting the presence of the EM distortion comprises determining that a deviation of the eigenvalues exceeds a threshold.
16. The method of claim 11 , wherein calculating the distortion compensating matrices further comprises comparing to ground truth data for a relative pose of the EM sensor to an EM emitter obtained by a non-EM tracking system.
17. The method of claim 16 , wherein the non-EM tracking system comprises an imaging system or an ultrasound system.
18. The method of claim 11 , further comprising calculating a degree of EM distortion in an ambient environment using a metric based on eigenvalues of the EM field matrix.
19. The method of claim 18 , wherein the metric is based on a combination of differences of the eigenvalues from a reference value.
20. The method of claim 19 , wherein the eigenvalues are normalized based on a determinant of the EM field matrix.