Sensor fusion for electromagnetic tracking
Head-mounted augmented reality (AR) devices can track pose of a wearer's head or pose of a hand-held user input device to enable wearer interaction in a three-dimensional AR environment. A pose sensor (e.g., an inertial measurement unit) in the user input device can provide data on pose (e.g., position or orientation) of the user input device. An electromagnetic (EM) tracking system can also provide pose data. For example, the handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field. The AR device can combine the output of the pose sensor and the EM tracking system to reduce drift in the estimated pose of the user input device or to transform the pose into a world coordinate system used by the AR device. The AR device can utilize a Kalman filter to combine the output of the pose sensor and the EM tracking system.
1. A wearable system comprising:
a head-mounted display;
a handheld user input device;
an electromagnetic (EM) tracking system comprising:
one or more EM emitter configured to generate an EM field; and
one or more EM sensors configured to sense the EM field,
wherein the EM tracking system is configured to output an EM field matrix associated with an estimated totem pose of the one or more EM sensors relative to the one or more EM emitter; and
one or more computer processors configured to:
access the EM field matrix from the EM tracking system;
determine, based at least on the EM field matrix, a first estimated initial totem pose and a second estimated initial totem pose having an opposite orientation from the first estimated initial totem pose;
in response to periodically receiving current IMU data and current EM field matrix:
generating a first error state based on the current EM field matrix and the first estimated initial totem pose;
generating a second error state based on the current EM field matrix and the second estimated initial totem pose;
determining one of the first and second estimated initial totem pose that exceeds a threshold; and
in response to determining that the first error state exceeds the threshold, establish the second estimated initial totem pose as an initial totem pose of the totem and disregard the first estimated initial totem pose; or
in response to determining that the second error state exceeds the threshold, establish the first estimated initial totem pose as the initial totem pose of the totem and disregard the second estimated initial totem pose.
2. The wearable system of claim 1 , wherein the one or more computer processors are further configured to:
determine a distance between the head-mounted display and the handheld user input device; and
identifying a third error state in response to determining that the distance is larger than a threshold distance.
3. The wearable system of claim 1 , wherein the IMU data comprises one or more of accelerometer data or gyroscope data.
4. The wearable system of claim 1 , wherein the first and second estimated initial totem poses are calculated by a Kalman filter.
5. The wearable system of claim 1 , wherein the first estimated initial totem pose is in a first hemisphere and the second estimated initial totem pose is in a second hemisphere opposite the first hemisphere.
6. The wearable system of any claim 1 , wherein the one or more computer processors is further configured to:
present, via the head-mounted display, virtual content to a user of the wearable system; or
enable interaction with an environment of the wearable system based at least partly on movement or rotation of the handheld user input device.
7. A computerized method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising:
accessing an EM field matrix from an EM tracking system;
determining, based at least on the EM field matrix, a first estimated initial totem pose and a second estimated initial totem pose having an opposite orientation from the first estimated initial totem pose;
accessing IMU data from a handheld user input device, the IMU data comprising one or more accelerometer data or gyroscope data;
periodically receiving the IMU data;
generating a first error state based on the EM field matrix and the first estimated initial totem pose;
generating a second error state based on the EM field matrix and the second estimated initial totem pose;
determining one of the first and second error states that exceeds a threshold; and
in response to determining that the first error state exceeds the threshold, establish the second estimated initial totem pose as the initial totem pose of the totem and disregard the first estimated initial totem pose; or
in response to determining that the second error state exceeds the threshold, establish the first estimated initial totem pose as the initial totem pose of the totem and disregard the second estimated initial totem pose.
8. The computerized method of claim 7 , wherein the first and second estimated initial totem poses are calculated by a Kalman filter.
9. The computerized method of claim 7 , wherein the first estimated initial totem pose is in a first hemisphere and the second estimated initial totem pose is in a second hemisphere opposite the first hemisphere.