Gravity estimation and bundle adjustment for visual-inertial odometry
Examples of the disclosure describe systems and methods for presenting virtual content on a wearable head device. In some embodiments, a state of a wearable head device is determined by minimizing a total error based on a reduced weight associated with a reprojection error. A view reflecting the determined state of the wearable head device is presented via a display of the wearable head device. In some embodiments, a wearable head device calculates a first preintegration term and second preintegration term based on the image data received via a sensor of the wearable head device and the inertial data received via a first IMU and a second IMU of the wearable head device. The wearable head device estimates a position of the device based on the first and second preintegration terms, and the wearable head device presents the virtual content based on the position of the device.
1. A method comprising:
receiving, via a sensor of a wearable head device, first sensor data indicative of a first feature in a first position;
receiving, via the sensor, second sensor data indicative of the first feature in a second position;
receiving, via an inertial measurement unit (IMU) on the wearable head device, inertial measurements;
determining a velocity based on the inertial measurements;
estimating a third position of the first feature based on the first position and the velocity;
determining a reprojection error based on the third position and the second position;
reducing a weight associated with the reprojection error;
determining a state of the wearable head device, wherein determining the state comprises minimizing a total error, and wherein the total error is based on the reduced weight associated with the reprojection error; and
presenting, via a display of the wearable head device, a view reflecting the determined state of the wearable head device.
2. The method of claim 1 , wherein the weight associated with the reprojection error is reduced based on a quantity of identified features.
3. The method of claim 1 , wherein reducing the weight associated with the reprojection error comprises determining if the reprojection error comprises a statistical outlier.
4. The method of claim 1 , wherein the total error is further based on an error from the inertial measurement unit.
5. The method of claim 1 , further comprising:
determining if the wearable head device is static for at least a threshold amount of time;
in accordance with a determination that the wearable head device is not static for at least the threshold amount of time, updating an information matrix; and
in accordance with a determination that the wearable head device is static for at least the threshold amount of time, forgoing updating the information matrix.
6. The method of claim 1 , further comprising pre-integrating the inertial measurements.
7. The method of claim 6 , further comprising:
receiving, via a second IMU on the wearable head device, second inertial measurements; and
pre-integrating the second inertial measurements.
8. The method of claim 6 , wherein:
the first sensor data is received at a first time, and
the inertial measurements being pre-integrated are associated with a movement of the wearable head device between the first time and a second time, the method further comprising:
determining whether a time interval between the first time and the second time is greater than a maximum time interval;
in accordance with a determination that the time interval is not greater than the maximum time interval, using the pre-integrated inertial measurements; and
in accordance with a determination that the time interval is greater than the maximum time interval, forgoing using the pre-integrated inertial measurements.
9. The method of claim 1 , further comprising applying a correction to the state based on an updated state of the wearable head device.
10. A system comprising:
a sensor of a wearable head device;
an inertial measurement unit of the wearable head device;
a display of the wearable head device;
one or more processors configured to execute a method comprising:
receiving, via the sensor of the wearable head device, first sensor data indicative of a first feature in a first position;
receiving, via the sensor, second sensor data indicative of the first feature in a second position;
receiving, via the inertial measurement unit on the wearable head device, inertial measurements;
determining a velocity based on the inertial measurements;
estimating a third position of the first feature based on the first position and the velocity;
determining a reprojection error based on the third position and the second position;
reducing a weight associated with the reprojection error;
determining a state of the wearable head device, wherein determining the state comprises minimizing a total error, and wherein the total error is based on the reduced weight associated with the reprojection error; and
presenting, via the display of the wearable head device, a view reflecting the determined state of the wearable head device.
11. The system of claim 10 , wherein the weight associated with the reprojection error is reduced based on a quantity of identified features.
12. The system of claim 10 , wherein reducing the weight associated with the reprojection error comprises determining if the reprojection error comprises a statistical outlier.
13. The system of claim 10 , wherein the total error is further based on an error from the inertial measurement unit.
14. The system of claim 10 , wherein the method further comprises:
determining if the wearable head device is static for at least a threshold amount of time;
in accordance with a determination that the wearable head device is not static for at least the threshold amount of time, updating an information matrix; and
in accordance with a determination that the wearable head device is static for at least the threshold amount of time, forgoing updating the information matrix.
15. The system of claim 10 , wherein the method further comprises pre-integrating the inertial measurements.
16. The system of claim 15 , wherein the method further comprises:
receiving, via a second IMU on the wearable head device, second inertial measurements; and
pre-integrating the second inertial measurements.
17. The system of claim 15 , wherein:
the first sensor data is received at a first time, and
the inertial measurements being pre-integrated are associated with a movement of the wearable head device between the first time and a second time, and
the method further comprises:
determining whether a time interval between the first time and the second time is greater than a maximum time interval;
in accordance with a determination that the time interval is not greater than the maximum time interval, using the pre-integrated inertial measurements; and
in accordance with a determination that the time interval is greater than the maximum time interval, forgoing using the pre-integrated inertial measurements.
18. The system of claim 10 , wherein the method further comprises applying a correction to the state based on an updated state of the wearable head device.
19. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to execute a method comprising:
receiving, via a sensor of a wearable head device, first sensor data indicative of a first feature in a first position;
receiving, via the sensor, second sensor data indicative of the first feature in a second position;
receiving, via an inertial measurement unit on the wearable head device, inertial measurements;
determining a velocity based on the inertial measurements;
estimating a third position of the first feature based on the first position and the velocity;
determining a reprojection error based on the third position and the second position;
reducing a weight associated with the reprojection error;
determining a state of the wearable head device, wherein determining the state comprises minimizing a total error, and wherein the total error is based on the reduced weight associated with the reprojection error; and
presenting, via a display of the wearable head device, a view reflecting the determined state of the wearable head device.
20. The non-transitory computer-readable medium of claim 19 , wherein the method further comprises pre-integrating the inertial measurements.