Techniques for accurate pose estimation
The described technology regards an augmented reality system and method for estimating a position of a location of interest relative to the position and orientation of a display, including a retroactive update to a previously rendered estimate of the position of the orientation of the display stored in a rewind buffer. Systems of the described technology include including a plurality of sensors, a processing module or other computation means, and a database. Methods of the described technology use data from the sensor package useful to accurately render graphical user interface information on a display.
1. A method of enhancing the accuracy of pose estimation of a display, comprising the steps of:
receiving position/orientation update data for a first time epoch;
storing the position/orientation data for the first time epoch in a rewind buffer;
updating, by a processor and based on the position/orientation update data, an Extended Kalman Filter-based estimation of a position and orientation of the display;
predicting, by the processor and based on the updated Extended Kalman Filter-based estimation of the position and orientation of the display, a future position and a future orientation for the display for the first time epoch using the Extended Kalman Filter;
after the expiration of the first time epoch, receiving absolute position/orientation data for a second time epoch, the second time epoch preceding the first time epoch;
retrieving, from the rewind buffer, the position/orientation update data for the first time epoch;
reupdating, by the processor and based on the absolute position/orientation data and the position/orientation update data, the Extended Kalman Filter-based estimation of the position and orientation of the display; and
repredicting, by the processor and based on the absolute position/orientation data, the position/orientation update data, and the updated Extended Kalman Filter-based estimation of the position and orientation of the display, the future position and the future orientation for the display for the first time epoch using the Extended Kalman Filter.
2. The method of claim 1 , further comprising the steps of:
receiving, from a camera, video data for the first time epoch;
displaying, on the display, the video data for the first time epoch;
receiving geodetic coordinates of a point of interest for the first time epoch;
overlaying, on the video data on the display, indicia of a location of the point of interest based on the geodetic coordinates and the future position and the future orientation for the display for the first epoch.
3. The method of claim 2 , wherein the point of interest corresponds to a obect in motion.
4. The method of claim 1 , wherein the absolute position/orientation data is received from an absolute position/orientation sensor selected from the set consisting of a Global Positioning System receiver and a three-axis magnetometer.
5. The method of claim 4 , wherein the position/orientation update data is received from a relative position/orientation sensor selected from the set consisting of a three-axis accelerometer, a three-axis angular rate gyro, and a camera.
6. The method of claim 5 , wherein the display, the absolute position/orientation sensor, and the relative position/orientation sensor are integrated into a head-mounted display.
7. The method of claim 5 , wherein the display, the absolute position/orientation sensor, and the relative position/orientation sensor are integrated into a smartphone.
8. A system for augmented reality with enhanced pose estimation comprising:
a relative position/orientation sensor;
an absolute position/orientation sensor;
a display;
a geospatial data store;
a processor; and
one or more non-transitory computer-readable media storing computer-executable instructions which, when executed by the processor perform a method of estimating a pose of the display, the method comprising the steps of:
receiving, from the relative position/orientation sensor, position/orientation update data for a first time epoch;
storing the position/orientation update data for the first time epoch in a rewind buffer;
updating, by the processor and based on the position/orientation update data, a state-based estimation of a position and orientation of the display;
predicting, by the processor and based on the state-based estimation of the position and orientation of the display, a future position and a future orientation for the display for the first time epoch;
receiving, from the geospatial data store, geodetic coordinates of a point of interest for the first time epoch;
displaying, on the display, indicia of a location of the point of interest based on the geodetic coordinates and the future position and the future orientation for the display for the first time epoch;
receiving, from the absolute position/orientation sensor, absolute position/orientation data for a second time epoch, the second time epoch preceding the first time epoch;
retrieving, from the rewind buffer, the position/orientation update data for the first time epoch;
reupdating, by the processor and based on the absolute position/orientation data and the position/orientation update data, the state-based estimation of the position and orientation of the display; and
repredicting, by the processor and based on the absolute position/orientation data, the position/orientation update data and the state-based estimation of the position and orientation of the display, the future position and the future orientation for the display for the first time epoch.
9. The system of claim 8 , further comprising a camera,
wherein the method further comprises the steps of:
receiving, from the camera video data; and
displaying, on the display, the video data; and
wherein the indicia of the location of the point of interest is overlaid on the point of interest in the displayed video data.
10. The system of claim 8 , wherein the system is integrated into a head-mounted display.
11. The system of claim 10 , wherein the indicia of the location of the point of interest is displayed as a geo-registered icon on a situational awareness ring in the view of a wearer of the head-mounted display.
12. The system of claim 8 , wherein the system is integrated into a smartphone.
13. The system of claim 8 , wherein the absolute position/orientation sensor is selected from the set consisting of a Global Positioning System receiver and a three-axis magnetometer.
14. The system of claim 8 , wherein the relative position/orientation sensor is selected from the set consisting of a three-axis accelerometer, a three-axis angular rate gyro, and a camera.
15. One or more non-transitory computer-readable media storing computer-executable instructions which, when executed by a processor, perform a method of estimating the pose of a display, the method comprising the steps of:
receiving, from a relative position/orientation sensor, position/orientation update data for a first time epoch;
storing the position/orientation data for the first time epoch in a rewind buffer;
updating, by a processor and based on the position/orientation update data, a state-based estimation of a position and orientation of the display;
predicting, by the processor and based on the state-based estimation of the position and orientation of the display, a future position and a future orientation for the display for the first time epoch;
receiving geodetic coordinates of a point of interest for the first time epoch;
displaying, on the display, indicia of a location of the point of interest based on the geodetic coordinates and the future position and the future orientation for the display for the first time epoch;
after the expiration of the first time epoch, receiving, from an absolute position/orientation sensor, absolute position/orientation data for a second time epoch, the second time epoch preceding the first time epoch;
retrieving, from the rewind buffer, the position/orientation update data for the first time epoch;
reupdating, by the processor and based on the absolute position/orientation data and the position/orientation update data, the state-based estimation of the position and orientation of the display; and
repredicting, by the processor and based on the absolute position/orientation data, the position/orientation update data and the state-based estimation of the position and orientation of the display, the future position and the future orientation for the display for the first time epoch.
16. The media of claim 15 , wherein the absolute position/orientation sensor is selected from the set consisting of a Global Positioning System receiver and a three-axis magnetometer.
17. The media of claim 16 , wherein the relative position/orientation sensor is selected from the set consisting of a three-axis accelerometer, a three-axis angular rate gyro, and a camera.
18. The media of claim 17 , wherein the display, the absolute position/orientation sensor, the relative position/orientation sensor, the processor, and the media are integrated into a head-mounted display.
19. The media of claim 15 , wherein the state-based estimation of the position and the orientation is an Extended Kalman Filter-based estimation of the position and the orientation.
20. The media of claim 15 , wherein the geodetic coordinates of the point of interest are received via a networked radio.