Techniques for accurate pose estimation
The described technology regards an augmented reality system and method for estimating a position of a location of interest relative to the position and orientation of a display based upon a retroactive adjustment of a previously rendered position and orientation of the display, by means of an adjust-update-predict (AUP) cycle, and calculating the location of interest relative to the position and orientation of the display. Systems of the described technology include including a plurality of sensors, a processing module or other computation means, and a database. Methods of the described technology use data from the sensor package useful to accurately render graphical user interface information on a display.
1. A method of enhancing the accuracy of pose estimation of a display, comprising the steps of:
receiving, from a camera, video data;
receiving geodetic coordinates of a point of interest;
receiving, from a plurality of sensors, position/orientation data;
retroactively adjusting, by a processor and based on the position/orientation data received from the sensors, a historical position prediction and a historical orientation prediction in an Extended Kalman Filter-based estimation of a position and orientation of the display,
wherein the historical position prediction and the historical orientation prediction were previously used to overlay indicia of a historical location of the point of interest on historical video data on the display;
updating, by the processor and based on the position/orientation data received from the sensors and the adjusted historical position prediction and adjusted historical orientation prediction for the display, the Extended Kalman Filter-based estimation of the position and orientation of the display;
predicting, by the processor and based on the updated Extended Kalman Filter-based estimation of the position and orientation of the display, a future position and a future orientation for the display using the Extended Kalman Filter;
displaying, on the display, the video data;
overlaying, on the video data on the display, indicia of a location of the point of interest based on the geodetic coordinates and the predicted future position and the predicted future orientation for the display.
2. The method of claim 1 , wherein the plurality of sensors includes a three-axis accelerometer and a three-axis gyroscope.
3. The method of claim 1 , wherein the plurality of sensors includes the camera.
4. The method of claim 1 , wherein the point of interest corresponds to a object in motion.
5. The method of claim 1 , wherein the position/orientation data consists of orientation data.
6. The method of claim 1 , wherein the display and the plurality of sensors are integrated into a head-mounted display.
7. The method of claim 1 , wherein the display and the plurality of sensors are integrated into a smartphone.
8. A head-mounted display with enhanced pose estimation, comprising:
a plurality of sensors;
a see-through display;
a processor; and
one or more non-transitory computer-readable media storing computer-executable instructions which, when executed by the processor perform a method of estimating a pose of the display, the method comprising the steps of:
receiving geodetic coordinates of a point of interest
receiving, from the plurality of sensors, position/orientation data;
retroactively adjusting, based on the position/orientation data received from the sensors, a historical position prediction and a historical orientation prediction in an Extended Kalman Filter-based estimation of a position and orientation of the display,
wherein the historical position prediction and the historical orientation prediction were previously used to display indicia of a historical location of the point of interest on the display;
updating, based on the position/orientation data received from the sensors and the adjusted historical position prediction and historical orientation prediction for the display, the Extended Kalman Filter-based estimation of the position and orientation of the display;
predicting, based on the updated Extended Kalman Filter-based estimation of the position and orientation of the display, a future position and a future orientation for the display;
displaying, on the see-through display, indicia of a location of the point of interest based on the geodetic coordinates and the predicted future position and the predicted future orientation for the display.
9. The head-mounted display of claim 8 , wherein the plurality of sensors includes a three-axis accelerometer and a three-axis gyroscope.
10. The head-mounted display of claim 9 , further comprising a wireless receiver and wherein the location of the point of interest is received via the wireless receiver.
11. The head-mounted display of claim 9 , wherein the location of the point of interest is received from a list of points of interest stored on the computer-readable medium.
12. The head-mounted display of claim 9 , wherein the indicia of the location of the point of interest is overlaid on the point of interest in the view of a wearer of the head-mounted display.
13. The head-mounted display of claim 9 , wherein the indicia of the location of the point of interest is displayed as a geo-registered icon on a situational awareness ring in the view of a wearer of the head-mounted display.
14. The head-mounted display of claim 8 , wherein the plurality of sensors includes a camera.
15. One or more non-transitory computer-readable media storing computer-executable instructions which, when executed by a processor, perform a method of estimating the pose of a display, the method comprising the steps of:
receiving geodetic coordinates of a point of interest;
receiving, from a plurality of sensors, position/orientation data;
retroactively adjusting, by the processor and based on the position/orientation data received from the sensors, a historical position prediction and a historical orientation prediction for the display in a state-based estimation of a position and orientation of the display;
wherein the historical position prediction and the historical orientation prediction were previously used to display indicia of a historical location of the point of interest on the display;
updating, by the processor and based on the position/orientation data received from the sensors and the updated historical position prediction and historical orientation prediction of the display, the state-based estimation of the position and orientation of the display;
predicting, by the processor and based on the updated state-based estimation of the position and orientation of the display, a future position and a future orientation for the display using the state-based estimation of the position and orientation of the display;
displaying, on the display, indicia of a location of the point of interest based on the geodetic coordinates and the predicted future position and the predicted future orientation for the display.
16. The non-transitory media of claim 15 , wherein the plurality of sensors includes a three-axis accelerometer and a three-axis rate gyroscope.
17. The non-transitory media of claim 16 , wherein the state-based estimation of the position and orientation of the display is an Extended Kalman Filter-based estimation of the position and orientation of the display.
18. The non-transitory media of claim 15 , wherein the display is a video display, wherein the plurality of sensors includes a camera, and wherein the method further comprises the steps of:
receiving, from the camera, video data; and
displaying, on the display, the video data.
19. The non-transitory media of claim 15 , wherein the indicia of the location of the point of interest is overlaid as a geo-registered icon on the point of interest on the display.
20. The non-transitory media of claim 15 , wherein the indicia of the location of the point of interest is displayed on a situational awareness ring on the display.