IP Library › Granted Patent US 12,379,215
Granted Patent B2
US 12,379,215 · App. 18/363,593 · Granted Aug 5, 2025

Efficient vision-aided inertial navigation using a rolling-shutter camera with inaccurate timestamps

Inventors: Stergios I. Roumeliotis (Los Altos Hills, CA); Chao Guo (Los Altos, CA)
Assignee: Regents of the University of Minnesota
G01C21/1656G06T7/277G06T2207/30241G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,379,215
App. No.
18/363,593
Granted
Aug 5, 2025
Kind
B2
Abstract

Vision-aided inertial navigation techniques are described. In one example, a vision-aided inertial navigation system (VINS) comprises an image source to produce image data at a first set of time instances along a trajectory within a three-dimensional (3D) environment, wherein the image data captures features within the 3D environment at each of the first time instances. An inertial measurement unit (IMU) to produce IMU data for the VINS along the trajectory at a second set of time instances that is misaligned with the first set of time instances, wherein the IMU data indicates a motion of the VINS along the trajectory. A processing unit comprising an estimator that processes the IMU data and the image data to compute state estimates for 3D poses of the IMU at each of the first set of time instances and 3D poses of the image source at each of the second set of time instances along the trajectory. The estimator computes each of the poses for the image source as a linear interpolation from a subset of the poses for the IMU along the trajectory.

Claims (88)

1. A vision-aided inertial navigation system (VINS) comprising:

an image source configured to produce image data at a first set of time instances along a trajectory within a three-dimensional (3D) environment, wherein:

the image data captures feature observations within the 3D environment at each of the first set of time instances,

the image source comprises at least one sensor capable of capturing a plurality of rows of image data, and

a sensor of the at least one sensor is configured to capture the plurality of rows of image data row-by-row so that each row is captured at a different time instance than any of the first set of time instances;

an inertial measurement unit (IMU) configured to produce IMU data for the VINS along the trajectory at a second set of time instances that is misaligned in time with the first set of time instances, wherein the IMU data indicates a motion of the VINS along the trajectory; and

a processor is configured to:

compute poses for the image source as an extrapolation from poses for the IMU that are closest in time along the trajectory,

compute each of the poses for the image source by storing and updating a state vector having a sliding window of poses for the image source, wherein each of the poses for the image source correspond to a different time instance of the first set of time instances at which the image data was captured by the image source, and

in response to the image source producing the image data, insert a most recent pose computed for the IMU into the state vector as an image source pose.

2. The VINS of claim 1 , wherein the processor is configured to compute each of the poses for the image source as at least one of:

a linear extrapolation, or

a higher order extrapolation from the poses for the IMU that are closest in time along the trajectory.

3. The VINS of claim 1 , wherein:

the poses of the IMU and the poses of the image source are each computed as positions and 3D orientations at each of the second set of time instances along the trajectory; and

the processor is configured to compute particular state estimates for a position, orientation and velocity of the VINS.

4. The VINS of claim 1 , wherein:

the processor is configured to store a sliding window of poses computed for the IMU along the trajectory; and

when computing each of the poses for the image source, the processor is configured to:

select the poses for the IMU that are adjacent in the sliding window and that have time instances closest to the time instance for the pose being computed for the image source, and

compute the poses for the image source as an extrapolation of the selected poses for the IMU.

5. The VINS of claim 1 , wherein:

the processor is configured to compute estimated positions for features observed within the 3D environment at each of the second set of time instances; and

for each of the second set of time instances, the processor is configured to compute the estimated positions for the features by applying an extrapolation ratio to the poses for the IMU that were used to compute the pose for the image source at that time instance.

6. The VINS of claim 5 , wherein, for each of the poses for the image source, the extrapolation ratio represents a ratio of:

a first distance, along the trajectory, from the pose of the image source to a first pose for the IMU used to compute the pose for the image source; and

a second distance, along the trajectory, from the pose of the image source to a second pose for the IMU used to compute the pose for the image source.

7. The VINS of claim 1 , wherein the inserted pose excludes state estimates for linear and rotational velocities and comprises:

six dimensions for the image source; and

a scalar representing a time offset, between the IMU and the image source, for the time at which the image data was received.

8. The VINS of claim 1 , wherein the image source is a rolling-shutter camera.

9. The VINS of claim 1 , wherein:

the processor is further configured to represent an estimate of the state vector as x=[x 1 x I k+(n−1) . . . x I xk ], wherein:

x I denotes a current pose for the image source, and

x I i , for i=k+(n−1), k+(n−2), . . . , k denotes the poses in the sliding window, corresponding to time instants of a previous n camera measurements; and

the current pose for the image source comprises:

a quaternion representation of an orientation of a global frame of reference in a frame of reference for the IMU,

a velocity of the frame of reference for the IMU in the global frame of reference,

a position of the frame of reference for the IMU in the global frame of reference,

a gyroscope bias for the IMU, and

an accelerometer bias for the IMU.

10. The VINS of claim 1 , wherein the processor is further configured to:

build a map of the 3D environment based on the state vector; and

navigate a mobile device based on the map, where the VINS is integrated into the mobile device.

11. A method for vision-aided inertial navigation comprising:

capturing, using an image source, image data at a first set of time instances along a trajectory within a three-dimensional (3D) environment, wherein:

the image source is a rolling-shutter camera integrated into a vision-aided inertial navigation system (VINS), and

the VINS is integrated into a mobile device;

receiving the image data from the rolling-shutter camera, at a processor integrated into the, wherein:

the image data captures feature observations within the 3D environment at each of the first set of time instances,

the image source comprises at least one sensor capable of capturing a plurality of rows of image data, and

a sensor of the at least one sensor is configured to capture the plurality of rows of image data row-by-row so that each row is captured at a different time instance than any of the first set of time instances;

receiving, at the processor, from an inertial measurement unit (IMU), IMU data at a second set of time instances that is misaligned in time with the first set of time instances, wherein the IMU data indicates motion of the VINS along the trajectory;

computing, using the processor, from the IMU data and the image data, state estimates for poses of the IMU at each of the first set of time instances and poses of the image source at each of the second set of time instances along the trajectory, wherein computing the state estimates comprises:

computing poses for the image source as an extrapolation from poses for the IMU that are closest in time along the trajectory,

computing each of the poses for the image source by storing and updating a state vector having a sliding window of poses for the image source, wherein each of the poses for the image source correspond to a different time instance of the first set of time instances at which the image data was captured by the image source, and

in response to the image source producing the image data, inserting a most recent pose computed for the IMU into the state vector as an image source pose; and

navigating, via the processor, the mobile device using the state vector.

12. The method of claim 11 , wherein each of the poses for the image source are computed as at least one of:

a linear extrapolation, or

a higher order extrapolation from the poses for the IMU that are closest in time along the trajectory.

13. The method of claim 11 , wherein:

the poses of the IMU and the poses of the image source are each computed as positions and 3D orientations at each of the second set of time instances along the trajectory; and

particular state estimates are computed for a position, orientation and velocity of the VINS.

14. The method of claim 11 , further comprising:

storing, using the processor, a sliding window of the poses computed for the IMU along the trajectory; and

when computing each of the poses for the image source:

selecting, with the processor, the poses for the IMU that are adjacent in the sliding window and that have time instances closest to the time instance for the pose being computed for the image source, and

computing, using the processor, the poses for the image source as an extrapolation of the selected poses for the IMU.

15. The method of claim 11 , further comprising:

computing, using the processor, estimated positions for features observed within the 3D environment at each of the second set of time instances; and

for each of the second set of time instances, computing, using the processor, the estimated positions for the features by applying an extrapolation ratio to the poses for the IMU that were used to compute the pose for the image source at that time instance.

16. The method of claim 15 , wherein, for each of the poses for the image source, the extrapolation ratio represents a ratio of:

a first distance, along the trajectory, from the pose of the image source to a first pose for the IMU used to compute the pose for the image source; and

a second distance, along the trajectory, from the pose of the image source to a second pose for the IMU used to compute the pose for the image source.

17. The method of claim 11 , wherein the inserted pose excludes state estimates for linear and rotational velocities and comprises:

six dimensions for the image source; and

a scalar representing a time offset, between the IMU and the image source, for the time at which the image data was received.

18. The method of claim 11 , further comprising representing an estimate of the state vector as x=[x I x I k+(n−1) . . . x I k ], wherein:

x I denotes a current pose for the image source, and

x I i , for i=k+(n−1), k+(n−2), . . . , k denotes the poses in the sliding window, corresponding to time instants of a previous n camera measurements; and

the current pose for the image source comprises:

a quaternion representation of an orientation of a global frame of reference in a frame of reference for the IMU,

a velocity of the frame of reference for the IMU in the global frame of reference,

a position of the frame of reference for the IMU in the global frame of reference,

a gyroscope bias for the IMU, and

an accelerometer bias for the IMU.

19. The method of claim 11 , further comprising building a map of the 3D environment using the processor, wherein the map is based on the state vector.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 7, 2024
From: ROUMELIOTIS, STERGIOS I.; GUO, CHAO
To: REGENTS OF THE UNIVERSITY OF MINNESOTA
Reel/Frame 066411/0552 →
Continuity (4)
Continuation 16025574 · Jul 2, 2018
Continuation 14733468 · Jun 8, 2015
Provisional Application 62014532 · Jun 19, 2014
Related Publication 20230408262A1 · Dec 21, 2023
References Cited (283)
US 5847755A · Wixson et al. · 1998 [cited by applicant]
US 6104861A · Tsukagoshi · 2000 [cited by applicant]
US 6496778B1 · Lin · 2002 [cited by applicant]
US 7015831B2 · Karlsson et al. · 2006 [cited by applicant]
US 7162338B2 · Goncalves et al. · 2007 [cited by applicant]
US 7747151B2 · Kochi et al. · 2010 [cited by applicant]
US 7991576B2 · Roumeliotis · 2011 [cited by applicant]
US 8467612B2 · Susca et al. · 2013 [cited by applicant]
US 8510039B1 · Troy et al. · 2013 [cited by applicant]
US 8577539B1 · Morrison et al. · 2013 [cited by applicant]
US 8761439B1 · Kumar et al. · 2014 [cited by applicant]
US 8965682B2 · Tangirala et al. · 2015 [cited by applicant]
US 8996311B1 · Morin et al. · 2015 [cited by applicant]
US 9026263B2 · Hoshizaki · 2015 [cited by applicant]
US 9031809B1 · Kumar et al. · 2015 [cited by applicant]
US 9227361B2 · Choi et al. · 2016 [cited by applicant]
US 9243916B2 · Roumeliotis et al. · 2016 [cited by applicant]
US 9303999B2 · Hesch et al. · 2016 [cited by applicant]
US 9607401B2 · Roumeliotis et al. · 2017 [cited by applicant]
US 9658070B2 · Roumeliotis et al. · 2017 [cited by applicant]
US 9709404B2 · Roumeliotis et al. · 2017 [cited by applicant]
US 9766074B2 · Roumeliotis et al. · 2017 [cited by applicant]
US 9996941B2 · Roumeliotis et al. · 2018 [cited by applicant]
US 10012504B2 · Roumeliotis et al. · 2018 [cited by applicant]
US 10203209B2 · Roumeliotis et al. · 2019 [cited by applicant]
US 10254118B2 · Roumeliotis et al. · 2019 [cited by applicant]
US 10339708B2 · Lynen et al. · 2019 [cited by applicant]
US 10371529B2 · Roumeliotis et al. · 2019 [cited by applicant]
US 10670404B2 · Roumeliotis et al. · 2020 [cited by applicant]
US 11719542B2 · Roumeliotis et al. · 2023 [cited by applicant]
US 20020198632A1 · Breed et al. · 2002 [cited by applicant]
US 20030149528A1 · Lin · 2003 [cited by applicant]
US 20040073360A1 · Foxlin · 2004 [cited by applicant]
US 20040167667A1 · Goncalves et al. · 2004 [cited by applicant]
US 20050013583A1 · Itoh · 2005 [cited by applicant]
US 20070038374A1 · Belenkii et al. · 2007 [cited by applicant]
US 20080167814A1 · Samarasekera et al. · 2008 [cited by applicant]
US 20080265097A1 · Stecko et al. · 2008 [cited by applicant]
US 20080279421A1 · Hamza et al. · 2008 [cited by applicant]
US 20090212995A1 · Wu et al. · 2009 [cited by applicant]
US 20090248304A1 · Roumeliotis et al. · 2009 [cited by applicant]
US 20100110187A1 · Von Flotow et al. · 2010 [cited by applicant]
US 20100211316A1 · Da Silva et al. · 2010 [cited by applicant]
US 20100220176A1 · Ziemeck et al. · 2010 [cited by applicant]
US 20110178708A1 · Zhang et al. · 2011 [cited by applicant]
US 20110206236A1 · Center, Jr. · 2011 [cited by applicant]
US 20110238307A1 · Psiaki et al. · 2011 [cited by applicant]
US 20120121161A1 · Eade et al. · 2012 [cited by applicant]
US 20120194517A1 · Izadi et al. · 2012 [cited by applicant]
US 20120203455A1 · Louis et al. · 2012 [cited by applicant]
US 20130138264A1 · Hoshizaki · 2013 [cited by applicant]
US 20130304383A1 · Bageshwar et al. · 2013 [cited by applicant]
US 20130335562A1 · Ramanandan · 2013 [cited by examiner]
US 20140316698A1 · Roumeliotis et al. · 2014 [cited by applicant]
US 20140333741A1 · Roumeliotis et al. · 2014 [cited by applicant]
US 20140372026A1 · Georgy et al. · 2014 [cited by applicant]
US 20150219767A1 · Humphreys et al. · 2015 [cited by applicant]
US 20150356357A1 · McManus et al. · 2015 [cited by applicant]
US 20150369609A1 · Roumeliotis et al. · 2015 [cited by applicant]
US 20160005164A1 · Roumeliotis et al. · 2016 [cited by applicant]
US 20160161260A1 · Mourikis · 2016 [cited by applicant]
US 20160305784A1 · Roumeliotis et al. · 2016 [cited by applicant]
US 20160327395A1 · Roumeliotis et al. · 2016 [cited by applicant]
US 20160364990A1 · Khaghani et al. · 2016 [cited by applicant]
US 20170176189A1 · D'aquila · 2017 [cited by applicant]
US 20170261324A1 · Roumeliotis et al. · 2017 [cited by applicant]
US 20170336511A1 · Nerurkar et al. · 2017 [cited by applicant]
US 20170343356A1 · Roumeliotis et al. · 2017 [cited by applicant]
US 20180023953A1 · Roumeliotis et al. · 2018 [cited by applicant]
US 20180082137A1 · Melvin et al. · 2018 [cited by applicant]
US 20180211137A1 · Hesch et al. · 2018 [cited by applicant]
US 20180259341A1 · Aboutalib et al. · 2018 [cited by applicant]
US 20180328735A1 · Roumeliotis et al. · 2018 [cited by applicant]
US 20190154449A1 · Roumeliotis et al. · 2019 [cited by applicant]
US 20190178646A1 · Roumeliotis et al. · 2019 [cited by applicant]
US 20190392630A1 · Sturm et al. · 2019 [cited by applicant]
US 20210004979A1 · Valentin et al. · 2021 [cited by applicant]
CN 110415344A · 2019 [cited by applicant]
WO 2000034803A2 · 2000 [cited by applicant]
WO 2015013418A2 · 2015 [cited by applicant]
WO 2015013534A1 · 2015 [cited by applicant]
WO 2018026544A1 · 2018 [cited by applicant]
Filing Receipt and Filed Application for U.S. Appl. No. 61/040,473, filed Mar. 28, 2008, mailed May 21, 2008. [cited by applicant]
Prosecution History from U.S. Appl. No. 14/733,468, dated Jul. 14, 2015 through Mar. 27, 2018, 47 pgs. [cited by applicant]
U.S. Appl. No. 14/768,733, filed Feb. 21, 2014, 501 pgs. [cited by applicant]
U.S. Appl. No. 15/130,736, filed Apr. 15, 2016, 74 pgs. [cited by applicant]
U.S. Appl. No. 15/470,595, filed Mar. 27, 2017, 81 pgs. [cited by applicant]
U.S. Appl. No. 15/601,261, filed May 22, 2017, 88 pgs. [cited by applicant]
U.S. Appl. No. 15/605,448, filed May 25, 2017, 82 pgs. [cited by applicant]
U.S. Appl. No. 15/706,149, filed Sep. 15, 2017, 65 pgs. [cited by applicant]
U.S. Appl. No. 16/425,422, filed May 29, 2019. [cited by applicant]
U.S. Appl. No. 14/796,574, filed Jul. 10, 2015, 89 pgs. [cited by applicant]
U.S. Appl. No. 61/767,701 by Stergios I. Roumeliotis, filed Feb. 21, 2013. [cited by applicant]
“Kalman filter”, Wikipedia, the Free Encyclopedia, accessed from https://en.wikipedia.org/w/index.php?title=Kalman_filter&oldid=615383582, Jul. 3, 2014, 27 pgs. [cited by applicant]
“Kalman filter”, Wikipedia, the Free Encyclopedia, accessed from https://en.wikipedia.org/w/index.php?title=kalman_filter&oldid=730505034 Jul. 19, 2016, 30 pgs. [cited by applicant]
“Project Tango”, retrieved from https://www.google.com/atap/projecttango on Nov. 2, 2015, 4 pgs. [cited by applicant]
Agarwal et al., “A Survey of Geodetic Approaches to Mapping and the Relationship to Graph-Based SLAM”, IEEE Robotics and Automation Magazine, Sep. 2014, vol. 31, 17 pp. [cited by applicant]
Ait-Aider et al., “Simultaneous object pose and velocity computation using a single view from a rolling shutter camera”, Proceedings of the IEEE European Conference on Computer Vision, May 7-13, 2006, pp. 56-68. [cited by applicant]
Antone, “Robust Camera Pose Recovery Using Stochastic Geometry”, Massachusetts Institute of Technology, Apr. 24, 2001, 187 pgs. [cited by applicant]
Ayache et al., “Maintaining Representations of the Environment of a Mobile Robot”, IEEE Transactions on Robotics and Automation, Dec. 1989, vol. 5, No. 6, pp. 804-819. [cited by applicant]
Bailey et al., “Simultaneous Localisation and Mapping (SLAM): Part II State of the Art”, IEEE Robotics and Automation Magazine, Sep. 2006, vol. 13, No. 3, 10 pgs. [cited by applicant]
Baker et al., “Removing rolling shutter wobble”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 13-18, 2010, pp. 2392-2399. [cited by applicant]
Bar-Shalom et al., “Estimation with Applications to Tracking and Navigation”, Chapter 7, Estimation with Applications to Tracking and Navigation, Jul. 2001, ISBN 0-471-41655-X, 20 pgs. [cited by applicant]
Bartoli et al., “Structure-from-Motion Using Lines: Representation, Triangulation, and Bundle Adjustment”, Computer Vision and Image Understanding, 2005, vol. 100, pp. 416-441. [cited by applicant]
Bayard et al., “An Estimation Algorithm for Vision-Based Exploration of Small Bodies in Space”, American Control Conference, Jun. 8-10, 2005, pp. 4589-4595. [cited by applicant]
Bierman, “Factorization Methods for Discrete Sequential Estimation”, Mathematics in Science and Engineering, Academic Press, vol. 128, 1977, 259 pgs. [cited by applicant]
Bloesch et al., “Iterated Extended Kalman Filter Based Visual-Inertial Odometry Using Direct Photometric Feedback”, International Journal of Robotics Research, Sep. 2017, vol. 36, 19 pgs. [cited by applicant]
Bloesch et al., “Robust Visual Inertial Odometry Using a Direct EKF-Based Approach”, Proceeding of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep. 2015, 7 pgs. [cited by applicant]
Bouguet, “Camera Calibration Toolbox for Matlab”, Retrieved from: http://www.vision.caltech.edu/bouguetj/calib_doc/, 2004, Last Updated: Oct. 14, 2015, 5 pgs. [cited by applicant]
Boyd et al., “Convex Optimization”, Cambridge University Press, 2004, 730 pp. (Applicant points out that, in accordance with MPEP 609.04(a), the 2004 year “continued in misc.”, publication is sufficiently earlier than t… [cited by applicant]
Breckenridge, “Interoffice Memorandum to T. K. Brown, Quaternions—Proposed Standard Conventions”, IOM 343-79-1199, Oct. 31, 1979, 12 pgs. [cited by applicant]
Burri et al., “The EuRoC Micro Aerial Vehicle Datasets”, International Journal of Robotics Research, 10, Sep. 2016, vol. 35, No. 10, 9 pgs. [cited by applicant]
Canny, “A Computational Approach to Edge Detection”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Nov. 1986 Vol. 8, No. 6, pp. 679-698. [cited by applicant]
Chauhan et al., “Femoral Artery Pressures Are More Reliable Than Radial Artery Pressures on Initiation of Cardiopulmonary Bypass”, Journal of Cardiothoracic and Vascular Anesthesia, Jun. 2000, vol. 14, No. 3, 3 pgs. [cited by applicant]
Chen, “Pose Determination from Line-to-Plane Correspondences: Existence Condition and Closed-Form Solutions”, Proceedings on the 3rd International Conference on Computer Vision, Dec. 4-7, 1990, pp. 374-378. [cited by applicant]
Chen et al., “Local Observability Matrix and its Application to Observability Analyses”, Proceedings of the 16th Annual Conference IEEE Industrial Electronics Society, Nov. 1990, 4 pgs. [cited by applicant]
Chiu et al., “Robust vision-aided navigation using sliding-window factor graphs”, 2013 IEEE International Conference on Robotics and Automation, May 6-10, 2013, pp. 46-53. [cited by applicant]
Chiuso et al., “Structure From Motion Causally Integrated Over Time”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Apr. 2002, vol. 24, No. 4, pp. 523-535. [cited by applicant]
Comport et al., “Accurate Quadrifocal Tracking for Robust 3D Visual Odometry”, IEEE International Conference on Robotics and Automation, 2007, pp. 40-45. [cited by applicant]
Cortes et al., “ADVIO: An Authentic Dataset for Visual-Inertial Odometry”, Obtained from arXiv:1807.09828, Jul. 25, 2018, 25 pgs. [cited by applicant]
Cumani et al., “Integrating Monocular Vision and Odometry for SLAM”, WSEAS Transactions on Computers, 2003, 6 pgs. [cited by applicant]
Davison et al., “Simultaneous Localisation and Map-Building Using Active Vision”, Jun. 2001, 18 pgs. [cited by applicant]
Deans, “Maximally Informative Statistics for Localization and Mapping”, Proceedings of the 2002 IEEE International Conference on Robotics & Automation, May 2002, pp. 1824-1829. [cited by applicant]
Dellaert et al., “Square Root SAM: Simultaneous Localization and Mapping via Square Root Information Smoothing”, International Journal of Robotics and Research, Dec. 2006, vol. 25, No. 12, pp. 1181-1203. [cited by applicant]
Diel, “Stochastic Constraints for Vision-Aided Inertial Navigation”, Massachusetts Institute of Technology, Department of Mechanical Engineering, Master Thesis, Jan. 2005, 110 pgs. [cited by applicant]
Diel et al., “Epipolar Constraints for Vision-Aided Inertial Navigation”, Seventh IEEE Workshops on Applications of Computer Vision (WACV/Motion'05), vol. 1, 2005, pp. 221-228. [cited by applicant]
Dong-Si et al., “Motion Tracking with Fixed-lag Smoothing: Algorithm and Consistency Analysis”, Proceedings of the IEEE International Conference on Robotics and Automation, May 9-13, 2011, 8 pgs. [cited by applicant]
Durrant-Whyte et al., “Simultaneous Localisation and Mapping (SLAM): Part I the Essential Algorithms”, IEEE Robotics & Automation Magazine, Jun. 2006, vol. 13, No. 2, pp. 99-110. [cited by applicant]
Dutoit et al., “Consistent Map-based 3D Localization on Mobile Devices”, Proceedings of the IEEE International Conference on robotics and Automation, May 2017, 8 pgs. [cited by applicant]
Eade et al., “Monocular SLAM as a Graph of Coalesced Observations”, IEEE 11th International Conference on Computer Vision, 2007, pp. 1-8. [cited by applicant]
Eade et al., “Scalable Monocular SLAM”, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '06), Jun. 17-22, 2006, vol. 1, 8 pgs. [cited by applicant]
Engel et al., “Direct Sparse Odometry”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Mar. 2018, vol. 40, No. 3, 15 pgs. [cited by applicant]
Engel et al., “LSD-SLAM: Large-Scale Direct Monocular SLAM”, Proceedings of the European Conference on Computer Vision, Sep. 2014, 16 pgs. [cited by applicant]
Engel et al., “Semi-Dense Visual Odometry for a Monocular Camera”, Proceedings of the IEEE International Conference on Computer Vision, Dec. 2013, 8 pgs. [cited by applicant]
Erdogan et al., “Planar Segmentation of RGBD Images using Fast Linear Fitting and Markov Chain Monte Carlo”, Proceedings of the IEEE International Conference on Computer and Robot Vision, May 27-30, 2012, pp. 32-39. [cited by applicant]
Eustice et al., “Exactly Sparse Delayed-state Filters for View-based SLAM”, IEEE Transactions on Robotics, Dec. 2006, vol. 22, No. 6, pp. 1100-1114. [cited by applicant]
Eustice et al., “Visually Navigating the RMS Titanic With SLAM Information Filters”, Proceedings of Robotics Science and Systems, Jun. 2005, 9 pgs. [cited by applicant]
Forster et al., “IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-posteriori Estimation”, Proceedings of Robotics: Science and Systems, Jul. 2015, 10 pgs. [cited by applicant]
Forster et al., “SVO: Semi-Direct Visual Odometry for Monocular and Multi-Camera Systems”, IEEE Transactions on Robotics, Apr. 2017, vol. 33, No. 2, 18 pgs. [cited by applicant]
Furgale et al., “Unified temporal and spatial calibration for multi-sensor systems”, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Nov. 3-7, 2013, pp. 1280-1286. [cited by applicant]
Garcia et al., “Augmented State Kalman Filtering for AUV Navigation”, Proceedings of the 2002 IEEE International Conference on Robotics & Automation, May 2002, 6 pgs. [cited by applicant]
George et al., “Inertial Navigation Aided by Monocular Camera Observations of Unknown Features”, IEEE International Conference on Robotics and Automation, Roma, Italy, Apr. 10-14, 2007, pp. 3558-3564. [cited by applicant]
Goldshtein et al., “Seeker Gyro Calibration via Model-Based Fusion of Visual and Inertial Data”, 10th International Conference on Information Fusion, 2007, pp. 1-8. [cited by applicant]
Golub et al., “Matrix Computations, Fourth Edition”, The Johns Hopkins University Press, 2013, 780 pp., (Applicant points out, in accordance with JVIPEP 609.04(a), that the year of publication, 2013, is sufficiently ear… [cited by applicant]
Golub et al., “Matrix Computations, Third Edition”, The Johns Hopkins University Press, 2012, 723 pp. (Applicant points out that, in accordance with MPEP 609.04(a), the 2012 year of publication is sufficiently earlier t… [cited by applicant]
Golub et al., “Matrix Multiplication Problems”, Chapter 1, Matrix Computations, Third Edition, ISBN 0-8018-5413-X, 1996, (Applicant points out, in accordance with MPEP 609.04(a), that the year of publication “continued … [cited by applicant]
Guivant et al., “Optimization of the Simultaneous Localization and Map-Building Algorithm for Real-Time Implementation”, IEEE Transactions on Robotics and Automation, Jun. 2001, vol. 17, No. 3, 16 pgs. [cited by applicant]
Guo et al., “An Analytical Least-Squares Solution to the Odometer-Camera Extrinsic Calibration Problem”, Proceedings of the IEEE International Conference on Robotics and Automation, May 2012, 7 pgs. [cited by applicant]
Guo et al., “Efficient Visual-Inertial Navigation using a Rolling-Shutter Camera with Inaccurate Timestamps”, Proceedings of Robotics: Science and Systems, Jul. 2014, 9 pgs. [cited by applicant]
Guo et al., “Efficient Visual-Inertial Navigation Using a Rolling-Shutter Camera with Inaccurate Timestamps”, University of Minnesota, Multiple Autonomous Robotic Systems Laboratory Technical Report No. 2014-001, Feb. 2… [cited by applicant]
Guo et al., “IMU-RGBD camera 3D pose estimation and extrinsic calibration: Observability analysis and consistency improvement”, Proceedings of the IEEE International Conference on Robotics and Automation, May 6-10, 2013… [cited by applicant]
Guo et al., “IMU-RGBD Camera 3D Pose Estimation and Extrinsic Calibration: Observability Analysis and Consistency Improvement”, Proceedings of the IEEE International Conference on Robotics and Automation. May 6-10, 2013… [cited by applicant]
Guo et al., “Observability-constrained EKF Implementation of the IMU-RGBD Camera Navigation Using Point and Plane Features”, Technical Report, University of Minnesota, Mar. 2013, 6 pgs. [cited by applicant]
Guo et al., “Resource-Aware Large-Scale Cooperative 3D Mapping from Multiple Cell Phones”, Multiple Autonomous Robotic Systems (MARS) Lab, ICRA Poster May 26-31, 2015, 1 pg. [cited by applicant]
Gupta et al., “Terrain Based Vehicle Orientation Estimation Combining Vision and Inertial Measurements”, Journal of Field Robotics vol. 25, No. 3, 2008, pp. 181-202. [cited by applicant]
Harris et al., “A combined corner and edge detector”, Proceedings of the Alvey Vision Conference, Aug. 31-Sep. 2, 1988, pp. 147-151. [cited by applicant]
Hermann et al., “Nonlinear Controllability and Observability”, IEEE Transactions on Automatic Control, Oct. 1977, vol. AC-22, No. 5, pp. 728-740, doi: 10.1109/TAC.1977.1101601. [cited by applicant]
Herrera et al., “Joint Depth and Color Camera Calibration with Distortion Correction”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Oct. 2012, vol. 34, No. 10, pp. 2058-2064. [cited by applicant]
Hesch et al., “Camera-IMU-Based Localization: Observability Analysis and Consistency Improvement”, The International Journal of Robotics Research, 2014, vol. 33, No. 1, pp. 182-201. [cited by applicant]
Hesch et al., “Consistency Analysis and Improvement for Single-Camera Localization”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2012, pp. 15-22. [cited by applicant]
Hesch et al., “Consistency analysis and improvement of vision-aided inertial navigation”, IEEE Transactions on Robotics, Feb. 2014, vol. 30, No. 1, pp. 158-176. [cited by applicant]
Hesch et al., “Observability-constrained Vision-aided Inertial Navigation”, University of Minnesota, Department of Computer Science and Engineering, MARS Lab, Feb. 2012, 24 pgs. [cited by applicant]
Hesch et al., “Towards Consistent Vision-aided Inertial Navigation”, Proceedings of the 10th International Workshop on the Algorithmic Foundations of Robotics, Jun. 13-15, 2012, 16 pgs. [cited by applicant]
Heyden et al., “Structure and Motion in 3D and 2D from Hybrid Matching Constraints”, 2007, 14 pgs. [cited by applicant]
Higham, “Matrix Inversion”, Chapter 14, Accuracy and Stability of Numerical Algorithms, Second Edition, ISBN 0-89871-521-0, 2002, 29 pp. (Applicant points out, “continued in misc”, in accordance with MPEP 609.04(a), tha… [cited by applicant]
Horn, “Closed-form solution of absolute orientation using unit quaternions”, Journal of the Optical Society of America A, vol. 4, Apr. 1987, 14 pgs. [cited by applicant]
Horn et al., “Closed-form solution of absolute orientation using orthonormal matrices”, Journal of the Optical Society of America A, Jul. 1988, vol. 5, No. 7, pp. 1127-1135. [cited by applicant]
Huang et al., “Observability-based rules for designing consistent EKF slam estimators”, International Journal of Robotics Research, Apr. 2010, vol. 29, No. 5, pp. 502-528. [cited by applicant]
Huang et al., “Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera”, Proceedings of the International Symposium on Robotics Research, Aug. 28-Sep. 1, 2011, 16 pgs. [cited by applicant]
Huster et al., “Relative Position Sensing by Fusing Monocular Vision and Inertial Rate Sensors”, Stanford University, Department of Electrical Engineering, Dissertation. Jul. 2003, 158 pgs. [cited by applicant]
Jia et al., “Probabilistic 3-D motion estimation for rolling shutter video rectification from visual and inertial measurements”, Proceedings of the IEEE International Workshop on Multimedia Signal Processing, Sep. 2012,… [cited by applicant]
Johannsson et al., “Temporally Scalable Visual Slam Using a Reduced Pose Graph”, in Proceedings of the IEEE International Conference on Robotics and Automation, May 6-10, 2013, 8 pgs. [cited by applicant]
Jones et al., “Visual-Inertial Navigation, Mapping and Localization: A Scalable Real-time Causal Approach”, International Journal of Robotics Research, Mar. 31, 2011, vol. 30, No. 4, pp. 407-430. [cited by applicant]
Julier, “A Sparse Weight Kalman Filter Approach to Simultaneous Localisation and Map Building”, Proceedings of the IEEE/RSJ International Conference on Intelligent robots and Systems, Oct. 2001, vol. 3, 6 pgs. [cited by applicant]
Julier et al., “A Non-divergent Estimation Algorithm in the Presence of Unknown Correlations”, Proceedings of the American Control Conference, Jun. 1997, vol. 4, 5 pgs. [cited by applicant]
Kaess et al., “iSAM: Incremental Smoothing and Mapping”, IEEE Transactions on Robotics, Manuscript, Sep. 7, 2008, 14 pgs. [cited by applicant]
Kaess et al., “iSAM2: Incremental Smoothing and Mapping Using the Bayes Tree”, International Journal of Robotics Research, vol. 31, No. 2, Feb. 2012, 19 pgs. [cited by applicant]
Kelly et al., “A general framework for temporal calibration of multiple proprioceptive and exteroceptive sensors”, Proceedings of International Symposium on Experimental Robotics, Dec. 18-21, 2010, 15 pgs. [cited by applicant]
Kelly et al., “Visual-inertial sensor fusion: Localization, mapping and sensor-to-sensor self-calibration”, International Journal of Robotics Research, Jan. 2011, vol. 30, No. 1, pp. 56-79. [cited by applicant]
Klein et al., “Improving the Agility of Keyframe-Based SLAM”, ECCV 2008, Part II, LNCS 5303, pp. 802-815. [cited by applicant]
Klein et al., “Parallel Tracking and Mapping for Small AR Workspaces”, Proceedings of the IEEE and ACM International Symposium on Mixed and Augmented Reality, Nov. 13-16, 2007, pp. 225-234. [cited by applicant]
Kneip et al., “Robust Real-Time Visual Odometry with a Single Camera and an IMU”, Proceedings of the British Machine Vision Conference, Aug. 29-Sep. 2, 2011, pp. 16.1-16.11. [cited by applicant]
Konolige et al., “Efficient Sparse Pose Adjustment for 2D Mapping”, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct. 18-22, 2010, pp. 22-29. [cited by applicant]
Konolige et al., “FrameSLAM: from Bundle Adjustment to Real-Time Visual Mapping”, IEEE Transactions on Robotics, Oct. 2008, vol. 24, No. 5, pp. 1066-1077. [cited by applicant]
Konolige et al., “View-based Maps”, International Journal of Robotics Research, Jul. 2010, vol. 29, No. 8, pp. 941-957. [cited by applicant]
Kottas et al., “A Resource-aware Vision-aided Inertial Navigation System for Wearable and Portable Computers”, IEEE International Conference on Robotics and Automation, Accepted Apr. 18, 2014, available online May 6, 20… [cited by applicant]
Kottas et al., “An Iterative Kalman Smoother for Robust 3D Localization and mapping”, ISRR, Tech Report, Oct. 16, 2014, 15 pgs. [cited by applicant]
Kottas et al., “An iterative Kalman smoother for robust 3D localization on mobile and wearable devices”, Proceedings of the IEEE International Conference on Robotics and Automation, May 26-30, 2015, pp. 6336-6343. [cited by applicant]
Kottas et al., “An Iterative Kalman Smoother for Robust 3D Localization on Mobile and Wearable devices”, Submitted confidentially to International Conference on Robotics & Automation, ICRA, 15, May 5, 2015, 8 pgs. [cited by applicant]
Kottas et al., “Detecting and dealing with hovering maneuvers in vision-aided inertial navigation systems”, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Nov. 3-7, 2013, pp. 317… [cited by applicant]
Kottas et al., “Efficient and Consistent Vision-aided Inertial Navigation using Line Observations”, Department of Computer Science & Engineering, University of Minnesota, MARS Lab, Sep. 2012, TR-2012-002, 14 pgs. [cited by applicant]
Kottas et al., “On the Consistency of Vision-aided Inertial Navigation”, Proceedings of the International Symposium on Experimental Robotics, Jun. 17-20, 2012, 15 pgs. [cited by applicant]
Kummerle et al., “g20: A General Framework for Graph Optimization”, in Proceedings of the IEEE International Conference on Robotics and Automation, May 9-13, 2011, pp. 3607-3613. [cited by applicant]
Langelaan, “State Estimation for Autonomous Flight in Cluttered Environments”, Stanford University, Department of Aeronautics and Astronautics, Dissertation, Mar. 2006, 128 pgs. [cited by applicant]
Latif et al., “Applying Sparse '1-Optimization to Problems in Robotics”, ICRA 2014 Workshop on Long Term Autonomy, Jun. 2014, 3 pgs. [cited by applicant]
Lee et al., “Pose Graph-Based RGB-D SLAM in Low Dynamic Environments”, ICRA Workshop on Long Term Autonomy, 2014, 19 pgs. [cited by applicant]
Leutenegger et al., “Keyframe-based visual-inertial odometry using nonlinear optimization”, The International Journal of Robotics Research, Mar. 2015, vol. 34, No. 3, pp. 1-21, DOI: 10.1177/0278364914554813. [cited by applicant]
Li et al, “High-Precision, Consistent EKF-based Visual-Inertial Odometry”, International Journal of Robotics Research, May 2013, vol. 32, No. 6, 33 pp, DOI: 10.1177/0278364913481251. [cited by applicant]
Li et al., “3-D motion estimation and online temporal calibration for camera-IMU systems”, Proceedings of the IEEE International Conference on Robotics and Automation, May 6-10, 2013, 8 pgs. [cited by applicant]
Li et al., “Improving the Accuracy of EKF-based Visual-Inertial Odometry”, IEEE International Conference on Robotics and Automation, May 14-18, 2012, 8 pgs. [cited by applicant]
Li et al., “Optimization-Based Estimator Design for Vision-Aided Inertial Navigation”, Proceedings of the Robotics: Science and Systems Conference, Jul. 9-13, 2012, 8 pgs. [cited by applicant]
Li et al., “Real-time Motion Tracking on a Cellphone using Inertial Sensing and a Rolling-Shutter Camera”, 2013 IEEE International Conference on Robotics and Automation (ICRA), May 6-10, 2013, 8 pgs. [cited by applicant]
Li et al., “Vision-aided inertial navigation with rolling-shutter cameras”, The International Journal of Robotics Research, retrieved from ijr.sagepub.com on May 22, 2015, 18 pgs. [cited by applicant]
Lim et al., “Zero-Configuration Indoor Localization over IEEE 802.11 Wireless Infrastructure”, Jun. 23, 2008, 31 pgs. [cited by applicant]
Liu et al., “Estimation of Rigid Body Motion Using Straight Line Correspondences”, Computer Vision, Graphics, and Image Processing, Jul. 1988, vol. 43, No. 1, pp. 37-52. [cited by applicant]
Liu et al., “ICE-BA: Incremental, Consistent and Efficient Bundle Adjustment for Visual-Inertial SLAM”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 1974-1982. [cited by applicant]
Liu et al., “Multi-aided inertial navigation for ground vehicles in outdoor uneven environments”, Proceedings of the IEEE International Conference on Robotics and Automation, Apr. 18-22, 2005, pp. 4703-4708. [cited by applicant]
Lowe, “Distinctive Image Features from Scale-Invariant Keypoints”, Retrieved from: https://www.robots.ox.ac.uk/˜vgg/research/affine/det_eval_files/lowe_ijcv2004.pdf , Jan. 5, 2004, Accepted for publication in the Intern… [cited by applicant]
Lucas et al., “An iterative image registration technique with an application to stereo vision”, Proceedings of 7th the International Joint Conference on Artificial Intelligence, 1981, pp. 121-130. [cited by applicant]
Lucas et al., “An Iterative Image Registration Technique with an Application to Stereo Vision”, Proceedings of the 7th International Joint Conference on Artificial Intelligence, Aug. 24-28, 1981, pp. 674-679. [cited by applicant]
Lupton et al., “Visual-Inertial-Aided Navigation for High-Dynamic Motion in Built Environments Without Initial Conditions”, IEEE Transactions on Robotics, Feb. 2012, vol. 28, No. 1, pp. 61-76, 10.1109/TRO.2011.2170332. [cited by applicant]
Lynen et al., “Get Out of My Lab: Large-scale, Real-Time Visual-Inertial Localization”, Proceedings of robotics: Science and Systems, Jul. 2015, 10 pgs. [cited by applicant]
Martinelli, “Closed-form Solution of Visual-inertial structure from Motion”, International Journal of Computer Vision, Jan. 2014, vol. 106, No. 2, 16 pgs. [cited by applicant]
Martinelli, “Vision and IMU Data Fusion: Closed-form Solutions for Attitude, Speed, Absolute Scale, and Bias Determination”, IEEE Transactions on Robotics, Feb. 2012, vol. 28, No. 1, pp. 44-60, DOI: 10.1109/TRO.2011.216… [cited by applicant]
Matas et al., “Robust Detection of Lines Using the Progressive Probabilistic Hough Transformation”, Computer Vision and Image Understanding, Apr. 2000, vol. 78, No. 1, pp. 119-137, doi:10.1006.cviu.1999.0831. [cited by applicant]
Maybeck, “Stochastic models, estimation and control”, Academy Press, May 28, 1979, vol. 1, Chapter 1, 19 pgs. [cited by applicant]
McLauchlan, “The Variable State Dimension Filter Applied to Surface-Based Structure From Motion CVSSP Technical Report VSSP-TR-4/99”, University of Surrey, Department of Electrical Engineering, Dec. 1999, 52 pgs. [cited by applicant]
Meltzer et al., “Edge Descriptors for Robust Wide-baseline Correspondence”, IEEE Conference on Computer Vision and Pattern Recognition, Jun. 23-28, 2008, pp. 1-8. [cited by applicant]
Mirzaei et al., “A Kalman Filter-Based Algorithm for IMU-Camera Calibration: Observability Analysis and Performance Evaluation”, IEEE Transactions on Robotics, Oct. 2008, vol. 24, No. 5, pp. 1143-1156, doi:10.1006.cviu.… [cited by applicant]
Mirzaei et al., “Globally Optimal Pose Estimation from Line Correspondences”, IEEE International Conference on Robotics and Automation, May 9-13, 2011, pp. 5581-5588. [cited by applicant]
Mirzaei et al., “Optimal Estimation of Vanishing Points in a Manhattan World”, IEEE International Conference on Computer Vision, Nov. 6-13, 2011, pp. 2454-2461. [cited by applicant]
Montiel et al., “Unified Inverse Depth Parametrization for Monocular SLAM”, Proceedings of Robotics: Science and Systems II (RSS-06), Aug. 16-19, 2006, 8 pgs. [cited by applicant]
Mourikis et al., “A Dual-Layer Estimator Architecture for Long-term Localization”, Proceedings of the Workshop on Visual Localization for Mobile Platforms, Jun. 24-26, 2008, 8 pgs. [cited by applicant]
Mourikis et al., “A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation”, IEEE International Conference on Robotics and Automation, Apr. 10-14, 2007, pp. 3565-3572. [cited by applicant]
Mourikis et al., “A Multi-State Constraint Kalman Filter for Vision-aided Inertial Navigation”, IEEE International Conference on Robotics and Automation, Sep. 28, 2006, 20 pgs. [cited by applicant]
Mourikis et al., “A Multi-State Constraint Kalman Filter for Vision-Aided Inertial Navigation”, University of Minnesota, Dept. of Computer Science and Engineering, 2006, 20 pgs. [cited by applicant]
Mourikis et al., “On the Treatment of Relative-Pose Measurements for Mobile Robot Localization”, Proceedings of the 2006 IEEE International Conference on Robotics and Automation, May 2006, pp. 2277-2284. [cited by applicant]
Mourikis et al., “SC-KF Mobile Robot Localization: A Stochastic Cloning Kalman Filter for Processing Relative-State Measurements”, IEEE Transactions on Robotics, vol. 23. No. 4, Aug. 2007, pp. 717-730. [cited by applicant]
Mourikis et al., “Vision-Aided Inertial Navigation for Spacecraft Entry, Descent, and Landing”, IEEE Transactions on Robotics, Apr. 2009, vol. 25, No. 2, pp. 264-280. [cited by applicant]
Mur-Artal et al., “ORB-SLAM: a Versatile and Accurate Monocular SLAM System”, IEEE Transactions on Robotics, Oct. 2015, vol. 31, No. 5, 17 pgs. [cited by applicant]
Mur-Artal et al., “ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras”, IEEE Transactions on Robotics, Jun. 2017, vol. 33, vol. 5, 9 pgs. [cited by applicant]
Mur-Artal et al., “Visual-inertial Monocular SLAM with Map Reuse”, IEEE Robotics and Automation Letters, Apr. 2017, vol. 2, No. 2, 8 pgs. [cited by applicant]
Nerurkar et al., “C-KLAM: Constrained Keyframe Localization and Mapping for Long-Term Navigation”, IEEE International Conference on Robotics and Automation, 2014, 3 pgs. [cited by applicant]
Nerurkar et al., “C-KLAM: Constrained Keyframe-Based Localization and Mapping”, Proceedings of the IEEE International Conference on Robotics and Automation, May 31-Jun. 7, 2014, 6 pgs. [cited by applicant]
Nerurkar et al., “Power-SLAM: A Linear-Complexity, Anytime Algorithm for SLAM”, International Journal of Robotics Research, May 2011, vol. 30, No. 6, 13 pgs. [cited by applicant]
Nister et al., “Scalable Recognition with a Vocabulary Tree”, IEEE Computer Vision and Pattern Recognition, 2006, 8 pgs. [cited by applicant]
Nister et al., “Visual Odometry for Ground Vehicle Applications”, Journal of Field Robotics, Jan. 2006, vol. 23, No. 1, 35 pgs. [cited by applicant]
Nocedal et al., “Numerical Optimization”, 2nd Ed. Springer, 2006, 683 pp. (Applicant points out, in accordance with MPEP 609.04(a), that the year of publication, 2006, is sufficiently earlier than the effective U.S. fil… [cited by applicant]
Oliensis, “A New Structure From Motion Ambiguity”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Jul. 2000, vol. 22, No. 7, 30 pgs. [cited by applicant]
Ong et al., “Six DoF Decentralised SLAM”, Proceedings of the Australasian Conference on Robotics and Automation, 2003, 10 pp. (Applicant points out, in accordance with MPEP 609.04(a), . . . continued in misc., that the … [cited by applicant]
Oth et al., “Rolling shutter camera calibration”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 23-28, 2013, pp. 1360-1367. [cited by applicant]
Perea et al., “Sliding Windows and Persistence: An Application of Topological Methods to Signal Analysis”, Foundations of Computational: Mathematics, Nov. 25, 2013, 34 pgs. [cited by applicant]
Prazenica et al., “Vision-Based Kalman Filtering for Aircraft State Estimation and Structure From Motion”, AIAA Guidance, Navigation, and Control Conference and Exhibit, Aug. 15-18, 2005, 13 pgs. [cited by applicant]
Qin et al., “VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator”, IEEE Transactions on Robotics, Aug. 2018, vol. 34, No. 4, 17 pgs. [cited by applicant]
Ragab et al., “EKF Based Pose Estimation Using Two Back-to-Back Stereo Pairs”, 14th IEEE International Conference on Image Processing, Sep. 16-19, 2007, 4 pgs. [cited by applicant]
Randeniya, “Automatic Geo-Referencing by Integrating Camera Vision and Inertial Measurements”, University of South Florida, Scholar Commons, Graduate Theses and Dissertations, 2007, 177 pgs. [cited by applicant]
Rosten et al., “Machine Learning for High-Speed Corner Detection”, Proceedings of the 9th European Conference on Computer Vision, May 2006, 14 pgs. [cited by applicant]
Roumeliotis et al., Prosecution History for U.S. Appl. No. 12/383,371, filed Mar. 23, 2009, issued on Sep. 19, 2017 as U.S. Pat. No. 9,766,074, 608 pgs. [cited by applicant]
Roumeliotis et al., Prosecution History for U.S. Appl No. 15/706,149, filed Sep. 15, 2017, issued on Jun. 2, 2020 as U.S. Pat. No. 10,670,404, 268 pgs. [cited by applicant]
Roumeliotis et al., “Augmenting Inertial Navigation With Image-Based Motion Estimation”, IEEE International Conference on Robotics and Automation, 2002, vol. 4, p. 8 (Applicant points out that, in accordance with MPEP 6… [cited by applicant]
Roumeliotis et al., “Stochastic Cloning: A Generalized Framework for Processing Relative State Measurements”, Proceedings of the 2012 IEEE International Conference on Robotics and Automation, May 11-15, 2002, pp. 1788-1… [cited by applicant]
Rublee et al., “ORB: An efficient alternative to SIFT or SURF”, 2011 International Conference on Computer Vision, Nov. 6-13, 2011, Barcelona, Spain, pp. 2564-2571, DOI: 10.1109/ICCV.2011.6126544. [cited by applicant]
Sartipi et al., “Decentralized Visual-Inertial Localization and Mapping on Mobile Devices for Augmented Reality”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, 9 pgs. [cited by applicant]
Schmid et al., “Automatic Line Matching Across Views”, Proceedings of the IEEE Computer Science Conference on Computer Vision and Pattern Recognition, Jun. 17-19, 1997, pp. 666-671. [cited by applicant]
Se et al., “Visual Motion Estimation and Terrain Modeling for Planetary Rovers”, International Symposium on Artificial Intelligence for Robotics and Automation in Space, Munich, Germany, Sep. 2005, 8 pgs. [cited by applicant]
Servant et al., “Improving Monocular Plane-based SLAM with Inertial Measurements”, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct. 18-22, 2010, pp. 3810-3815. [cited by applicant]
Shoemake et al., “Animating rotation with quaternion curves”, ACM Siggraph Computer Graphics, Jul. 22-26, 1985, vol. 19, No. 3, pp. 245-254. [cited by applicant]
Sibley et al., “Sliding Window Filter with Application to Planetary Landing”, Journal of Field Robotics, Sep./Oct. 2010, vol. 27, No. 5, pp. 587-608. [cited by applicant]
Smith et al., “On the Representation and Estimation of Spatial Uncertainty”, International Journal of Robotics Research, 1986, vol. 5, No. 4, pp. 56-68 (Applicant points out that, in accordance with MPEP 609.04(a), the … [cited by applicant]
Smith et al., “Real-time Monocular Slam with Straight Lines”, British Machine Vision Conference, Sep. 2006, vol. 1, pp. 17-26. [cited by applicant]
Soatto et al., “Motion Estimation via Dynamic Vision”, IEEE Transactions on Automatic Control, vol. 41, No. 3, Mar. 1996, pp. 393-413. [cited by applicant]
Soatto et al., “Recursive 3-D Visual Motion Estimation Using Subspace Constraints”, International Journal of Computer Vision, Mar. 1997, vol. 22, No. 3, pp. 235-259. [cited by applicant]
Spetsakis et al., “Structure from Motion Using Line Correspondences”, International Journal of Computer Vision, Jun. 1990, vol. 4, No. 3, pp. 171-183. [cited by applicant]
Strelow, “Motion Estimation From Image and Inertial Measurements”, Carnegie Mellon University, School of Computer Science, Dissertation, Nov. 2004, CMU-CS-04-178, 164 pgs. [cited by applicant]
Sturm, “Tracking and Mapping in Project Tango”, Project Tango, accessed from https://jsturm.de/publications/data/sturm2015_dagstuhl.pdf accessed on Jun. 1, 2021, published 2015, (Applicant points out, in accordance with… [cited by applicant]
Taylor et al., “Parameterless Automatic Extrinsic Calibration of Vehicle Mounted Lidar-Camera Systems”, Conference Paper, Mar. 2014, 3 pgs. [cited by applicant]
Taylor et al., “Structure and Motion from Line Segments in Multiple Images”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Nov. 1995, vol. 17, No. 11, pp. 1021-1032. [cited by applicant]
Thorton et al., “Triangular Covariance Factorizations for Kalman Filtering”, Technical Memorandum 33-798, National Aeronautics and Space Administration, Oct. 15, 1976, 212 pgs. [cited by applicant]
Thrun et al., “The Graph SLAM Algorithm with Applications to Large-Scale Mapping of Urban Structures”, The International Journal of Robotics Research, May 2006, vol. 25, No. 5-6, pp. 403-429. [cited by applicant]
Torr et al., “Robust Parameterization and Computation of the Trifocal Tensor”, Image and Vision Computing, vol. 15, No. 8, Aug. 1997, 25 pgs. [cited by applicant]
Trawny et al., “Indirect Kalman Filter for 3D Attitude Estimation”, University of Minnesota, Department of Computer Science & Engineering, MARS Lab, Mar. 2005, 25 pgs. [cited by applicant]
Trawny et al., “Vision-Aided Inertial Navigation for Pin-Point Landing using Observations of Mapped Landmarks”, Journal of Field Robotics, vol. 24, No. 5, May 2007, pp. 357-378. [cited by applicant]
Triggs et al., “Bundle Adjustment—A Modern Synthesis”, Vision Algorithms: Theory & Practice, Apr. 12, 2002, LNCS 1883, 71 pgs. [cited by applicant]
Triggs et al., “Bundle Adjustment—A Modern Synthesis”, Proceedings of the International Workshop on Vision Algorithms: Theory and Practice, Lecture Notes in Computer Science, Sep. 21-22, 1999, vol. 1883, pp. 298-372. [cited by applicant]
Weiss et al., “Real-time Metric State Estimation for Modular Vision-inertial Systems”, IEEE International Conference on Robotics and Automation, May 9-13, 2011, pp. 4531-4537. [cited by applicant]
Weiss et al., “Real-time Onboard Visual-Inertial State Estimation and Self-Calibration of MAVs in Unknown Environments”, IEEE International Conference on Robotics and Automation, May 14-18, 2012, pp. 957-964. [cited by applicant]
Weiss et al., “Versatile Distributed Pose Estimation and sensor Self-Calibration for an Autonomous MAV”, IEEE International Conference on Robotics and Automations, May 14-18, 2012, pp. 31-38. [cited by applicant]
Weng et al., “Motion and Structure from Line Correspondences: Closed-Form Solution, Uniqueness, and Optimization”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Mar. 1992, vol. 14, No. 3, pp. 318-336. [cited by applicant]
Williams et al., “Feature and Pose Constrained Visual Aided Inertial Navigation for Computationally Constrained Aerial Vehicles”, IEEE International Conference on Robotics and Automation, May 9-13, 2011, pp. 431-438. [cited by applicant]
Wu et al., “A Square Root Inverse Filter for Efficient Vision-aided Inertial Navigation on Mobile Devices”, Proceedings of Robotics: Science and Systems, Jul. 2015, 9 pgs. [cited by applicant]
Yu, “Model-less Pose Tracking”, The Chinese University of Hong Kong, Thesis Submitted for the Degree of Doctor of Philosophy, dated Jul. 2007, 153 pgs. [cited by applicant]
Yu et al., “Controlling Virtual Cameras Based on a Robust Model-Free Pose Acquisition Technique”, IEEE Transactions on Multimedia, No. 1, 2009, pp. 184-190. [cited by applicant]
Zhou et al., “Determining 3D Relative Transformations for Any Combination of Range and Bearing Measurements”, IEEE Transactions on Robotics, Apr. 2013, vol. 29, No. 2, pp. 458-474. [cited by applicant]