IP Library › Granted Patent US 12,509,251
Granted Patent B2
US 12,509,251 · App. 18/499,567 · Granted Dec 30, 2025

Autonomous online navigation and parameter estimation in the vicinity of small celestial bodies

Inventors: Mehregan Dor (Atlanta, GA); Travis Driver (Atlanta, GA); Panagiotis Tsiotras (Atlanta, GA)
Assignee: Georgia Tech Research Corporation
B64G1/247B64G3/00G01C21/24G06T7/73G06V20/56G06V20/64G06T2200/04G06T2207/10032G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,509,251
App. No.
18/499,567
Granted
Dec 30, 2025
Kind
B2
Abstract

An exemplary system and method are disclosed for a standalone vision-based system for autonomous online navigation, e.g., around an unknown target small body. The exemplary system (also referred to as AstroSLAM) is predicated on the formulation of the SLAM problem as an incrementally growing factor graph. Using the GTSAM library and the iSAM2 engine, the exemplary system and method combine sensor fusion with the prior orbital motion to provide navigation and parameter estimation that improves the performance over a baseline SLAM implementation. The exemplary system and method can incorporate orbital motion constraints into the factor graph by devising a relative dynamics factor that can link the relative pose of the spacecraft to the problem of predicting trajectories stemming from the motion of the spacecraft in the vicinity of the small body.

Claims (58)

1 . A computer-implemented comprising:

receiving a plurality of images of a small celestial body by a spacecraft, wherein each image of the plurality of images is associated with a time;

initializing a plurality of spacecraft states and small celestial body states by the spacecraft;

based on the received plurality of images, generating a 3D map of the small celestial body including a plurality of landmarks by the spacecraft, wherein each landmark is associated with a location in the 3D map;

generating a factor graph corresponding to the plurality of landmarks and the plurality of spacecraft states and small celestial body states by the spacecraft;

receiving a new image by the spacecraft;

based on the new image and some or all of the plurality of images, identifying one or more new landmarks in the 3D map and a changed location for at least one of the plurality of landmarks by the spacecraft;

updating the factor graph based on the one or more new landmarks and the changed location by the spacecraft;

receiving relative dynamical constraints by the spacecraft, wherein the relative dynamical constraints include at least one RelDyn factor that links a relative pose of the spacecraft to predicted trajectories of the spacecraft stemming from a motion of the spacecraft in the vicinity of the small celestial body;

solving a navigation problem based on the factor graph and the relative dynamical constraints by the spacecraft to generate relative orientations of the spacecraft and the small celestial body, relative positions of the spacecraft and the small celestial body, and relative velocities of the relative positions of the spacecraft and the small celestial body;

performing path planning for the spacecraft by the spacecraft using some or all of the relative orientations of the spacecraft and the small celestial body, the relative positions of the spacecraft and the small celestial body, the relative velocities of the relative positions of the spacecraft and the small celestial body, and the 3D map; and

causing the spacecraft to travel based on the path planning by the spacecraft.

2 . The computer-implemented method of claim 1 , further comprising:

receiving visual odometry constraints and relative kinematic constraints; and

solving the navigation problem based on the factor graph and the received visual odometric constraints and the relative kinematic and dynamical constraints.

3 . The computer-implemented of claim 1 , further comprising receiving prior information about the small celestial object and initializing the plurality of small celestial body states based on the prior information.

4 . The computer-implemented method of claim 3 , wherein the prior information comprises a spin state estimate.

5 . The computer-implemented method of claim 1 , further comprising solving the navigation problem based on the factor graph by the spacecraft to generate an estimate of a center-of-mass position of the small celestial body.

6 . The computer-implemented method of claim 1 , wherein the small celestial body comprises an asteroid or a comet.

7 . A spacecraft comprising:

at least one computing device; and

a computer-readable medium with computer-executable instructions stored thereon that when executed by the at least one computing device cause the at least one computing device to:

receive a plurality of images of a small celestial body, wherein each image of the plurality of images is associated with a time;

initialize a plurality of spacecraft states and small celestial body states;

based on the received plurality of images, generate a 3D map of the small celestial body including a plurality of landmarks, wherein each landmark is associated with a location in the 3D map;

generate a factor graph corresponding to the plurality of landmarks and the plurality of spacecraft states and small celestial body states;

receive a new image;

based on the new image and some or all of the plurality of images, identify one or more new landmarks in the 3D map and a changed location for at least one of the plurality of landmarks;

update the factor graph based on the one or more new landmarks and the changed location;

receive relative dynamical constraints by the spacecraft, wherein the relative dynamical constraints include at least one RelDyn factor that links a relative pose of the spacecraft to predicted trajectories of the spacecraft stemming from a motion of the spacecraft in the vicinity of the small celestial body;

solve a navigation problem based on the factor graph and the relative dynamical constraints to generate relative orientations of the spacecraft and the small celestial body, relative positions of the spacecraft and the small celestial body, and relative velocities of the relative positions of the spacecraft and the small celestial body;

perform path planning for the spacecraft using some or all of the relative orientations of the spacecraft and the small celestial body, the relative positions of the spacecraft and the small celestial body, the relative velocities of the relative positions of the spacecraft and the small celestial body, and the 3D map; and

cause the spacecraft to travel based on the path planning.

8 . The spacecraft of claim 7 , further comprising computer-executable instructions that when executed by the at least one computing device cause the at least one computing device to:

receive visual odometry constraints and relative kinematic constraints; and

solve the navigation problem based on the factor graph and the received visual odometric constraints and the relative kinematic and dynamical constraints.

9 . The spacecraft of claim 7 , further comprising receiving prior information about the small celestial object and initializing the plurality of small celestial body states based on the prior information.

10 . The spacecraft of claim 9 , wherein the prior information comprises a spin state estimate.

11 . The spacecraft of claim 7 , further comprising solving the navigation problem based on the factor graph by the spacecraft to generate an estimate of a center-of-mass position of the small celestial body.

12 . The spacecraft of claim 7 , wherein the small celestial body comprises an asteroid or a comet.

13 . A non-transitory computer-readable medium with computer-executable instructions stored thereon that when executed by the at least one computing device cause the at least one computing device to:

receive a plurality of images of a small celestial body, wherein each image of the plurality of images is associated with a time;

initialize a plurality of spacecraft states of a spacecraft and small celestial body states;

based on the received plurality of images, generate a 3D map of the small celestial body including a plurality of landmarks, wherein each landmark is associated with a location in the 3D map;

generate a factor graph corresponding to the plurality of landmarks and the plurality of spacecraft states and small celestial body states;

receive a new image;

based on the new image and some or all of the plurality of images, identify one or more new landmarks in the 3D map and a changed location for at least one of the plurality of landmarks;

update the factor graph based on the one or more new landmarks and the changed location;

receive relative dynamical constraints by the spacecraft, wherein the relative dynamical constraints include at least one RelDyn factor that links a relative pose of the spacecraft to predicted trajectories of the spacecraft stemming from a motion of the spacecraft in the vicinity of the small celestial body;

solve a navigation problem based on the factor graph and the relative dynamical constraints to generate relative orientations of the spacecraft and the small celestial body, relative positions of the spacecraft and the small celestial body, and relative velocities of the relative positions of the spacecraft and the small celestial body;

perform path planning for the spacecraft using some or all of the relative orientations of the spacecraft and the small celestial body, the relative positions of the spacecraft and the small celestial body, the relative velocities of the relative positions of the spacecraft and the small celestial body, and the 3D map; and

cause the spacecraft to travel based on the path planning.

14 . The computer-readable medium of claim 13 , further comprising computer-executable instructions that when executed by the at least one computing device cause the at least one computing device to:

receive visual odometry constraints and relative kinematic constraints; and

solve the navigation problem based on the factor graph and the received visual odometric constraints and the relative kinematic and dynamical constraints.

15 . The computer-readable medium of claim 13 , further comprising receiving prior information about the small celestial object and initializing the plurality of small celestial body states based on the prior information.

16 . The computer-readable medium of claim 15 , wherein the prior information comprises a spin state estimate.

17 . The computer-readable medium of claim 13 , further comprising solving the navigation problem based on the factor graph by the spacecraft to generate an estimate of a center-of-mass position of the small celestial body.

Continuity (2)
Provisional Application 63381821 · Nov 1, 2022
Related Publication 20240158104A1 · May 16, 2024
References Cited (59)
US 20220017239A1 · Chen · 2022 [cited by examiner]
US 20230286675A1 · Ward · 2023 [cited by examiner]
Dor, M., Skinner, K. A., Driver, T., and Tsiotras, P. (2021). Visual SLAM for 29. | Asteroid Relative Navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), V… [cited by examiner]
Baldini, F., Harvard, A., Chung, S., Nesnas, I., and Bhaskaran, S. (2018). 5. Autonomous Small Body Mapping and Spacecraft Navigation Via Real-Time SPC-SLAM. In 69th IAF International Astronautical Congress (IAC), Breme… [cited by examiner]
Anderson, S. and Barfoot, T. D. (2015). Full STEAM ahead: Exactly sparse Gaussian process regression for batch continuous-time trajectory estimation on SE(3). In 2015 IEEE/RSJ International Conference on Intelligent Rob… [cited by applicant]
Anderson, S., Barfoot, T. D., Tong, C. H., and S{umlaut over ( )}arkk{umlaut over ( )}a, S. (2015). Batch nonlinear continuous-time trajectory estimation as exactly sparse Gaussian process regression. Autonomous Robots,… [cited by applicant]
Andrle, M. S. and Crassidis, J. L. (2013). Geometric integration of quaternions. Journal of Guidance, Control, and Dynamics, 36(6):1762-1767. [cited by applicant]
Antreasian, P., Adam, C., Leonard, J., Lessac-Chenen, E., Levine, A., McAdams, J., McCarthy, L., Nelson, D., Page, B., Pelgrift, J., Sahr, E., Wibben, D., Williams, B., Williams, K., Berry, K., Getzandanner, K., Moreau,… [cited by applicant]
Baldini, F., Harvard, A., Chung, S., Nesnas, I., and Bhaskaran, S. (2018). Autonomous Small Body Mapping and Spacecraft Navigation Via Real-Time SPC-SLAM. In 69th IAF International Astronautical Congress (IAC), Bremen, … [cited by applicant]
Barnouin, O., Daly, M., Palmer, E., Johnson, C., Gaskell, R., Al Asad, M., Bierhaus, E., Craft, K., Ernst, C., Espiritu, R., et al. (2020). Digital terrain mapping by the OSIRIS-REx mission. Planetary and Space Science,… [cited by applicant]
Bay, H., Tuytelaars, T., and Van Gool, L. (2006). SURF: Speeded up robust features. In European Conference on Computer Vision, pp. 404-417, Graz, Austria. Springer. [cited by applicant]
Bercovici, B. and McMahon, J. W. (2019). Robust autonomous small-body shape reconstruction and relative navigation using range images. Journal of Guidance, Control, and Dynamics, 42(7):1473-1488. [cited by applicant]
Berry, K., Getzandanner, K., Moreau, M., Rieger, S., Antreasian, P., Adam, C., Wibben, D., Leonard, J., Levine, A., Geeraert, J., Lorenz, D., and Lauretta, D. S. (2022). Contact with Bennu! Flight Performance versus Pre… [cited by applicant]
Bhaskaran, S. and Kennedy, B. (2014). Closed loop terminal guidance navigation for a kinetic impactor spacecraft. Acta Astronautica, 103:322-332. [cited by applicant]
Bhaskaran, S., Nandi, S., Broschart, S., Wallace, M., Cangahuala, L. A., and Olson, C. (2011). Small body landings using autonomous onboard optical navigation. The Journal of the Astronautical Sciences, 58(3):409-427. [cited by applicant]
Chirikjian, G. (2011). Stochastic Models, Information Theory, and Lie Groups, vol. 2: Analytic Methods and Modern Applications, vol. 2. Springer Science & Business Media. [cited by applicant]
Cho, D.-M., Jung, D., and Tsiotras, P. (2009). A 5-DOF experimental platform for spacecraft rendezvous and docking. In AIAA Infotech@ Aerospace Conference and AIAA Unmanned . . . Unlimited Conference, p. 1869, Seattle, … [cited by applicant]
Christian, J. and Lightsey, G. (2012). Onboard image-processing algorithm for a spacecraft optical navigation sensor system. Journal of Spacecraft and Rockets, 49(2):337-352. [cited by applicant]
Church, E., Bourbeau, T., Curriden, J., Deguzman, A., Jaen, F., Ma, H., Mahoney, K., Miller, C., Short, B., Waldorff, K., Walthall, O., and Lauretta, D. (2020). Flash LIDAR On-Orbit Performance at Asteroid Bennu. In 43r… [cited by applicant]
Cocaud, C. and Kubota, T. (2010). SURF-based SLAM scheme using octree occupancy grid for autonomous landing on asteroids. In Proceedings of the 10th International Symposium on Artificial Intelligence, Robotics and Autom… [cited by applicant]
Cocaud, C. and Kubota, T. (2012). Autonomous navigation near asteroids based on visual SLAM. In Proceedings of the 23rd International Symposium on Space Flight Dynamics, Pasadena, CA, Oct. 2012. [cited by applicant]
Dellaert, F. (2021). Factor graphs: Exploiting structure in robotics. Annual Review of Control, Robotics, and Autonomous Systems, 4:141-166. [cited by applicant]
Dellaert, F. and Kaess, M. (2017). Factor graphs for robot perception. Foundations and Trends in Robotics, 6(1-2):1-139. [cited by applicant]
Delpech, M., Bissonnette, V., and Rastel, L. (2015). Vision-based navigation for proximity operations around asteroid 99942 Apophis. In Proceedings of 25th International Symposium on Space Flight Dynamics, Munich, Germa… [cited by applicant]
Dong, J., Mukadam, M., Dellaert, F., and Boots, B. (2016). Motion planning as probabilistic inference using Gaussian processes and factor graphs. In Robotics: Science and Systems (RSS), vol. 12, Ann Arbor, MI. [cited by applicant]
Dor, M., Skinner, K. A., Driver, T., and Tsiotras, P. (2021). Visual SLAM for Asteroid Relative Navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Virtual… [cited by applicant]
Driver, T., Dor, M., Skinner, K., and Tsiotras, P. (2020). Space Carving in Space: A Visual-SLAM Approach to 3D Shape Reconstruction of a Small Celestial Body. In AAS/AIAA Astrodynamics Specialist Conference, Virtual. [cited by applicant]
Fraser, D. C. (1967). A New Technique for the Optimal Smoothing of Data. PhD thesis, Massachusetts Institute of Technology. [cited by applicant]
Galvez-L{acute over ( )}opez, D. and Tardos, J. D. (2012). Bags of binary words for fast place recognition in image sequences. IEEE Transactions on Robotics, 28(5):1188-1197. [cited by applicant]
Gaskell, R., Barnouin-Jha, O., Scheeres, D., Konopliv, A., Mukai, T., Abe, S., Saito, J., Ishiguro, M., Kubota, T., and Hashimoto, T. (2008). Characterizing and navigating small bodies with imaging data. Meteoritics & P… [cited by applicant]
Getzandanner, K. M., Berry, K. E., Antreasian, P. G., Leonard, J. M., Adam, C. D., Wibben, D. R., Moreau, M. C., Highsmith, D. E., and Lauretta, D. S. (2022). Small-Body Proximity Operations & TAG: Navigation Experience… [cited by applicant]
Goossens, S., Rowlands, D. D., Mazarico, E., Liounis, A. J., Small, J. L., Highsmith, D. E., Swenson, J. C., Lyzhoft, J. R., Ashman, B. W., Getzandanner, K. M., et al. (2021). Mass and shape determination of (101955) Be… [cited by applicant]
Kaess, M., Johannsson, H., Roberts, R., Ila, V., Leonard, J., and Dellaert, F. (2012). iSAM2: Incremental smoothing and mapping using the Bayes tree. The International Journal of Robotics Research, 31(2):216-235. [cited by applicant]
Koller, D. and Friedman, N. (2009). Probabilistic Graphical Models: Principles and Techniques. MIT Press. [cited by applicant]
Konopliv, A., Asmar, S., Park, R., Bills, B., Centinello, F., Chamberlin, A., Ermakov, A., Gaskell, R., Rambaux, N., and Raymond, C. (2014). The Vesta gravity field, spin pole and rotation period, landmark positions, an… [cited by applicant]
Lauretta, D. (2021). The OSIRIS-REx Touch-and-Go Sample Acquisition Event and Implications for the Nature of the Returned Sample. In 52nd Lunar and Planetary Science Conference, Virtual. [cited by applicant]
Leonard, J., Moreau, M., Antreasian, P., Getzandanner, K., Church, E., Miller, C., Daily, M., Barnouin, O., and Lauretta, D. (2022a). Cross-Calibration of GNC and OLA LIDAR Systems Onboard OSIRIS-REx. In 44th Annual AAS… [cited by applicant]
Leonard, J. M., et al. (2020). OSIRIS-REx Shape Model Performance During the Navigation Campaign. In 43rd Annual AAS Guidance and Control Conference, Breckenridge, CO. [cited by applicant]
Leonard, J. M., Geeraert, J. L., Pelgrift, J. Y., Antreasian, P. G., Adam, C. D., Wibben, D. R., Getzandanner, K. M., Ashman, B. W., and Lauretta, D. S. (2022b). Navigation prediction performance during OSIRISREx proxim… [cited by applicant]
Lepetit, V., Moreno-Noguer, F., and Fua, P. (2009). Epnp: An accurate O(n) solution to the pnp problem. International Journal of Computer Vision, 81(2):155. [cited by applicant]
Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91-110. [cited by applicant]
Matsuzaki, T., Kameda, H., Tsujimichi, S., and Kosuge, K. (2000). Maneuvering target tracking using constant velocity and constant angular velocity model. In IEEE International Conference on Systems, Man and Cybernetics… [cited by applicant]
Miller, J. (2019). Measurements and calibrations. In Planetary Spacecraft Navigation, pp. 255-292. Springer International Publishing, Cham. pp. 255-292. [cited by applicant]
Miller, J. and Rourke, K. (1977). The application of differential VLBI to planetary approach orbit determination. The Deep Space Network Progress Report, 42:40. [cited by applicant]
Miller et al., J. K. (2002). Determination of shape, gravity and rotation state of asteroid 433 Eros. Icarus, 155(1):3-17. [cited by applicant]
Nakath, D., Clemens, J., and Rachuy, C. (2020). Active asteroid-SLAM. Journal of Intelligent & Robotic Systems, 99(2):303-333. [cited by applicant]
Nakath, D., Clemens, J., and Schill, K. (2018). Multi-Sensor Fusion and Active Perception for Autonomous Deep Space Navigation. In 21st International Conference on Information Fusion (FUSION), pp. 2596-2605, Cambridge, … [cited by applicant]
Nathues, A., Sierks, H., Gutierrez-Marques, P., Schroeder, S., Maue, T., Buettner, I., Richards, M., Chistensen, U., and Keller, U. (2011). DAWN FC2 calibrated Vesta images v1.0. NASA Planetary Data System. [cited by applicant]
Nesnas, I. A. D., Hockman, B. J., Bandopadhyay, S., Morrell, B. J., Lubey, D. P., Villa, J., Bayard, D. S., Osmundson, A., Jarvis, B., Bersani, M., and Bhaskaran, S. (2021). Autonomous exploration of small bodies toward… [cited by applicant]
Preusker, F., Scholten, F., Matz, K., Roatsch, T., Jaumann, R., Raymond, C., and Russell, C. (2016). DAWN FC2 Derived Vesta DTM SPG v1. 0. NASA Planetary Data System. [cited by applicant]
Prockter, L., Murchie, S., Cheng, A., Krimigis, S., Farquhar, R., Santo, A., and Trombka, J. (2002). The NEAR shoemaker mission to asteroid 433 Eros. Acta Astronautica, 51(1):491-500. [cited by applicant]
Rathinam, A. and Dempster, A. (2017). Monocular vision based simultaneous localization and mapping for close proximity navigation near an asteroid. In 68th IAF International Astronautical Congress, Adelaide, Australia. … [cited by applicant]
Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011). ORB: an efficient alternative to SIFT and SURF. In IEEE International Conference on Computer Vision, pp. 2564-2571, Barcelona, Spain. [cited by applicant]
Starek, J. A., A, ckme, se, B., Nesnas, I. A., and Pavone, M. (2016). Spacecraft Autonomy Challenges for Next-Generation Space Missions, pp. 34. Springer Berlin Heidelberg, Berlin, Heidelberg. [cited by applicant]
Villa, J., Mcmahon, J., Hockman, B., and Nesnas, I. (2022). Autonomous navigation and dense shape reconstruction using stereophotogrammetry at small celestial bodies. In 44th Annual AAS Guidance, Navigation, and Control… [cited by applicant]
Williams, B. (2002). Technical challenges and results for navigation of NEAR Shoemaker. Johns Hopkins APL Techincal Digest, 23(1):34-45. [cited by applicant]
Xie, M., Escontrela, A., and Dellaert, F. (2020). A Factor-Graph Approach for Optimization Problems with Dynamics Constraints. arXiv preprint arXiv:2011.06194. [cited by applicant]
Yan, X., Indelman, V., and Boots, B. (2017). Incremental sparse GP regression for continuous-time trajectory estimation and mapping. Robotics and Autonomous Systems, 87:120-132. [cited by applicant]
Yoshikawa, M., Kawaguchi, J., Fujiwara, A., and Tsuchiyama, A. (2015). HAYABUSA sample return mission. Asteroids IV, 1:397-418. [cited by applicant]