IP Library Granted Patent US 12,263,597
Granted Patent B2
US 12,263,597 · App. 18/316,485 · Granted Apr 1, 2025

Robot localization using variance sampling

Inventor: Matthew Jacob Klingensmith (Somerville, MA)
Assignee: Boston Dynamics, Inc.
B25J9/1664B25J13/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,263,597
App. No.
18/316,485
Granted
Apr 1, 2025
Kind
B2
Abstract

A method of localizing a robot includes receiving odometry information plotting locations of the robot and sensor data of the environment about the robot. The method also includes obtaining a series of odometry information members, each including a respective odometry measurement at a respective time. The method also includes obtaining a series of sensor data members, each including a respective sensor measurement at the respective time. The method also includes, for each sensor data member of the series of sensor data members, (i) determining a localization of the robot at the respective time based on the respective sensor data, and (ii) determining an offset of the localization relative to the odometry measurement at the respective time. The method also includes determining whether a variance of the offsets determined for the localizations exceeds a threshold variance. When the variance among the offsets exceeds the threshold variance, a signal is generated.

Claims (38)

1. A method of robot localization, the method comprising:

receiving, at data processing hardware of a robot, odometry information indicating a position of the robot within an environment at each of a plurality of times;

receiving, at the data processing hardware, sensor data of the environment from at least one sensor of the robot, the sensor data indicating a localization of the robot for each of the plurality of times;

determining, by the data processing hardware, an offset of the localization of the robot relative to the odometry information for each of the plurality of times, and a variance of the offset;

adjusting, by the data processing hardware, relative weights of the odometry information and the sensor data based on comparing the variance of the offset to a threshold variance; and

controlling, by the data processing hardware, movement of the robot.

2. The method of claim 1 , further comprising:

determining, by the data processing hardware, a current location of the robot using the adjusted relative weights of the odometry information and the sensor data; and

further controlling, by the data processing hardware, movement of the robot based on the current location.

3. The method of claim 1 further comprising, when the variance of the offset exceeds the threshold variance, reducing, by the data processing hardware, a weight of the sensor data relative a weight of the odometry information.

4. The method of claim 3 further comprising, when the variance of the offset exceeds the threshold variance, generating an unreliable sensor data signal.

5. The method of claim 4 wherein the unreliable sensor data signal comprises a notification to an operator of the robot.

6. The method of claim 1 , further comprising, when the variance of the offset does not exceed the threshold variance, increasing, by the data processing hardware, a weight of the odometry information relative to a weight of the sensor data.

7. The method of claim 1 , wherein the sensor data comprises a reference sensor data member and a plurality of comparison sensor data members.

8. The method of claim 7 , further comprising determining, by the data processing hardware, the localization of the robot based on a pairwise analysis of the reference sensor data member to each of the plurality of comparison sensor data members.

9. The method of claim 1 , wherein the at least one sensor of the robot comprises a three-dimensional volumetric image sensor, and the sensor data comprises a three-dimensional volumetric point cloud.

10. The method of claim 1 , wherein the variance is a sample variance.

11. The method of claim 1 further comprising:

receiving, by the data processing hardware, the odometry information and the sensor data during the movement.

12. A robot comprising:

a body;

at least one sensor;

data processing hardware; and

memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:

receiving odometry information indicating a position of the robot within an environment at each of a plurality of times;

receiving sensor data of the environment from the at least one sensor, the sensor data indicating a localization of the robot for each of the plurality of times;

determining an offset of the localization of the robot relative to the odometry information for each of the plurality of times, and a variance of the offset; and

adjusting relative weights of the odometry information and the sensor data based on comparing the variance of the offset to a threshold variance.

13. The robot of claim 12 , wherein the operations further comprise:

determining a current location of the robot using the adjusted relative weights of the odometry information and the sensor data; and

controlling movement of the robot based on the current location.

14. The robot of claim 12 , wherein the operations further comprise reducing a weight of the sensor data relative a weight of the odometry information.

15. The robot of claim 14 , wherein the operations further comprise generating an unreliable sensor data signal when the variance of the offset exceeds the threshold variance.

16. The robot of claim 12 , wherein the operations further comprise increasing a weight of the odometry information relative to a weight of the sensor data when the variance of the offset does not exceed the threshold variance.

17. The robot of claim 12 , wherein the sensor data comprises a reference sensor data member and a plurality of comparison sensor data members.

18. The robot of claim 17 , wherein the operations further comprise determining the localization of the robot based on a pairwise analysis of the reference sensor data member to each of the plurality of comparison sensor data members.

19. The robot of claim 12 , wherein the at least one sensor of the robot comprises a three-dimensional volumetric image sensor, and the sensor data comprises a three-dimensional volumetric point cloud.

20. The robot of claim 12 , wherein the variance is a sample variance.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 4, 2026
From: KLINGENSMITH, MATTHEW JACOB
To: BOSTON DYNAMICS, INC.
Reel/Frame 073688/0106 →
Continuity (3)
Continuation 16893608 · Jun 5, 2020
Provisional Application 63013634 · Apr 22, 2020
Related Publication 20230278214A1 · Sep 7, 2023
References Cited (63)
US 7177737B2 · Karlsson et al. · 2007 [cited by applicant]
US 7774158B2 · Domingues Goncalves et al. · 2010 [cited by applicant]
US 8020657B2 · Allard et al. · 2011 [cited by applicant]
US 8577517B2 · Phillips et al. · 2013 [cited by applicant]
US 8781629B2 · Ota · 2014 [cited by applicant]
US 8873831B2 · Ahn et al. · 2014 [cited by applicant]
US 9020637B2 · Schnittman · 2015 [cited by applicant]
US 9037396B2 · Pack et al. · 2015 [cited by applicant]
US 9250078B2 · Kaiser et al. · 2016 [cited by applicant]
US 9329598B2 · Pack et al. · 2016 [cited by applicant]
US 9360300B2 · DiBernado et al. · 2016 [cited by applicant]
US 9519289B2 · Munich et al. · 2016 [cited by applicant]
US 9534899B2 · Gutmann · 2017 [cited by examiner]
US 9630319B2 · Vicenti · 2017 [cited by applicant]
US 9744670B2 · Romanov et al. · 2017 [cited by applicant]
US 9751210B2 · Fong et al. · 2017 [cited by applicant]
US 9770823B2 · Huang et al. · 2017 [cited by applicant]
US 9870624B1 · Narang et al. · 2018 [cited by applicant]
US 9902069B2 · Farlow et al. · 2018 [cited by applicant]
US 9969089B2 · Goel et al. · 2018 [cited by applicant]
US RE47108E · Jacobus et al. · 2018 [cited by applicant]
US 10096129B2 · Narang et al. · 2018 [cited by applicant]
US 10353399B2 · Ebrahimi Afrouzi · 2019 [cited by applicant]
US 10386850B2 · Heinla et al. · 2019 [cited by applicant]
US 10391630B2 · Fong et al. · 2019 [cited by applicant]
US 10611023B2 · Fong et al. · 2020 [cited by applicant]
US 10890921B2 · Gillett · 2021 [cited by applicant]
US 11348269B1 · Ebrahimi Afrouzi · 2022 [cited by examiner]
US 11521332B1 · Chen · 2022 [cited by examiner]
US 11685049B2 · Klingensmith · 2023 [cited by applicant]
US 12059814B2 · Bollini et al. · 2024 [cited by applicant]
US 20100076599A1 · Jacobus et al. · 2010 [cited by applicant]
US 20130138247A1 · Gutmann et al. · 2013 [cited by applicant]
US 20150212521A1 · Pack · 2015 [cited by examiner]
US 20180231981A1 · Storfer et al. · 2018 [cited by applicant]
US 20190110652A1 · Moroniti et al. · 2019 [cited by applicant]
US 20210132625A1 · Gillett · 2021 [cited by applicant]
US 20210283783A1 · Gillett · 2021 [cited by applicant]
US 20210365029A1 · Molina Cabrera et al. · 2021 [cited by applicant]
US 20220026920A1 · Afrouzi et al. · 2022 [cited by applicant]
US 20220187841A1 · Afrouzi et al. · 2022 [cited by applicant]
US 20220241980A1 · Bollini et al. · 2022 [cited by applicant]
US 20240189989A1 · Di Carlo et al. · 2024 [cited by applicant]
US 20240192690A1 · Ebrahimi Afrouzi · 2024 [cited by examiner]
US 20240351217A1 · Bollini et al. · 2024 [cited by applicant]
CN 116802021 · 2023 [cited by applicant]
DE 102018101388A1 · 2019 [cited by applicant]
EP 4284602A1 · 2023 [cited by applicant]
JP 2001062766A · 2001 [cited by applicant]
JP 2007196300A · 2007 [cited by applicant]
KR 20230138487A · 2023 [cited by applicant]
WO WO2019183727A1 · 2019 [cited by applicant]
WO WO2021216263A1 · 2021 [cited by applicant]
WO WO2022164793A1 · 2022 [cited by applicant]
International Search Report, Application No. PCT/US2021/025294, Jun. 14, 2021, 17 pages. [cited by applicant]
Winkler et al., “Path planning with force-based foothold adaptation and virtual model control for torque controlled quadruped robots,” May 31, 2014, 7 pages. [cited by applicant]
“Advanced Skills by Learning Locomotion and Local Navigation End-to-End,” video screen shots takenfrom https://www.youtube.com/watch?v=Xoe8a_2t24U, Sep. 27, 2022, downloaded Mar. 13, 2024, 24 pages. [cited by applicant]
“Advanced Skills by Learning Locomotion and Local Navigation End-to-End,” downloaded Mar. 13, 2024, https://sites.google.com/leggedrobotics.com/end-to-end-loco-navigation, 8 pages. [cited by applicant]
Martins et al., “Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework Using Visual and Depth Cues.” J Intell Robot Syst. Sep. 2020;99(3):555-69. [cited by applicant]
Rudin, et al., “Advanced Skills by Learning Locomotion and Local Navigation End-to-End.” 2022 IEEE/RSJ Inter'l Con Intell Robots & Systems (IROS), Sep. 26, 2022, pp. 2497-2503; doi: 10.1109/IROS47612.2022.9981198. [cited by applicant]
Sidaoui et al., “Human-in-the-loop Augmented Mapping.” 2018 IEEE/RSJ Inter'l Conf Intelligent Robots & Systems (IROS), Oct. 1, 2018 (pp. 3190-3195). [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2022/013684, mailed May 6, 2022; 12 pages. [cited by applicant]
Third Party Submission Document received in U.S. Appl. No. 17/648,869, 60 pages, dated Aug. 10, 2022. [cited by applicant]
Cited By (1)
US 12,673,417