IP Library Granted Patent US 11,326,887
Granted Patent B2
US 11,326,887 · App. 16/890,171 · Granted May 10, 2022

Landmark placement for localization

Inventor: Dirk Holz (Mountain View, CA)
Assignee: X Development LLC
G01C21/265G01C21/28G01S5/16G01S5/163G01S7/4808G01S17/04G01S17/87G05D1/0044G05D1/0236G05D1/0274G05D1/0297G01C21/206G05D2201/0216
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,326,887
App. No.
16/890,171
Granted
May 10, 2022
Kind
B2
Abstract

Embodiments are provided that include receiving sensor data from a sensor positioned at a plurality of positions in an environment. The environment includes a plurality of landmarks. The embodiments also include determining, based on the sensor data, a subset of the plurality of landmarks detected at each of the plurality of positions. The embodiments further include determining, based on the subset of the plurality of landmarks detected at each of the plurality of positions, a detection frequency of each landmark. The embodiments additionally include determining, based on the determined detection frequency of each landmark, a localization viability metric associated with each landmark. The embodiments still further include providing for display, via a user interface, a map of the environment. The map includes an indication of the localization viability metric associated with each landmark.

Claims (58)

1. A method comprising:

moving a robot to a plurality of positions within an environment;

obtaining sensor data using at least one sensor on the robot at each of the plurality of positions;

detecting a plurality of landmarks based on the sensor data obtained at each of the plurality of positions, wherein each landmark of the plurality of landmarks corresponds to a respective region surrounding the landmark, and wherein two or more different landmarks of the plurality of landmarks have different respective regions;

determining a detection frequency for each landmark, wherein the detection frequency corresponds to a number of times that the landmark was detected within the respective region surrounding the landmark relative to a number of positions at which sensor data was obtained within the respective region; and

determining which of the plurality of landmarks are viable for localization of the robot based on determining the detection frequency for each landmark.

2. The method of claim 1 , further comprising:

determining, based on the determined detection frequency for each landmark, a localization viability metric associated with each landmark, wherein determining which of the plurality of landmarks are viable for localization comprises determining which of the plurality of landmarks are viable for localization based on the localization viability metric associated with each landmark.

3. The method of claim 2 , further comprising selecting a viability metric level for each landmark from a plurality of viability metric levels, wherein a first viability metric level indicates that the landmark is viable and a second viability metric level indicates that the landmark is not viable.

4. The method of claim 3 , wherein selecting the viability metric level for each landmark from a plurality of viability metric levels comprises determining, for each landmark, a range of detection frequencies within which the detection frequency associated with the landmark falls, and selecting the viability metric level based on which range of detection frequencies the detection frequency associated with the landmark falls.

5. The method of claim 1 , further comprising:

determining, using the plurality of landmarks, a pose of the robot at each position; and

determining a pose estimation confidence of the robot for each determined pose,

wherein determining which of the plurality of landmarks are viable for localization of the robot comprises determining which of the plurality of landmarks are viable for localization of the robot based on one or more pose estimation confidences associated with each landmark.

6. The method of claim 1 , further comprising:

determining a map of the environment while moving the robot to the plurality of positions; and

updating the map based on determining which of the plurality of landmarks are viable for localization of the robot.

7. The method of claim 1 , further comprising:

receiving additional sensor data from a sensor positioned at a second plurality of positions in the environment; and

updating, based on the additional sensor data, which of the plurality of landmarks are viable for localization of the robot.

8. The method of claim 1 , further comprising:

determining, based on which of the plurality of landmarks are viable for localization of the robot, a new landmark for use in localizing the robot.

9. The method of claim 1 , further comprising:

determining, based on which of the plurality of landmarks are viable for localization of the robot, one or more landmarks to adjust.

10. The method of claim 1 , further comprising:

determining, based on which of the plurality of landmarks are viable for localization of the robot, one or more landmarks to remove from the plurality of landmarks.

11. The method of claim 1 , wherein each landmark comprises a dedicated landmark placed at a fixed location in the environment.

12. A system comprising:

a sensor;

one or more processors;

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer readable medium and executable by the one or more processors to:

move a robot to a plurality of positions within an environment;

obtain sensor data using the sensor at each of the plurality of positions;

detect a plurality of landmarks based on the sensor data obtained at each of the plurality of positions, wherein each landmark of the plurality of landmarks corresponds to a respective region surrounding the landmark, and wherein two or more different landmarks of the plurality of landmarks have different respective regions;

determine a detection frequency for each landmark, wherein the detection frequency corresponds to a number of times that the landmark was detected within the respective region surrounding the landmark relative to a number of positions at which sensor data was obtained within the respective region; and

determine which of the plurality of landmarks are viable for localization of the robot based on determining the detection frequency for each landmark.

13. The system of claim 12 , wherein the program instructions are further executable by the one or more processors to:

determine, based on the determined detection frequency for each landmark, a localization viability metric associated with each landmark, wherein determining which of the plurality of landmarks are viable for localization comprises determining which of the plurality of landmarks are viable for localization based on the localization viability metric associated with each landmark.

14. The system of claim 13 , wherein the program instructions are further executable by the one or more processors to:

select a viability metric level for each landmark from a plurality of viability metric levels, wherein a first viability metric level indicates that the landmark is viable and a second viability metric level indicates that the landmark is not viable.

15. The system of claim 14 , wherein selecting the viability metric level for each landmark from a plurality of viability metric levels comprises determining, for each landmark, a range of detection frequencies within which the detection frequency associated with the landmark falls, and selecting the viability metric level based on which range of detection frequencies.

16. The system of claim 12 , wherein the program instructions are further executable by the one or more processors to:

determine, using the plurality of landmarks, a pose of the robot at each position; and

determine a pose estimation confidence of the robot for each determined pose,

wherein determining which of the plurality of landmarks are viable for localization of the robot comprises determining which of the plurality of landmarks are viable for localization of the robot based on one or more pose estimation confidences associated with each landmark.

17. The system of claim 12 , wherein the program instructions are further executable by the one or more processors to:

determining a map of the environment while moving the robot to the plurality of positions; and

update the map based on determining which of the plurality of landmarks are viable for localization of the robot.

18. A non-transitory computer readable medium having stored therein instructions executable by one or more processors to cause a computing system to perform functions comprising:

moving a robot to a plurality of positions within an environment;

obtaining sensor data using at least one sensor on the robot at each of the plurality of positions;

detecting a plurality of landmarks based on the sensor data obtained at each of the plurality of positions, wherein each landmark of the plurality of landmarks corresponds to a respective region surrounding the landmark, and wherein two or more different landmarks of the plurality of landmarks have different respective regions;

determining a detection frequency for each landmark, wherein the detection frequency corresponds to a number of times that the landmark was detected within the respective region surrounding the landmark relative to a number of positions at which sensor data was obtained within the respective region; and

determining which of the plurality of landmarks are viable for localization of the robot based on determining the detection frequency for each landmark.

19. The non-transitory computer readable medium of claim 18 , the functions further comprising:

determining, based on the determined detection frequency for each landmark, a localization viability metric associated with each landmark, wherein determining which of the plurality of landmarks are viable for localization comprises determining which of the plurality of landmarks are viable for localization based on the localization viability metric associated with each landmark.

20. The non-transitory computer readable medium of claim 18 , the functions further comprising selecting a viability metric level for each landmark from a plurality of viability metric levels, wherein a first viability metric level indicates that the landmark is viable and a second viability metric level indicates that the landmark is not viable.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2025
From: GOOGLE LLC
To: GDM HOLDING LLC
Reel/Frame 071109/0342 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2023
From: X DEVELOPMENT LLC
To: GOOGLE LLC
Reel/Frame 064658/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 11, 2021
From: HOLZ, DIRK
To: X DEVELOPMENT LLC
Reel/Frame 058089/0085 →