Systems and methods for using multiple hypotheses in a visual simultaneous localization and mapping system
The invention is related to methods and apparatus that use a visual sensor and dead reckoning sensors to process Simultaneous Localization and Mapping (SLAM). These techniques can be used in robot navigation. Advantageously, such visual techniques can be used to autonomously generate and update a map. Unlike with laser rangefinders, the visual techniques are economically practical in a wide range of applications and can be used in relatively dynamic environments, such as environments in which people move. One embodiment further advantageously uses multiple particles to maintain multiple hypotheses with respect to localization and mapping. Further advantageously, one embodiment maintains the particles in a relatively computationally-efficient manner, thereby permitting the SLAM processes to be performed in software using relatively inexpensive microprocessor-based computer systems.
1. A method for efficiently managing content of a landmark database in a visual simultaneous localization and mapping system, the method comprising:
autonomously navigating a mobile device using one or more maps of the visual simultaneous localization and mapping system, and while autonomously navigating the mobile device, performing the following:
creating new landmarks from images captured by a camera mounted on the mobile device;
determining a density of the created landmarks in the one or more maps, wherein a landmark is associated with a pose of the mobile device that is determined when the landmark is created or updated when re-observed, wherein the pose corresponds to a position and orientation of the mobile device; and
discontinuing the adding of new landmarks to the one or more maps at least partially in response to a determination that the density of landmarks is higher than a threshold such that memory can be freed for other tasks;
wherein autonomously navigating, creating, determining and discontinuing are performed under the control of electronic hardware.
2. The method as defined in claim 1 wherein discontinuing the adding of new landmarks further comprises discontinuing the adding of landmarks in only selected high-density parts of the one or more maps.
3. The method as defined in claim 1 wherein discontinuing the adding of new landmarks further comprises discontinuing the adding of landmarks in all parts of the one or more maps.
4. The method as defined in claim 1 wherein the predetermined threshold adapts to an operating environment.
5. The method as defined in claim 1 wherein the predetermined threshold is between about 5 to 10 landmarks per square meter of mapped area.
6. A circuit for managing content of a landmark database in a visual simultaneous localization and mapping system, the circuit comprising:
a means for autonomously navigating a mobile device using one or more maps of the visual simultaneous localization and mapping system;
a means for creating new landmarks from images captured by a camera mounted on the mobile device;
a means for determining a density of the created landmarks in the one or more maps, wherein a landmark is associated with a pose of the mobile device that is determined when the landmark is created or updated when re-observed, wherein the pose corresponds to a position and orientation of the mobile device; and
a means for discontinuing the adding of new landmarks to the one or more maps at least partially in response to a determination that the density of landmarks is higher than a threshold such that memory can be freed for other tasks.
7. The circuit as defined in claim 6 wherein the means for discontinuing the adding of new landmarks is further configured to discontinue adding landmarks in only selected high-density parts of the one or more maps.
8. The circuit as defined in claim 6 wherein the means for discontinuing is further configured to adapt the threshold to an operating environment.
9. The circuit as defined in claim 6 wherein the circuit is embodied in a robot for navigation of the robot.
10. An apparatus comprising:
a processor; and
a computer program executable by the processor, the computer program embodied in a tangible non-transitory computer readable medium for efficiently managing content of a landmark database in a visual simultaneous localization and mapping system, the computer program comprising:
a module with instructions configured to autonomously navigate a mobile device using one or more maps of the visual simultaneous localization and mapping system;
a module with instructions configured to create new landmarks from images captured by a camera mounted on the mobile device;
a module with instructions configured to determine a density of the created landmarks in the one or more maps, wherein a landmark is associated with a pose of the mobile device that is determined when the landmark is created or updated when re-observed, wherein the pose corresponds to a position and orientation of the mobile device; and
a module with instructions configured to discontinue the adding of new landmarks to the one or more maps at least partially in response to a determination that the density of landmarks is higher than a threshold such that memory can be freed for other tasks.
11. The apparatus as defined in claim 10 , wherein the module with instructions configured to discontinue the adding of new landmarks further comprises instructions configured to discontinue the adding of landmarks in only selected high-density parts of the one or more maps.
12. The apparatus as defined in claim 10 , wherein the module with instructions configured to discontinue the adding of new landmarks further comprises instructions configured to adapt the threshold to an operating environment.
13. A circuit adapted to efficiently manage content of a landmark database in a visual simultaneous localization and mapping system, the circuit comprising:
a circuit configured to autonomously navigate a mobile device using one or more maps of the visual simultaneous localization and mapping system;
a circuit configured to create new landmarks from images captured by a camera mounted on the mobile device;
a circuit configured to determine a density of landmarks in the one or more maps, wherein a landmark is associated with a pose of the mobile device that is determined when the landmark is created or updated when re-observed, wherein the pose corresponds to a position and orientation of the mobile device; and
a circuit configured to discontinue the adding of new landmarks to the one or more maps at least partially in response to a determination that the density of landmarks is higher than a threshold such that memory can be freed for other tasks.
14. The circuit as defined in claim 13 , wherein the circuit configured to discontinue the adding of new landmarks is further configured to discontinue the adding of landmarks in only selected high-density parts of the one or more maps.
15. The circuit as defined in claim 13 , wherein the circuit configured to discontinue the adding of new landmarks is further configured to adapt the threshold to an operating environment.
16. The circuit as defined in claim 13 , wherein the circuit is embodied in a robot for navigation of the robot.
17. The method of claim 1 , wherein a landmark comprises a plurality of visual features with their corresponding 3D coordinates.
18. The circuit of claim 6 , wherein a landmark comprises a plurality of visual features with their corresponding 3D coordinates.
19. The apparatus of claim 10 , wherein a landmark comprises a plurality of visual features with their corresponding 3D coordinates.
20. The circuit of claim 13 , wherein a landmark comprises a plurality of visual features with their corresponding 3D coordinates.