Method and device of mapping and localization method using the same
A mapping method is provided. The environment is scanned to obtain depth information of environmental obstacles. The image of the environment is captured to generate an image plane. The depth information of environmental obstacles is projected onto the image plane, so as to obtain projection positions. At least one feature vector is calculated from a predetermined range around each projection position. The environmental obstacle depth information and the environmental feature vector are merged to generate a sub-map at a certain time point. Sub-maps at all time points are combined to generate a map. In addition, a localization method using the map is also provided.
1. A mapping method, comprising:
scanning an environment to obtain depth information of environmental obstacles;
capturing an image of the environment to generate an image plane;
projecting the enviromnental obstacle depth information onto the image plane, so as to obtain a plurality of projection positions;
calculating at least one feature vector from a predetermined range around each of the projection positions in the image plane, wherein when the feature vector cannot be calculated from the predetermined range around each of the projection positions, at least an upper feature vector or at least a lower feature vector is calculated from upper or lower image data of each of the projection positions, wherein the upper feature vector or the lower feature vector is calculated according to upper or lower image data in the image plane relative to a vertical axis of each of the projection positions;
combining the depth information of environmental obstacles and each of the feature vectors to generate a sub-map for a certain time point; and
combining the sub-maps at all time points to generate a map.
2. The mapping method as claimed in claim 1 , wherein the feature vector is determined according to corner points, edges and regional extremum in the predetermined range of each of the projection positions on the image plane.
3. The mapping method as claimed in claim 1 , wherein the step of combining the depth information of environmental obstacles and each of the feature vectors comprises:
converting the depth information of environmental obstacles to a world coordinate system; and
combining each of the feature vectors with the depth information of environmental obstacles represented by the world coordinate system.
4. The mapping method as claimed in claim 1 , wherein the step of calculating the feature vectors according to the projection positions in the image plane further comprises abandoning the feature vectors with high similarities.
5. The mapping method as claimed in claim 1 , wherein the step of scanning the environment is performed with a laser range finder.
6. The mapping method as claimed in claim 1 , wherein the step of capturing the image of the environment is performed with a digital camera.
7. A localization method, comprising:
obtaining a map comprising the depth information of environmental obstacles and at least one feature vector of an image corresponding to the depth information of environmental obstacles, wherein the step of obtaining the map comprises:
scanning an environment to obtain the environmental obstacle depth information;
capturing an image of the environment to generate an image plane;
projecting the depth information of environmental obstacles onto the image plane, so as to obtain a plurality of projection positions;
calculating at least one feature vector from a predetermined range around each of the projection positions in the image plane, wherein when the feature vector cannot be calculated from the predetermined range around each of the projection positions, at least an upper feature vector or at least a lower feature vector is calculated from upper or lower image data of each of the projection positions, wherein the upper feature vector or the lower feature vector is calculated according to upper or lower image data in the image plane relative to a vertical axis of each of the projection positions;
combining the depth information of environmental obstacles and each of the feature vectors to generate a sub-map for a certain time point; and
combining the sub-maps at all time points to generate the map;
using a visual sensing device to obtain an environmental image;
extracting at least one image feature point from the environmental image; and
performing a localization according to the image feature point and the map.
8. The localization method as claimed in claim 7 , wherein the step of performing the localization according to the image feature point and the map is to calculate a world coordinates of the visual sensing device by using an image coordinates and a world coordinates of the image feature point of the environmental image, so as to implement the localization.
9. The localization method as claimed in claim 7 , wherein the feature vector is determined according to corner points, edges and regional extremum in the predetermined range of each of the projection positions on the image plane.
10. The localization method as claimed in claim 7 , wherein the step of combining the depth information of environmental obstacles and each of the feature vectors comprises:
converting the depth information of environmental obstacles to a world coordinate system; and
combining each of the feature vectors with the depth information of environmental obstacles represented by the world coordinate system.
11. A mapping device, comprising:
at least one range sensing device, for scanning an environment to generate the depth information of environmental obstacles;
at least one visual sensing device, for capturing an image of the environment to generate an image plane, wherein the range sensing device and the visual sensing device are integrated as an embedded handheld device; and
an image processing device, coupled to the range sensing device and the visual sensing device, for projecting the depth information of environmental obstacles onto the image plane to obtain a plurality of projection positions, calculating at least one feature vector from a predetermined range around each of the projection positions in the image plane, and combining the depth information of environmental obstacles and the feature vectors to generated a map,
wherein when the feature vector cannot be calculated from the predetermined range around each of the projection positions, the image processing device further calculates at least an upper feature vector or at least a lower feature vector from upper or lower image data of each of the projection positions, wherein the upper feature vector or the lower feature vector is calculated according to upper or lower image data in the image plane relative to a vertical axis of each of the projection positions.
12. The mapping device as claimed in claim 11 , wherein the range sensing device is a laser range finder.
13. The mapping device as claimed in claim 11 , wherein the visual sensing device is a digital camera.
14. The mapping device as claimed in claim 11 , wherein the range sensing device, the visual sensing device and the image processing device are deployed on a mobile platform.