Camera-only-localization in sparse 3D mapped environments
Techniques for localizing a vehicle include obtaining an image from a camera, identifying a set of image feature points in the image, obtaining an approximate location of the vehicle, determining a set of sub-volumes (SVs) of a map to access based on the approximate location, obtaining map feature points and associated map feature descriptors associated with the set of SVs, determining a set of candidate matches between the set of image feature points and the obtained map feature points, determining a set of potential poses of the camera from candidate matches from the set of candidate matches and an associated reprojection error estimated for remaining points to select a first pose of the set of potential poses having a lowest associated reprojection error, determining the first pose is within a threshold value of an expected vehicle location, and outputting a vehicle location based on the first pose.
1. A method comprising:
accessing sensor data associated with an object;
determining, based on the sensor data, a set of feature points;
determining an approximate location for the object;
determining, based on the approximate location and a map header, a subset of a map, wherein the subset of the map specifies a set of map feature points within a volume of area, and wherein the map header specifies, for each map feature point of the set of map feature points, a number of feature descriptors in a set of feature descriptors; and
comparing the set of feature points to the set of map feature points to determine a pose of the object.
2. The method of claim 1 , wherein:
the set of map feature points is a first set of map feature points;
the map includes a second set of map feature points that includes the first set of map feature points; and
the map header specifies a start location of the first set of map feature points within the second set of map feature points.
3. The method of claim 2 , wherein the map header further specifies a number of map feature points within the first set of map feature points.
4. The method of claim 1 , wherein:
the determining of the subset of the map and the comparing of the set of feature points to the set of map feature points is performed by a first device;
the map is stored in a second device; and
the method further comprises retrieving, by the first device, the subset of the map from the second device.
5. The method of claim 4 , wherein the retrieving of the subset of the map is performed over a network.
6. The method of claim 1 , wherein the map header specifies a size of the volume of area associated with the subset of the map.
7. The method of claim 1 , wherein the map specifies, for each map feature point of the set of map feature points, a location of the respective map feature point in three dimensions.
8. The method of claim 1 , wherein the map specifies, for each map feature point of the set of map feature points, the set of feature descriptors.
9. The method of claim 1 , wherein the set of feature descriptors includes at least one of a chroma value, a luma value, or a grayscale value.
10. The method of claim 1 , wherein:
the set of sensor data includes an image; and
the set of feature points includes image feature points of the image.
11. A device comprising:
one or more processors; and
a non-transitory memory coupled to the one or more processors and storing instructions, that when executed, cause the one or more processors to:
receive sensor data that includes a set of feature points;
determine an approximate location of an object;
determine, based on the approximate location and a map header, a subset of a map that specifies a set of map feature points, wherein the map specifies, for each map feature point of the set of map feature points, a set of feature descriptors including at least one of a chroma value, a luma value, or a grayscale value; and
compare the set of feature points to the set of map feature points to determine a pose of the object.
12. The device of claim 11 , wherein:
the set of map feature points is a first set of map feature points;
the map includes a second set of map feature points that includes the first set of map feature points; and
the map header specifies a start location of the first set of map feature points within the second set of map feature points.
13. The device of claim 12 , wherein the map header further specifies a number of map feature points within the first set of map feature points.
14. The device of claim 11 , wherein:
the device is a first device;
the map is stored in a second device; and
the instructions further cause the one or more processors to retrieve the subset of the map from the second device.
15. The device of claim 14 , wherein the subset of the map is retrieved from the second device over a network.
16. The device of claim 11 , wherein the subset of the map is associated with a volume of area and the map header specifies a size of the volume of area.
17. The device of claim 11 , wherein the map specifies, for each map feature point of the set of map feature points, a location of the respective map feature point in three dimensions.
18. A method comprising:
receiving an image associated with an object;
determining, based on the image, a set of feature points;
determining an approximate location for the object;
determining, based on the approximate location and a map header, a set of map feature points within a volume of area, and wherein the map specifies, for each map feature point of the set of map feature points, a set of feature descriptors including at least one of a chroma value, a luma value, or a grayscale value; and
determining a pose of the object based on the set of feature points and the set of map feature points.
19. The method of claim 18 , wherein the map specifies a size of the volume of area.