Loop closure using multi-modal sensor data

1 . A method comprising:

obtaining a sample Light Detection and Ranging (LIDAR) profile generated by a drone;

selecting a reference position based on the sample LIDAR profile;

determining a LIDAR profile-based translation and rotation relative to a reference LIDAR profile of the reference position to substantially align the obtained sample LIDAR profile with the reference LIDAR profile;

determining an image-based translation and rotation relative to a reference image of the reference position to substantially align a sample image generated by the drone with the reference image;

determining whether the LIDAR profile-based translation and rotation and the image-based translation and rotation satisfy a similarity threshold;

verifying, using a result of the determination, a predicted position of the drone; and

controlling navigation of the drone using a result of the verification.

2 . The method of claim 1 , wherein obtaining the sample LIDAR profile generated by the drone comprises:

obtaining a point cloud assembled by measurements taken by a LIDAR system of the drone over a period of time.

3 . The method of claim 1 , wherein selecting the reference position based on the sample LIDAR profile comprises:

obtaining one or more reference LIDAR profiles;

identifying the reference LIDAR profile as a profile of the one or more reference LIDAR profiles that satisfies a similarity criteria with the sample LIDAR profile; and

selecting the reference position as a position of the identified reference LIDAR profile.

4 . The method of claim 3 , wherein identifying the reference LIDAR profile of the one or more reference LIDAR profiles that satisfies the similarity criteria with the sample LIDAR profile uses one or more operations of an iterative-closest point (ICP) algorithm.

5 . The method of claim 3 , wherein selecting the reference position as the position of the identified reference LIDAR profile comprises:

selecting the reference position as the position of the identified reference LIDAR profile from one or more positions represented by the identified reference LIDAR profile.

6 . The method of claim 3 , comprising:

generating multiple similarity indexes by comparing the sample LIDAR profile to at least one of the one or more reference LIDAR profiles;

wherein identifying the reference LIDAR profile as the profile of the one or more reference LIDAR profiles that satisfies the similarity criteria with the sample LIDAR profile comprises:

identifying, using the multiple similarity indexes, the most similar similarity index of the multiple similarity indexes that satisfies the similarity criteria; and

selecting the reference LIDAR as the profile associated with the identified most similar similarity index of the multiple similarity indexes.

7 . The method of claim 1 , wherein determining the LIDAR profile-based translation and rotation relative to the reference LIDAR profile of the reference position comprises:

determining one or more values representing a difference between the reference LIDAR profile and the sample LIDAR profile; and

determining the LIDAR profile-based translation and rotation using the difference.

8 . The method of claim 7 , wherein the one or more values representing the difference between the reference LIDAR profile and the sample LIDAR profile include a rotation matrix and translation vector.

9 . The method of claim 1 , wherein determining the image-based translation and rotation relative to the reference image of the reference position comprises:

obtaining the sample image generated by the drone;

determining one or more values representing a difference between the reference image and the sample image; and

determining the image-based translation and rotation using the difference.

10 . The method of claim 9 , wherein the one or more values representing the difference between the reference image and the sample image include a rotation matrix and translation vector.

11 . The method of claim 1 , wherein determining whether the LIDAR profile-based translation and rotation and the image-based translation and rotation satisfy the similarity threshold comprises:

generating one or more values representing a cosine similarity using one or more values representing the LIDAR profile-based translation and rotation and the image-based translation and rotation; and

comparing the one or more values representing the cosine similarity to a threshold cosine similarity.

12 . The method of claim 1 , comprising:

determining that the LIDAR profile-based translation and rotation and the image-based translation and rotation satisfy the similarity threshold; and

verifying, using the determination, the predicted position of the drone.

13 . The method of claim 12 , wherein the predicted position is predicted by the drone using visual inertial odometry (VIO).

14 . The method of claim 1 , wherein selecting the reference position comprises:

determining a position of the drone; and

selecting the reference position as a position within a threshold distance from the determined position of the drone.

15 . A non-transitory computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising:

obtaining a sample Light Detection and Ranging (LIDAR) profile generated by a drone;

selecting, from a plurality of candidate reference positions, a reference position with a reference LIDAR profile and a reference image using the sample LIDAR profile;

determining a LIDAR profile-based translation and rotation relative to the reference LIDAR profile of the reference position to substantially align the sample LIDAR profile with the reference LIDAR profile;

determining an image-based translation and rotation relative to the reference image of the reference position to substantially align a sample image generated by the drone with the reference image;

determining whether the LIDAR profile-based translation and rotation and the image-based translation and rotation satisfy a similarity threshold;

verifying, using a result of the determination, a predicted position of the drone; and

controlling navigation of the drone using a result of the verification.

16 . The medium of claim 15 , wherein obtaining the sample LIDAR profile generated by the drone comprises:

obtaining a point cloud assembled by measurements taken by a LIDAR system of the drone over a period of time.

17 . The medium of claim 15 , wherein selecting the reference position based on the sample LIDAR profile comprises:

obtaining one or more reference LIDAR profiles;

identifying the reference LIDAR profile as a profile of the one or more reference LIDAR profiles that satisfies a similarity criteria with the sample LIDAR profile; and

selecting the reference position as a position of the identified reference LIDAR profile.

18 . The medium of claim 17 , wherein identifying the reference LIDAR profile of the one or more reference LIDAR profiles that satisfies the similarity criteria with the sample LIDAR profile uses one or more operations of an iterative-closest point (ICP) algorithm.

19 . The medium of claim 17 , wherein selecting the reference position as the position of the identified reference LIDAR profile comprises:

selecting the reference position as the position of the identified reference LIDAR profile from one or more positions represented by the identified reference LIDAR profile.

20 . A system comprising:

one or more processors; and

machine-readable media interoperably coupled with the one or more processors and storing one or more instructions that, when executed by the one or more processors, perform operations comprising:

obtaining a sample Light Detection and Ranging (LIDAR) profile generated by a drone;

selecting, from a plurality of candidate reference positions, a reference position with a reference LIDAR profile and a reference image using the sample LIDAR profile;

determining a LIDAR profile-based translation and rotation relative to the reference LIDAR profile of the reference position to substantially align the sample LIDAR profile with the reference LIDAR profile;

determining an image-based translation and rotation relative to the reference image of the reference position to substantially align a sample image generated by the drone with the reference image;

determining whether the LIDAR profile-based translation and rotation and the image-based translation and rotation satisfy a similarity threshold;

verifying, using a result of the determination, a predicted position of the drone; and

controlling navigation of the drone using a result of the verification.