Voting space-based loop closure detection
A method for detecting a loop closure is described. A device accesses pose information and a three-dimensional map of feature points generated by a visual inertia system of the device. The device splits the pose information into a translational part and a rotational part. The device limits the translational part to two-dimensional coordinates and estimates two-dimensional information of the limited translational part based on an accumulator voting space. The device determines an updated pose of the device based on the estimated two-dimensional information, the rotational part, and the three-dimensional map. The pose information is updated with the updated pose.
1. A method for detecting a loop closure, comprising:
accessing pose information of a device and a three-dimensional map of feature points generated by a visual inertia system of the device;
splitting the pose information into a translational part and a rotational part;
limiting the translational part to two-dimensional coordinates;
estimating two-dimensional information of the limited translational part based on an accumulator voting space;
determining an updated pose of the device based on the estimated two-dimensional information, the rotational part, and the three-dimensional map;
updating the pose information from the visual inertia system of the device with the updated pose;
accessing a current image from a camera of the device;
identifying feature points based on the current image;
generating a two-dimensional map of the feature points in the current image;
accessing a prior three-dimensional map of the feature points;
accessing prior pose information of the device corresponding to the prior three-dimensional map of the feature points;
forming all possible pairs of two-dimensional coordinates of the feature points with prior three-dimensional coordinates of the feature points, the two-dimensional coordinates of the feature points based on the two-dimensional map of the feature points, the prior three-dimensional coordinates of the feature points based on the prior three-dimensional map of the feature points;
filtering the pairs of the two-dimensional coordinates of the feature points with the prior three-dimensional coordinates of the feature points;
for each filtered pair, computing the two-dimensional information; and
identifying a dominant feature point based on the computed two-dimensional information,
wherein the determining of the updated pose of the device is based on the two-dimensional coordinates of the dominant feature point.
2. The method of claim 1 , wherein the translational part includes three-dimensional coordinates, wherein the limited translational part does not include a coordinate corresponding to a gravitational axis, wherein the rotational part includes a vertical axis, a transverse axis, and a longitudinal axis.
3. The method of claim 1 , wherein the three-dimensional map of the feature points does not include image descriptors.
4. The method of claim 1 , further comprising:
identifying a drift of the visual inertia system of the device based on the updated pose and the pose information; and
minimizing the drift by updating the visual inertia system of the device with the updated pose.
5. The method of claim 1 , further comprising:
capturing an image of a physical object with an image sensor of the device;
identifying the physical object based on the three-dimensional map;
retrieving a virtual object based on the identified physical object; and
displaying the virtual object in a display of the device.
6. The method of claim 5 , further comprising:
updating a location of the virtual object relative to the display of the device based on the updated pose of the device.
7. A computing apparatus, the computing apparatus comprising:
a visual inertia system comprising a camera and an inertia sensor;
a processor; and
a memory storing instructions that, when executed by the processor, configure the apparatus to:
access pose information of the computing apparatus and a three-dimensional map of feature points generated by the visual inertia system;
split the pose information into a translational part and a rotational part;
limit the translational part to two-dimensional coordinates;
estimate two-dimensional information of the limited translational part based on an accumulator voting space;
determine an updated pose of the computing apparatus based on the estimated two-dimensional information, the rotational part, and the three-dimensional map;
update the pose information from the visual inertia system of the computing apparatus with the updated pose;
access a current image from a camera of the device;
identify feature points based on the current image;
generate a two-dimensional map of the feature points in the current image;
access a prior three-dimensional map of the feature points;
access prior pose information of the device corresponding to the prior three-dimensional map of the feature points;
form all possible pairs of two-dimensional coordinates of the feature points with prior three-dimensional coordinates of the feature points, the two-dimensional coordinates of the feature points based on the two-dimensional map of the feature points, the prior three-dimensional coordinates of the feature points based on the prior three-dimensional may of the feature points;
filter the pairs of the two-dimensional coordinates of the feature points with the prior three-dimensional coordinates of the feature points;
for each filtered pair, compute the two-dimensional information; and
identify a dominant feature point based on the computed two-dimensional information,
wherein the determining of the updated pose of the device is based on the two-dimensional coordinates of the dominant feature point.
8. The computing apparatus of claim 7 , wherein the translational part includes three-dimensional coordinates, wherein the limited translational part does not include a coordinate corresponding to a gravitational axis, wherein the rotational part includes a vertical axis, a transverse axis, and a longitudinal axis.
9. The computing apparatus of claim 7 , wherein the three-dimensional map of the feature points does not include image descriptors.
10. The computing apparatus of claim 7 , wherein the instructions further configure the apparatus to:
identify a drift of the visual inertia system of the computing apparatus based on the updated pose and the pose information; and
minimize the drift by updating the visual inertia system of the computing apparatus with the updated pose.
11. The computing apparatus of claim 7 , wherein the instructions further configure the apparatus to:
capture an image of a physical object with the camera;
identify the physical object based on the three-dimensional map;
retrieve a virtual object based on the identified physical object; and
display the virtual object in a display of the computing apparatus.
12. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that, when executed by a computer, cause the computer to:
access pose information of a device and a three-dimensional map of feature points generated by a visual inertia system of the device;
split the pose information into a translational part and a rotational part;
limit the translational part to two-dimensional coordinates;
estimate two-dimensional information of the limited translational part based on an accumulator voting space;
determine an updated pose of the device based on the estimated two-dimensional information, the rotational part, and the three-dimensional map;
update the pose information from the visual inertia system of the device with the updated pose;
access a current image from a camera of the device;
identify feature points based on the current image;
generate a two-dimensional map of the feature points in the current image;
access a prior three-dimensional map of the feature points;
access prior pose information of the device corresponding to the prior three-dimensional map of the feature points;
form all possible pairs of two-dimensional coordinates of the feature points with prior three-dimensional coordinates of the feature points, the two-dimensional coordinates of the feature points based on the two-dimensional map of the feature points, the prior three-dimensional coordinates of the feature points based on the prior three-dimensional map of the feature points filter the pairs of the two-dimensional coordinates of the feature points with the prior three-dimensional coordinates of the feature points;
for each filtered pair, compute the two-dimensional information; and
identify a dominant feature point based on the computed two-dimensional information,
wherein the determining of the updated pose of the device is based on the two-dimensional coordinates of the dominant feature point.