Learning good features for visual odometry
A computer-implemented method, system, and computer program product are provided for pose estimation. The method includes receiving, by a processor, a plurality of images from one or more cameras. The method also includes generating, by the processor with a feature extraction convolutional neural network (CNN), a feature map for each of the plurality of images. The method additionally includes estimating, by the processor with a feature weighting network, a score map from a pair of the feature maps. The method further includes predicting, by the processor with a pose estimation CNN, a pose from the score map and a combined feature map. The method also includes controlling an operation of a processor-based machine to change a state of the processor-based machine, responsive to the pose.
1. A computer-implemented method for pose estimation, the method comprising:
receiving, by a processor, a plurality of images from one or more cameras;
generating, by the processor with a feature extraction convolutional neural network (CNN), a feature map for each of the plurality of images;
estimating, by the processor with a feature weighting network, a score map from a pair of the feature maps;
predicting, by the processor with a pose estimation CNN, a pose from the score map and a combined feature map; and
controlling an operation of a processor-based machine to change a state of the processor-based machine, responsive to the pose.
2. The computer-implemented method as recited in claim 1 , wherein the pair of the feature maps are combined prior to entering the feature weighting network.
3. The computer-implemented method as recited in claim 1 , wherein the score map is incorporated directly into the combined feature map before entering to the pose estimation CNN.
4. The computer-implemented method as recited in claim 1 , wherein the pose estimation CNN includes training utilizing two-dimensional (2D) keypoint displacement loss employing points.
5. The computer-implemented method as recited in claim 1 , wherein the pose estimation CNN includes training utilizing three-dimensional (3D) pose regression loss.
6. The computer-implemented method as recited in claim 1 , wherein the combined feature map includes training utilizing motion loss.
7. The computer-implemented method as recited in claim 6 , wherein the motion loss is determined with an optical flow CNN.
8. The computer-implemented method as recited in claim 1 , wherein the combined feature map includes training utilizing semantics loss.
9. The computer-implemented method as recited in claim 8 , wherein the semantics loss is determined with a semantic segmentation CNN.
10. A computer program product for pose estimation, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
receiving, by a processor, a plurality of images from one or more cameras;
generating, by the processor with a feature extraction convolutional neural network (CNN), a feature map for each of the plurality of images;
estimating, by the processor with a feature weighting network, a score map from a pair of the feature maps;
predicting, by the processor with a pose estimation CNN, a pose from the score map and a combined feature map; and
controlling an operation of a processor-based machine to change a state of the processor-based machine, responsive to the pose.
11. The computer program product as recited in claim 10 , wherein the pair of the feature maps are combined prior to entering the feature weighting network.
12. The computer-implemented method as recited in claim 10 , wherein the score map is incorporated directly into the combined feature map before entering to the pose estimation CNN.
13. The computer-implemented method as recited in claim 10 , wherein the pose estimation CNN includes training utilizing two-dimensional (2D) keypoint displacement loss employing points.
14. The computer-implemented method as recited in claim 10 , wherein the pose estimation CNN includes training utilizing three-dimensional (3D) pose regression loss.
15. The computer-implemented method as recited in claim 10 , wherein the combined feature map includes training utilizing motion loss.
16. The computer-implemented method as recited in claim 15 , wherein the motion loss is determined with an optical flow CNN.
17. The computer-implemented method as recited in claim 10 , wherein the combined feature map includes training utilizing semantics loss.
18. The computer-implemented method as recited in claim 17 , wherein the semantics loss is determined with a semantic segmentation CNN.
19. A pose estimation system, the pose estimation system comprising:
a processing system including a processor and memory coupled to the processor, the processing system programmed to:
receive a plurality of images from one or more cameras;
generate, with a feature extraction convolutional neural network (CNN), a feature map for each of the plurality of images;
estimate, with a feature weighting network, a score map from a pair of the feature maps;
predict, with a pose estimation CNN, a pose from the score map and a combined feature map; and
control an operation of a processor-based machine to change a state of the processor-based machine, responsive to the pose.
20. The system as recited in claim 19 , wherein the score map is incorporated directly into the combined feature map prior to entering to the pose estimation CNN.