NEURAL NETWORK ARCHITECTURE METHOD FOR DEEP ODOMETRY ASSISTED BY STATIC SCENE OPTICAL FLOW
A method of visual odometry for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs includes instructions, which when executed by a computing device, causes the computing device to perform the following steps comprising: extracting representative features from a pair input images in a first convolution neural network (CNN) in a visual odometry model; merging, in a first merge module, outputs from the first CNN; decreasing feature map size in a second CNN; generating a first flow output for each layer in a first deconvolution neural network (DNN); merging, in a second merge module, outputs from the second CNN and the first DNN; generating a second flow output for each layer in a second DNN; and reducing accumulated errors in a recurrent neural network (RNN).
1 . A method of visual odometry for a non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, causes the computing device to perform the following steps comprising:
extracting representative features from a pair input images in a first convolution neural network (CNN) in a visual odometry model;
merging, in a first merge module, outputs from the first CNN;
decreasing feature map size in a second CNN;
generating a first flow output for each layer in a first deconvolution neural network (DNN);
merging, in a second merge module, outputs from the second CNN and the first DNN;
generating a second flow output for each layer in a second DNN; and
reducing accumulated errors in a recurrent neural network (RNN).
2 . The method according to claim 1 , wherein the outputs from the first CNN include the representative features of a first image of the pair and the representative features of a second image of the pair.
3 . The method according to claim 1 , wherein merging outputs from the first CNN comprises:
using one of a patch-wise correlation or a simple concatenation during merging.
4 . The method according to claim 1 , wherein the second CNN constitutes a first input of a motion estimate.
5 . The method according to claim 4 , wherein the first DNN constitutes a second input of the motion estimate.
6 . The method according to claim 4 further comprising:
generating a set of motion parameters in the RNN in response to the motion estimate and a set of motion parameters associated with an immediately previous pair of input images.
7 . The method according to claim 6 further comprising:
providing the set of motion parameters associated with the current pair of input images to the RNN.
8 . The method according to claim 6 further comprising:
training the visual odometry model by using the set of motion parameters.
9 . The method according to claim 1 , wherein the first flow output has a first resolution, and the second flow output has a second resolution higher than the first resolution.
10 . The method according to claim 9 further comprising:
training the visual odometry model by using the second flow output;
generating the motion parameters based on the motion estimate.