Training and testing of a neural network method for deep odometry assisted by static scene optical flow

1. A method of visual odometry for a non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, causes the computing device to perform the following steps comprising:

in response to images in pairs, generating a prediction of static scene optical flow for each pair of the images in a visual odometry model;

generating a set of motion parameters for each pair of the images in the visual odometry model;

training the visual odometry model by using the prediction of static scene optical flow and the motion parameters; and

predicting motion between a pair of consecutive image frames by the trained visual odometry model;

extracting representative features from a first image of a pair in a first convolution neural network (CNN);

extracting representative features from a second image of the pair in the first CNN;

merging, in a first merge module, outputs from the first CNN;

decreasing feature map size in a second CNN;

generating a first flow output for each layer in a first deconvolution neural network (DNN);

merging, in a second merge module, outputs from the second CNN and the first DNN, and generating a first motion estimate; and

generating a second flow output for each layer in a second DNN, the second flow output serving as a first optical flow prediction.

2. The method according to claim 1 further comprising:

generating a set of motion parameters associated with the pair in a recurrent neural network (RNN).

3. The method according to claim 2 further comprising:

training the visual odometry model by using at least one of the first optical flow prediction and the first set of motion parameters.

4. The method according to claim 1 further comprising:

entering the visual odometry model to a test mode.

5. The method according to claim 4 further comprising:

receiving another pair of consecutive image frames; and

providing the first set of motion parameters to the RNN.

6. A system for visual odometry, the system comprising:

an internet server, comprising:

an I/O port, configured to transmit and receive electrical signals to and from a client device;

a memory;

one or more processing units; and

one or more programs stored in the memory and configured for execution by the one or more processing units, the one or more programs including instructions for:

in response to images in pairs, generating a prediction of static scene optical flow for each pair of the images in a visual odometry model;

generating a set of motion parameters for each pair of the images in the visual odometry model;

training the visual odometry model by using the prediction of static scene optical flow and the motion parameters;

predicting motion between a pair of consecutive image frames by the trained visual odometry model;

extracting representative features from a first image of a pair in a first convolution neural network (CNN); and

extracting representative features from a second image of the pair in the first CNN;

merging, in a first merge module, outputs from the first CNN;

decreasing feature map size in a second CNN;

generating a first flow output for each layer in a first deconvolution neural network (DNN);

merging, in a second merge module, outputs from the second CNN and the first DNN, and generating a first motion estimate; and

generating a second flow output for each layer in a second DNN, the second flow output serving as a first optical flow prediction.

7. The system according to claim 6 further comprising:

generating a set of motion parameters associated with the pair in a recurrent neural network (RNN).

8. The system according to claim 7 further comprising:

training the visual odometry model by using at least one of the first optical flow prediction and the first set of motion parameters.

9. The system according to claim 6 further comprising:

entering the visual odometry model to a test mode.

10. The system according to claim 9 further comprising:

receiving another pair of consecutive image frames; and

providing the first set of motion parameters to the RNN.