System and method for estimating vehicular motion based on monocular video data
View Patent ↗A vehicle movement parameter, such as ego-speed, is estimated using real-time images captured by a single camera. The captured images may be analyzed by a pre-trained convolutional neural network to estimate vehicle movement based on monocular video data. The convolutional neural network may be pre-trained using filters from a synchrony autoencoder that were trained using unlabeled video data captured by the vehicle's camera while the vehicle was in motion. A parameter corresponding to the estimated vehicle movement may be output to the driver or to a driver assistance system for use in controlling the vehicle.
1. A system for estimating vehicular speed based on monocular video data by encoding spatio-temporal motion features, the system comprising:
a camera mounted to a vehicle and configured to capture monocular video data;
a memory configured to store data and processor-executable instructions;
a processor configured to execute the processor-executable instructions stored in the memory to:
receive, from the camera, pre-training video data captured while the vehicle is in motion,
train one or more filters of a synchrony autoencoder using the pre-training video data, and
pre-train a convolutional neural network using the trained one or more filters from the synchrony autoencoder,
following said pre-training of the convolutional neural network, the processor is further configured to
receive real-time video data from the camera while the vehicle is in motion,
provide the real-time video data to the pre-trained convolutional neural network,
receive an estimated vehicle movement parameter from the pre-trained convolutional neural network, and
output the estimated vehicle movement parameter to at least one of an audio/visual system and a driver assistance system of the vehicle wherein the driver assistance system is configured to perform one of a steering and an acceleration/deceleration operation at least partly based on the vehicle movement parameter.
2. The system of claim 1 , wherein the processor is further configured to pre-process the pre-training video data by ZCA-whitening prior to training the one or more filters of the synchrony autoencoder.
3. The system of claim 1 , wherein the processor is configured to pre-train the convolutional neural network by initializing a first layer of the convolutional neural network using the trained one or more filters from the synchrony autoencoder.
4. The system of claim 3 , wherein the convolutional neural network is further trained, in an iterative fashion, in which an output of the first layer is passed to a next layer of the convolutional neural network which, in turn passes a corresponding output to another layer of the convolutional neural network.
5. The system of claim 1 , wherein the processor is configured to duplicate a first channel of the pre-training video data as a second channel of pre-training video data, wherein the first and second channels of pre-training video data are provided to train the one or more filters of the synchrony autoencoder.
6. The system of claim 1 , wherein the vehicle movement parameter comprises an ego-speed of the vehicle.
7. A method for estimating vehicular speed based on monocular video data by encoding spatio-temporal motion features, the method comprising:
pre-training a convolutional neural network, wherein said pre-training comprises:
receiving, from a mono-camera mounted to a vehicle, pre-training video data captured while the vehicle was in motion;
training one or more filters of a synchrony autoencoder, stored in a vehicle memory, using the pre-training video data;
pre-training the convolutional neural network, stored in the vehicle memory, using the trained one or more filters from the synchrony autoencoder,
wherein following said pre-training of the convolutional neural network, the method further comprises
receiving real-time video data from the camera while the vehicle is in motion;
providing the real-time video data to the pre-trained convolutional neural network;
receiving an estimated vehicle movement parameter from the pre-trained convolutional neural network;
outputting the estimated vehicle movement parameter to at least one of an audio/visual system and a driver assistance system of the vehicle and performing, by a driver assistance system of the vehicle, one of a steering and an acceleration/deceleration operation at least partly based on the vehicle movement parameter.
8. The method of claim 7 , further comprising pre-processing the pre-training video data by ZCA-whitening prior to said training of the one or more filters of the synchrony autoencoder.
9. The method of claim 7 , wherein pre-training the convolutional neural network comprises pre-training the convolutional neural network by initializing a first layer of the convolutional neural network using the trained one or more filters from the synchrony autoencoder.
10. The method of claim 7 , further comprising duplicating a first channel of the pre-training video data as a second channel of pre-training video data, wherein the first and second channels of pre-training video data are provided to train the one or more filters of the synchrony autoencoder.
11. The method of claim 7 , wherein the vehicle movement parameter comprises an ego-speed of the vehicle.