Vehicle re-identification techniques using neural networks for image analysis, viewpoint-aware pattern recognition, and generation of multi- view vehicle representations
This disclosure relates to improved vehicle re-identification techniques. The techniques described herein utilize artificial intelligence (AI) and machine learning functions to re-identify vehicles across multiple cameras. Vehicle re-identification can be performed using an image of the vehicle that is captured from any single viewpoint. Attention maps may be generated that identify regions of the vehicle that include visual patterns that overlap between the viewpoint of the captured image and one or more additional viewpoints. The attention maps are used to generate a multi-view representation of the vehicle that provides a global view of the vehicle across multiple viewpoints. The multi-view representation of the vehicle can then be compared to previously captured image data to perform vehicle re-identification.
1. A system for re-identifying a vehicle comprising:
a camera system comprising a plurality of cameras;
one or more computing devices comprising one or more processors and one or more non-transitory storage devices for storing instructions, wherein execution of the instructions by the one or more processors causes the one or more computing device to:
receive an image of a vehicle from a camera included in the camera system;
identify, with a trained neural network, a viewpoint of the image;
generate attention maps from the image that identify regions of the vehicle which include overlapping visual patterns between the identified viewpoint and one or more additional viewpoints;
generate a multi-view representation of the vehicle utilizing the attention maps; and
perform vehicle re-identification by comparing the multi-view representation to vehicles identified in previously captured images.
2. The system of claim 1 , wherein the multi-view representation includes inferred vehicle information associated with the one or more additional viewpoints that provides a global view of the vehicle.
3. The system of claim 2 , wherein a conditional generative adversarial network is utilized to infer the vehicle information that is used to generate the multi-view representation of the vehicle.
4. The system of claim 3 , wherein the conditional generative adversarial network comprises a generative neural network and a discriminative neural network that compete against one another to generate the multi-view representation of the vehicle.
5. The system of claim 4 , wherein the conditional generative adversarial network is trained using a second generative neural network that utilizes real image data of vehicles.
6. The system of claim 1 , wherein comparing the multi-view representation to vehicles identified in previously captured images includes utilizing a pairwise distance metric learning function to compute distance metrics indicating a similarity between the vehicle in the image and the vehicles identified in the previously captured images.
7. The system of claim 1 , wherein the attention maps are generated, at least in part, using an attention model that enables identification of context vectors indicating the overlapping visual patterns of the vehicle between the identified viewpoint and the one or more additional viewpoints.
8. The system of claim 1 , wherein execution of the instructions by the one or more processors causes the computing device to:
use a trained neural network to extract vehicle features from the image, including features that identify a color of the vehicle, a model of the vehicle, and a type of the vehicle.
9. The system of claim 8 , wherein:
the trained neural network comprises a convolutional neural network that is trained using vehicle attribute labels; and
the convolutional neural network is utilized to train an attention model that is used to generate the attention maps.
10. The system of claim 1 , wherein the system for re-identifying the vehicle is utilized in connection with a surveillance system or a transportation system.
11. A method for re-identifying a vehicle comprising:
receiving an image of a vehicle from a camera included in a camera system;
identifying, with a trained neural network, a viewpoint of the image;
generating attention maps from the image that identify regions of the vehicle which include overlapping visual patterns between the identified viewpoint and one or more additional viewpoints;
generating a multi-view representation of the vehicle utilizing the attention maps; and
performing vehicle re-identification by comparing the multi-view representation to vehicles identified in previously captured images.
12. The method of claim 11 , wherein the multi-view representation includes inferred vehicle information associated with the one or more additional viewpoints that provides a global view of the vehicle.
13. The method of claim 12 , wherein a conditional generative adversarial network is utilized to infer the vehicle information that is used to generate the multi-view representation of the vehicle.
14. The method of claim 13 , wherein the conditional generative adversarial network comprises a generative neural network and a discriminative neural network that compete against one another to generate the multi-view representation of the vehicle.
15. The method of claim 14 , wherein the conditional generative adversarial network is trained using a second generative neural network that utilizes real image data of vehicles.
16. The method of claim 11 , wherein comparing the multi-view representation to vehicles identified in previously captured images includes utilizing a pairwise distance metric learning function to compute distance metrics indicating a similarity between the vehicle in the image and the vehicles identified in the previously captured images.
17. The method of claim 11 , wherein the attention maps are generated, at least in part, using an attention model that enables identification of context vectors indicating the overlapping visual patterns of the vehicle between the identified viewpoint and the one or more additional viewpoints.
18. The method of claim 11 , wherein a trained neural network is used to extract vehicle features from the image, including features that identify a color of the vehicle, a model of the vehicle, and a type of the vehicle.
19. The method of claim 18 , wherein:
the trained neural network comprises a convolutional neural network that is trained using vehicle attribute labels; and
the convolutional neural network is utilized to train an attention model that is used to generate the attention maps.
20. The method of claim 11 , wherein the method for re-identifying the vehicle is utilized in connection with a surveillance system or a transportation system.