Vehicle reidentification
A framework for decision fusion utilizing features extracted from vehicle images and their detected wheels. Siamese networks are exploited to extract key signatures from pairs of vehicle images. The present disclosure melds different types of similarity scores (whole vehicle and wheels/hubcaps) between target and test vehicles to robustly integrate different similarity scores and provide a more informed decision for vehicle matching. The present disclosure provides improved accuracy for side-view vehicle matching even under different illumination conditions and elevation angles.
1. A system for matching vehicles, the system comprising:
a hardware processor configured to
access an image that shows a side of a target vehicle,
detect in the side image of the target vehicle a front wheel or a back wheel or both, and form corresponding images of the wheels of the target vehicle,
receive one or more images, each image showing a side of a respective vehicle, and for each image
compare the side image of the vehicle and the side image of the target vehicle using a whole vehicle matching network to produce a whole-vehicle similarity score,
detect in the side image of the vehicle a front wheel or a back wheel or both, and form corresponding images of the wheels of the vehicle,
compare the image of the vehicle front wheel and the image of the target vehicle front wheel using a front wheel matching network to produce a front wheel similarity score, and/or compare the image of the vehicle back wheel and the image of the target vehicle back wheel using a back wheel matching network to produce a back wheel similarity score,
combine the whole-vehicle similarity score with either the front wheel similarity score or the back wheel similarity score or both to produce an overall similarity score, and
determine whether the vehicle matches the target vehicle if the overall similarity score meets a predetermined similarity score threshold.
2. The system of claim 1 , wherein to produce the overall similarity score, the hardware processor configured to compute an average of the whole vehicle similarity score and either the front wheel similarity score or the back wheel similarity score or both.
3. The system of claim 1 , wherein to produce the overall similarity score, the hardware processor configured to compute a weighted average of the whole vehicle similarity score and either the front wheel similarity score or the back wheel similarity score or both.
4. The system of claim 1 , wherein the whole vehicle matching network used by the hardware processor comprises a Siamese network.
5. The system of claim 4 , wherein the hardware processor is configured to train and operate the whole vehicle matching network, the whole vehicle matching network including two identical branches each having a plurality of layers with rectified linear activation functions wherein a max pooling step is applied after each of the plurality of layers, the whole vehicle matching network further including a differencing phase where the output of the two branches are subtracted from each other to generate a differenced output, the whole vehicle matching network further including a matching component including a plurality of layers with rectified linear activation functions that operates on the differenced output to generate a similarity score.
6. The system of claim 1 , wherein the front wheel matching network and the back wheel matching network used by the hardware processor each comprises a Siamese network.
7. The system of claim 6 , wherein the hardware processor is configured to train and operate the front wheel matching network and the back wheel matching network, the front wheel matching network and back wheel matching network being a Siamese network including two identical branches each having a plurality of layers with rectified linear activation functions, the Siamese network further including a differencing phase where the output of the two branches are subtracted from each other to generate a differenced output, the Siamese network further including a dense layer with a sigmoid activation function that operates on the differenced output to generate a wheel similarity score.
8. The system of claim 1 , wherein the hardware processor is configured to
detect in vehicle side images a front wheel or a back wheel or both, and
form corresponding wheel images by pre-process wheel locking.
9. The system of claim 1 , comprising
one or more cameras configured to acquire the side images of the respective vehicles,
wherein the hardware processor is configured to receive the side images of the respective vehicles from the cameras.
10. The system of claim 1 , comprising
a storage module; and
a network interface configured to
receive the target vehicle side image over a network, and
save the target vehicle side image to the storage module,
wherein the hardware processor is configured to save the target vehicle wheel images to the storage module.
11. The system of claim 10 , wherein the hardware processor, the storage module, and the network interface are disposed in a vehicle.
12. A method for matching vehicles, the method comprising:
receiving a target vehicle image depicting a side view of a target vehicle;
detecting, with a hardware processor, in the side view of the target vehicle in the target vehicle image a vehicle wheel and forming an image of the detected vehicle wheel of the target vehicle;
receiving a test vehicle image depicting a side view of a test vehicle;
comparing, with a hardware processor, the side view of the target vehicle in the target vehicle image and the side view of the test vehicle in the test vehicle image using a whole vehicle matching network to produce a whole-vehicle similarity score;
detecting, with a hardware processor, in the side view of the test vehicle in the test vehicle image a vehicle wheel and forming an image of the detected vehicle wheel of the test vehicle;
comparing the image of the test vehicle wheel and the image of the target vehicle wheel using a wheel matching network to produce a wheel similarity score;
fusing the whole-vehicle similarity score with the wheel similarity score to produce a fused vehicle similarity score; and
determining whether the test vehicle matches the target vehicle.
13. The method of claim 12 , wherein producing the fused similarity score includes computing an average of the whole vehicle similarity score and the wheel similarity score.
14. The method of claim 12 , wherein producing the fused similarity score includes computing a weighted average of the whole vehicle similarity score and the wheel similarity score.
15. The method of claim 12 , wherein the whole vehicle matching network used by the hardware processor comprises a Siamese network.
16. The method of claim 15 , wherein the hardware processor is configured to train and operate the whole vehicle matching network, the whole vehicle matching network including two identical branches each having a plurality of layers with rectified linear activation functions wherein a max pooling step is applied after each of the plurality of layers, the whole vehicle matching network further including a differencing phase where the output of the two branches are subtracted from each other to generate a differenced output, the whole vehicle matching network further including a matching component including a plurality of layers with rectified linear activation functions that operates on the differenced output to generate a similarity score.
17. The method of claim 12 , wherein the wheel matching network used by the hardware processor is a Siamese network.
18. The method of claim 17 , wherein the hardware processor is configured to train and operate the Siamese network, the Siamese network including two identical branches each having a plurality of layers with rectified linear activation functions, the Siamese network further including a differencing phase where the output of the two branches are subtracted from each other to generate a differenced output, the Siamese network further including a dense layer with a sigmoid activation function that operates on the differenced output to generate a wheel similarity score.
19. The method of claim 12 , wherein the detecting in the target vehicle image includes:
detecting, with a hardware processor, in the side view of the target vehicle in the target vehicle image a front target vehicle wheel and a back target vehicle wheel, and forming respective images of the detected vehicle wheels of the target vehicle; and wherein the detecting in the test vehicle image includes:
detecting, with a hardware processor, in the side view of the test vehicle in the test vehicle image a front test vehicle wheel and a back test vehicle wheel, and forming respective images of the detected vehicle wheels of the test vehicle.
20. The method of claim 19 , wherein the forming of respective images of the detected vehicle wheels of the target vehicle and the forming of respective images of the detected vehicle wheels of the test vehicle include forming wheel images by pre-process wheel locking.