Object-centric fine-grained image classification
Systems and methods are disclosed for classifying vehicles by performing scale aware detection; performing detection assisted sampling for convolutional neural network (CNN) training, and performing deep CNN fine-grained image classification to classify the vehicle type.
1. A method for classifying a vehicle type, comprising:
performing scale aware detection;
performing detection assisted sampling for convolutional neural network (CNN) training; and
performing deep CNN fine grained image classification to classify the vehicle type;
wherein the scale aware detection is trained using a Regionlet detector, the Regionlet detector is a boosting classifier composed of weak classifiers:
H
(
x
)
=
∑
i
=
1
T
h
i
(
x
)
,
wherein T is the total number of training stages, h(x) is the weak classifier learned at stage t in training, x is the input image,
wherein the weak classifier h(x) is written as a function of the spatial location of Regionlets in h, and a feature used for h:
h t ( x )= G ( p t , f t , x ),
wherein p is a set of Regionlet locations, f is the feature extracted in the set of Regionlet locations.
2. The method of claim 1 , comprising using a selective search to generate object proposals for detection training and testing.
3. The method of claim 2 , wherein object proposals with more than 70% overlap with the ground truth are selected as positive samples during training and object proposals with less than 0.3% overlap with the ground truth are used as negative training samples.
4. The method of claim 1 , comprising apply multinomial sampling to images of the vehicle type.
5. The method of claim 1 , comprising applying regionlet re-localization method to learn a support vector regression model to predict an actual object location.
6. The method of claim 1 , comprising providing non-max suppression by taking the object proposal which gives the maximum detection response.
7. The method of claim 1 , comprising detecting an object with awareness of object scales and occlusions.
8. The method of claim 7 , wherein small detection responses are linked to small or occluded objects, or false alarms.
9. The method of claim 1 , comprising constructing a saliency aware dataset and using a scale aware object detection.
10. The method of claim 1 , comprising achieving occlusion awareness by training with visible objects.
11. The method of claim 1 , comprising labeling only a salient object in one image and checking consistency with a fine-grained category label.
12. The method of claim 1 , comprising labeling only one object as a detection ground truth for each image.
13. The method of claim 1 , comprising selecting an object based on mixed criteria of saliency.
14. The method of claim 13 , wherein the object is selected based on one or more of the following preferences: a big object preferred over small object, a visible object preferred over occluded object; a central object preferred over corner object, and consistency of an object's fine-grained category label with the image label.
15. A system to classify vehicles, comprising: a scale aware detector receiving an input image;
a deep convolutional neural network (CNN) coupled to the scale aware detector to classify a vehicle;
a detection assisted sampling module coupled to the scale aware detector, the sampling module generating data for CNN training; and
a deep CNN training module coupled to the detection assisted sampling module and the deep CNN;
wherein the scale aware detector is trained using a Regionlet detector, the Regionlet detector is a boosting classifier composed of weak classifiers:
H
(
x
)
=
∑
i
=
1
T
h
i
(
x
)
,
wherein T is the total number of training stages, h(x) is the weak classifier learned at stage tin training, x is the input image,
wherein the weak classifier h(x) is written as a function of the spatial location of Regionlets in h, and a feature used for h:
h t ( x )= G ( p t , f t , x ),
wherein p is a set of Regionlet locations, f is the feature extracted in the set of Regionlet locations.
16. The system of claim 15 , comprising a camera or a database of car images to provide input images.
17. The system of claim 15 , comprising a Regionlet detector for training the scale aware detection.
18. The system of claim 15 , comprising a vehicle or a machine controlled in part using the vehicle classification from the deep CNN.