Method and system for facilitating recognition of vehicle parts based on a neural network

1. A computer-implemented method for recognizing parts of a vehicle, the method comprising:

generating a convolution feature map of an image of the vehicle;

determining, based on the convolution feature map, one or more proposed regions, wherein a respective proposed region corresponds to a target of a respective vehicle part;

determining a class and a bounding box of a first vehicle part corresponding to a first proposed region based on a feature of the first proposed region;

optimizing classes and bounding boxes of the vehicle parts based on correlated features of the corresponding proposed regions;

generating a result which indicates a list including an insurance claim item and corresponding damages based on the optimized classes and bounding boxes of the vehicle parts.

2. The method of claim 1 , wherein determining the one or more proposed regions is performed by a fully convolutional network, and wherein determining the one or more proposed regions comprises:

performing, for a plurality of convolutional mapping positions, a convolutional operation for a respective convolutional mapping position with a sliding window in the convolution feature map;

obtaining a feature vector of the respective convolutional mapping position;

predicting whether the respective convolutional mapping position comprises a foreground target with respect to a plurality of predetermined anchors based on the feature vector of the respective convolutional mapping position; and

predicting a border of a proposed region in the respective convolutional mapping position that corresponds to each anchor.

3. The method of claim 1 , wherein the correlated features of the corresponding proposed regions include one or more of:

a size of a respective proposed region;

a position relationship between the proposed regions;

a distance between the proposed regions; and

an intersection-over-union ratio of the proposed regions.

4. The method of claim 1 , wherein prior to optimizing the classes and the bounding boxes of the vehicle parts based on the correlated features of the corresponding proposed regions, the method further comprises:

obtaining the classes and the bounding boxes of the vehicle parts corresponding to the proposed regions; and

extracting the correlated features of the proposed regions.

5. The method of claim 1 , further comprising:

determining an energy function and a corresponding probability function of the conditional random field; and

solving the energy function while minimizing the probability function,

wherein the energy function comprises:

a data term which is based on a probability of each proposed region belonging to each class; and

a smoothing term which is based on the correlated features of the proposed regions.

6. The method of claim 5 , wherein optimizing the classes and bounding boxes of the vehicle parts is performed by a recurrent neural network, wherein the method further comprises:

solving the energy function while minimizing the probability function by performing multiple iterative operations through the recurrent neural network to obtain an approximation of the probability function,

wherein an iterative operation comprises updating the probability of each proposed region belonging to each class based on a pre-trained compatibility matrix, and

wherein the pre-trained compatibility matrix indicates a probability of compatibility between the classes of the vehicle.

7. The method of claim 1 , wherein parameters used to perform the method are jointly trained from end to end based on training samples.

8. The method of claim 7 , wherein the parameters which are jointly trained from end to end are trained by:

inputting the training samples into a convolution module, wherein the training samples indicate corresponding classes and bounding boxes;

obtaining a prediction result based on the classes and the bounding boxes optimized by the conditional random field module,

wherein the prediction result comprises predicted classes and predicted bounding boxes of a plurality of target regions;

determining a prediction error for the target regions based on the prediction result and the corresponding classes and bounding boxes indicated by the training samples;

determining a loss function based on the prediction error,

wherein the loss function comprises a cross term of the prediction error for the target regions; and

counter-propagating the prediction error based on the loss function by counter-propagating the prediction error of a first target region to a set of target regions correlated with the first target region.

9. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, wherein the method comprises:

generating a convolution feature map of an image of the vehicle;

determining, based on the convolution feature map, one or more proposed regions, wherein a respective proposed region corresponds to a target of a respective vehicle part;

determining a class and a bounding box of a first vehicle part corresponding to a first proposed region based on a feature of the first proposed region;

optimizing classes and bounding boxes of the vehicle parts based on correlated features of the corresponding proposed regions;

generating a result which indicates a list including an insurance claim item and corresponding damages based on the optimized classes and bounding boxes of the vehicle parts.

10. The storage medium of claim 9 , wherein determining the one or more proposed regions is performed by a fully convolutional network, and wherein determining the one or more proposed regions comprises:

performing, for a plurality of convolutional mapping positions, a convolutional operation for a respective convolutional mapping position with a sliding window in the convolution feature map;

obtaining a feature vector of the respective convolutional mapping position;

predicting whether the respective convolutional mapping position comprises a foreground target with respect to a plurality of predetermined anchors based on the feature vector of the respective convolutional mapping position; and

predicting a border of a proposed region in the respective convolutional mapping position that corresponds to each anchor.

11. The storage medium of claim 9 , wherein the correlated features of the corresponding proposed regions include one or more of:

a size of a respective proposed region;

a position relationship between the proposed regions;

a distance between the proposed regions; and

an intersection-over-union ratio of the proposed regions.

12. The storage medium of claim 9 , wherein prior to optimizing the classes and the bounding boxes of the vehicle parts based on the correlated features of the corresponding proposed regions, the method further comprises:

obtaining the classes and the bounding boxes of the vehicle parts corresponding to the proposed regions; and

extracting the correlated features of the proposed regions.

13. The storage medium of claim 9 , wherein the method further comprises:

determining an energy function and a corresponding probability function of the conditional random field; and

solving the energy function while minimizing the probability function,

wherein the energy function comprises:

a data term which is based on a probability of each proposed region belonging to each class; and

a smoothing term which is based on the correlated features of the proposed regions.

14. The storage medium of claim 13 , wherein optimizing the classes and bounding boxes of the vehicle parts is performed by a recurrent neural network, and wherein the method further comprises:

solving the energy function while minimizing the probability function by performing multiple iterative operations through the recurrent neural network to obtain an approximation of the probability function,

wherein an iterative operation comprises updating the probability of each proposed region belonging to each class based on a pre-trained compatibility matrix, and

wherein the pre-trained compatibility matrix indicates a probability of compatibility between the classes of the vehicle.

15. The storage medium of claim 9 , wherein parameters used to perform the method are jointly trained from end to end based on training samples.

16. The storage medium of claim 15 , wherein the parameters which are jointly trained from end to end are trained by:

inputting the training samples into a convolution module, wherein the training samples indicate corresponding classes and bounding boxes;

obtaining a prediction result based on the classes and the bounding boxes optimized by the conditional random field module,

wherein the prediction result comprises predicted classes and predicted bounding boxes of a plurality of target regions;

determining a prediction error for the target regions based on the prediction result and the corresponding classes and bounding boxes indicated by the training samples;

determining a loss function based on the prediction error,

wherein the loss function comprises a cross term of the prediction error for the target regions; and

counter-propagating the prediction error based on the loss function by counter-propagating the prediction error of a first target region to a set of target regions correlated with the first target region.

17. A computer system for recognizing parts of a vehicle, the system comprising:

a convolution module configured to generate a convolution feature map of an image of the vehicle;

a region proposal module configured to determine, based on the convolution feature map, one or more proposed regions, wherein a respective proposed region corresponds to a target of a respective vehicle part;

a classification module configured to determine a class and a bounding box of a first vehicle part corresponding to a first proposed region based on a feature of the first proposed region;

a conditional random field module configured to optimize classes and bounding boxes of the vehicle parts based on correlated features of the corresponding proposed regions; and

a reporting module configured to generate a result which indicates a list including an insurance claim item and corresponding damages based on the optimized classes and bounding boxes of the vehicle parts.

18. The computer system of claim 17 , wherein the region proposal module is a fully convolutional network, comprising:

a convolution processing module configured to:

perform, for a plurality of convolutional mapping positions, a convolutional operation for a respective convolutional mapping position with a sliding window in the convolution feature map; and

obtain a feature vector of the respective convolutional mapping position;

a bounding box classification module configured to predict whether the respective convolutional mapping position comprises a foreground target with respect to a plurality of predetermined anchors based on the feature vector of the respective convolutional mapping position; and

a bounding box prediction module configured to predict a border of a proposed region in the respective convolutional mapping position that corresponds to each anchor.

19. The computer system of claim 17 , wherein the conditional random field module is further configured to, prior to optimizing the classes and bounding boxes of the vehicle parts based on the correlated features of the corresponding proposed regions:

obtain the classes and bounding boxes of the vehicle parts corresponding to the proposed regions; and

extract the correlated features of the proposed regions.

20. The computer system of claim 17 , wherein the conditional random field module is further configured to:

determine an energy function and a corresponding probability function of the conditional random field; and

solve the energy function while minimizing the probability function,

wherein the energy function comprises:

a data term which is based on a probability of each proposed region belonging to each class; and

a smoothing term which is based on the correlated features of the proposed regions.