System and method for performing image processing based on a damage assessment image judgement model
A system is provided for performing image processing. During operation, the system can obtain a video stream of a target object, and obtain, from the video stream, a plurality of images ranked according to a first sequence. The target object can be, e.g., a damage vehicle. The system can then extract a set of feature vectors corresponding to the plurality of images. The system may sequentially provide the set of feature vectors to a trained damage assessment image judgment model. Next, the system can apply the trained damage assessment image judgment model to determine whether each image in the plurality of images corresponds to a damage assessment image.
1. A computer-implemented method for performing image processing, the method comprising:
capturing, by a computing device, a video stream of a target object, wherein the video stream includes a plurality of images ranked according to a first sequence;
providing, based on the first sequence, the plurality of images to a set of models for extracting a first set of feature vectors corresponding to a current image in the plurality of images, which comprises:
applying a categorization model to a current image to extract a set of basic feature attributes
applying an optical flow model to the current image to construct optical flow information between a sequence of consecutive images; and
applying a target detection and segmentation model to the current image to extract a set of semantic feature attributes;
applying a trained damage assessment image judgement model to the first set of feature vectors corresponding to the current image, which involves:
providing, to the trained damage assessment image judgement model, the first set of feature vectors and a previous output of the trained damage assessment image judgement model corresponding to a second set of feature vectors associated with a previous image in the first sequence;
generating a current output corresponding to the current image; and
determining, based on the current output, whether the current image corresponds to a damage assessment image.
2. The method of claim 1 , wherein the set of basic feature attributes further comprises:
whether the current image includes at least a damage area of the target object, and whether a photographing angle is skewed.
3. The method of claim 1 , wherein the trained damage assessment image judgment model is obtained by:
obtaining a plurality of video streams corresponding to a plurality of training-sample target objects;
for each video stream associated with each training-sample target object:
obtaining a plurality of samples in chronological order from the video stream;
extracting a set of feature vectors and a set of tag values from the plurality of samples;
providing the set of feature vectors to a damage assessment image judgment model; and
training the damage assessment image judgment model, based on the set of feature vectors and the set of tag values, to minimize a prediction damage function.
4. The method of claim 1 , wherein the damage assessment judgement model comprises a Long Short Term Memory (LSTM) neural network, a recursive neural network (RNN), or a Gated Recurrent Unit (GRU) neural network.
5. An apparatus for performing image processing, comprising:
one or more processors; and
a storage medium storing instructions that, when executed by the one or more processors, cause the apparatus to perform a method comprising:
capturing, by a computing device, a video stream of a target object, wherein the video stream includes a plurality of images ranked according to a first sequence;
providing, based on the first sequence, the plurality of images to a set of models for extracting a first set of feature vectors corresponding to a current image in the plurality of images, which comprises:
applying a categorization model to a current image to extract a set of basic feature attributes;
applying an optical flow model to the current image to construct optical flow information between a sequence of consecutive images; and
applying a target detection and segmentation model to the current image to extract a set of semantic feature attributes;
applying a trained damage assessment image judgement model to the first set of feature vectors corresponding to the current image, which involves:
providing, to the trained damage assessment image judgement model, the first set of feature vectors and a previous output of the trained damage assessment image judgement model corresponding to a second set of feature vectors associated with a previous image in the first sequence;
generating a current output corresponding to the current image; and
determining, based on the current output, whether the current image corresponds to a damage assessment image.
6. The apparatus of claim 5 , wherein the set of basic feature attributes further comprises:
whether the current image includes at least a damage area of the target object and whether a photographing angle is skewed.
7. The apparatus of claim 5 , wherein the trained damage assessment image judgment model is obtained by:
obtaining a plurality of video streams corresponding to a plurality of training-sample target objects;
for each video stream associated with each target object:
obtaining a plurality of samples in chronological order from the video stream;
extracting a set of feature vectors and a set of tag values from the plurality of samples;
providing the set of feature vectors to a damage assessment image judgment model; and
training the damage assessment image judgment model, based on the set of feature vectors and the set of tag values, to minimize a prediction damage function.
8. The apparatus of claim 5 , wherein the damage assessment judgement model comprises a Long Short Term Memory (LSTM) neural network, a recursive neural network (RNN), or a Gated Recurrent Unit (GRU) neural network.
9. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for image processing, the method comprising:
capturing, by a computing device, a video stream of a target object, wherein the video stream includes a plurality of images ranked according to a first sequence;
providing, based on the first sequence, the plurality of images to a set of models for extracting a first set of feature vectors corresponding to a current image in the plurality of images, which comprises:
applying a categorization model to a current image to extract a set of basic feature attributes
applying an optical flow model to the current image to construct optical flow information between a sequence of consecutive images; and
applying a target detection and segmentation model to the current image to extract a set of semantic feature attributes;
applying a trained damage assessment image judgement model to the first set of feature vectors corresponding to the current image, which involves:
providing, to the trained damage assessment image judgement model, the first set of feature vectors and a previous output of the trained damage assessment image judgement model corresponding to a second set of feature vectors associated with a previous image in the first sequence;
generating a current output corresponding to the current image; and
determining, based on the current output, whether the current image corresponds to a damage assessment image.
10. The non-transitory computer-readable storage medium of claim 9 , wherein the set of basic feature attributes further comprises:
whether the current image includes at least a damage area of the target object and whether a photographing angle is skewed.
11. The non-transitory computer-readable storage medium of claim 9 , wherein the trained damage assessment image judgment model is obtained by:
obtaining a plurality of video streams corresponding to a plurality of training-sample target objects;
for each video stream associated with each target object:
obtaining a plurality of samples in chronological order from the video stream;
extracting a set of feature vectors and a set of tag values from the plurality of samples;
providing the set of feature vectors to a damage assessment image judgment model; and
training the damage assessment image judgment model, based on the set of feature vectors and the set of tag values, to minimize a prediction damage function.
12. The non-transitory computer-readable storage medium of claim 9 , wherein the damage assessment judgement model comprises a Long Short Term Memory (LSTM) neural network, a recursive neural network (RNN), or a Gated Recurrent Unit (GRU) neural network.
13. The method of claim 1 , wherein the set of basic feature attributes comprise one or more of:
whether the current image is blurred, whether the current image includes at least a damage area of the target object, whether light is sufficient, whether a photographing angle is skewed, and whether a photographing distance is appropriate.
14. The method of claim 1 , wherein the set of semantic feature attributes comprises one or more of:
a number of bounding boxes, a number of segmentation categories, an image ratio of a maximum bounding box, and an image ratio of a maximum segmentation category.