Method of segmenting abnormal robust for complex autonomous driving scenes and system thereof

1. A method of segmenting an abnormal robust for complex autonomous driving scenes, comprising following steps:

S1, a segmentation module

firstly, taking an input image as an input of a segmentation network to obtain a semantic feature map of a middle layer;

secondly, quantifying, based on two discrete metrics of an entropy E x for softmax and a distance D y for softmax, an uncertainty of the semantic feature map;

S2, a synthesis module

directly comparing, through a model, high-level semantic information on images by introducing a perceptual loss between an original input image and a synthesized image, rather than merely utilizing low-level visual features such as colors and textures;

S3, a difference module

distinguishing, through a training network, the original input image from the synthesized image, and associating a generated feature map with an uncertainty feature map pixel by pixel; eventually; decoding, through a decoder, feature maps, and performing, based on a FPN, high-level skip connections on the feature maps respectively to eventually obtain segmentation prediction results of abnormal objects in complex scenes;

S4, a model training module

overwriting, by marking a void class in a Ground Truth image as an abnormal class, objects that do not belong to a training class,

wherein the model training module performs iterative trainings continuously based on a constructed loss function until training rounds or training accuracy reaches a set threshold, and save a network model with a minimum loss value; wherein considering that the abnormal objects do not belong to a class in a training set, the model training module is capable of performing a robust detection on the abnormal objects in complex real-world scenes on a premise of ensuring that the model does not overfit specific objects in non distributed data during a training process; and

S5, an integrated prediction module

integrating, by using weighted average values, prediction results from different inputs, wherein a size of an optimal weight is automatically selected by a grid searching, wherein the segmentation module, the synthesis module, the difference module, the model training module, and the integrated prediction module are executed by a hardware processor;

wherein in S1, a difference value between two distances is used to understand a false segmentation prediction, and the two distances are defined respectively as follows:

E

x

=

-

∑

c

p

⁡

(

c

)

⁢

log

2

⁢

p

⁡

(

c

)

,

D

y

=

(

1

2

-

max

c

p

⁡

(

c

)

)

+

(

1

2

+

max

c

p

⁡

(

c

)

)

,

where p(c) represents a softmax probability of a c class.

2. The method of segmenting the abnormal robust for the complex autonomous driving scenes according to claim 1 , wherein in S2, the perceptual loss between a pixel x of the input image and a pixel r of the synthesized image is defined as follows:

p

pcep

=

∑

i

=

1

N

1

K

i

⁢



S

i

(

x

)

-

S

i

(

r

)



2

2

,

where S i represents an i-th layer which includes K, elements and N training layers.

3. The method of segmenting the abnormal robust for the complex autonomous driving scenes according to claim 1 , wherein in S4, firstly, predicted semantics in regions to be predicted are closely related to different abnormal scenes; secondly, the regions to be predicted are also closely related to uncertain pixels with high confidence, thereby guiding a network to perform an abnormal prediction based on uncertain information of pixels at the same time.

4. A system used for the method of segmenting the abnormal robust for the complex autonomous driving scenes according to claim 1 , comprising:

the segmentation module, configured to transmit an obtained input image to the segmentation network to obtain segmentation prediction images, and then quantify the uncertainty of a segmentation prediction by means of calculating two different discrete metrics;

the synthesis module, configured to match a generated data distribution with a data distribution of the input image by utilizing a conditional generative adversarial network;

the difference module, configured to model and calculate the input image, an generated image, the semantic feature map and the uncertainty feature map based on an encoder, a fusion module and a decoder, to generate the segmentation prediction images for the abnormal objects;

the integrated prediction module, configured to take an obtained uncertainty prediction as a focus of the difference module, and predict input image data by utilizing a trained network model to obtain the segmentation prediction results of the abnormal objects.

5. A system used for the method of segmenting the abnormal robust for the complex autonomous driving scenes according to claim 2 , comprising:

the segmentation module, configured to transmit an obtained input image to the segmentation network to obtain segmentation prediction images, and then quantify the uncertainty of a segmentation prediction by means of calculating two different discrete metrics;

the synthesis module, configured to match a generated data distribution with a data distribution of the input image by utilizing a conditional generative adversarial network;

the difference module, configured to model and calculate the input image, an generated image, the semantic feature map and the uncertainty feature map based on an encoder, a fusion module and a decoder, to generate the segmentation prediction images for the abnormal objects;

the integrated prediction module, configured to take an obtained uncertainty prediction as a focus of the difference module, and predict input image data by utilizing a trained network model to obtain the segmentation prediction results of the abnormal objects.

6. A system used for the method of segmenting the abnormal robust for the complex autonomous driving scenes according to claim 3 , comprising:

the segmentation module, configured to transmit an obtained input image to the segmentation network to obtain segmentation prediction images, and then quantify the uncertainty of a segmentation prediction by means of calculating two different discrete metrics;

the synthesis module, configured to match a generated data distribution with a data distribution of the input image by utilizing a conditional generative adversarial network;

the difference module, configured to model and calculate the input image, an generated image, the semantic feature map and the uncertainty feature map based on an encoder, a fusion module and a decoder, to generate the segmentation prediction images for the abnormal objects;

the integrated prediction module, configured to take an obtained uncertainty prediction as a focus of the difference module, and predict input image data by utilizing a trained network model to obtain the segmentation prediction results of the abnormal objects.