Computer-implemented method of detecting foreign object on background object in an image, apparatus for detecting foreign object on background object in an image, and computer-program product

1. A computer-implemented method of detecting a foreign object on a background object in an image, comprising:

extracting image features of the image based on image characteristics of the background object and a suspected foreign object;

detecting a salient region in the image based on a human visual attention model;

generating a salient region mask based on detection of the salient region;

obtaining the image features in a region surrounded by the salient region mask;

performing feature combination and feature vector length normalization on the image features of the region surrounded by the salient region mask to generate normalized feature vectors;

identifying the suspected foreign object using an image classifier, wherein the image classifier uses the normalized feature vectors as input for identifying the suspected foreign object; and

determining a color consistency of the region surrounded by the salient region mask;

wherein determining the color consistency comprises:

calculating variance of hue features of the region surrounded by the salient region mask; and

determining whether the variance of the hue features is greater than a first threshold value;

wherein the variance of the hue features being greater than the first threshold value indicates the color consistency of the region surrounded by the salient region mask being below a second threshold value.

2. The computer-implemented method of claim 1 , further comprising pre-training an initial image classifier to obtain the image classifier;

wherein pre-training the initial image classifier comprises:

providing a training sample, a training classification label corresponding to the training sample, training image features corresponding to the training sample, and a training salient region mask corresponding to the training sample, the training image features comprising training features of a first type and training features of a second type;

performing feature combination and feature vector length normalization on the training image features of a region surrounded by the training salient region mask to generate training normalized feature vectors;

classifying the training sample based on the training normalized feature vectors to generate an initial classification label;

determining a degree of match between the initial classification label and the training classification label; and

tuning the initial image classifier based on the degree of match.

3. The computer-implemented method of claim 1 , further comprising outputting a classification label corresponding to the suspected foreign object and a relative location of the suspected foreign object on the background object.

4. The computer-implemented method of claim 1 , wherein the background object is a photovoltaic panel.

5. The computer-implemented method of claim 1 , further comprising segmenting the salient region into a plurality of sub-regions, based on a determination that the color consistency of the region surrounded by the salient region mask is below a threshold value; and

determining a contour of a refined salient region based on results of segmenting the salient region into the plurality of sub-regions.

6. The computer-implemented method of claim 1 , wherein the image features extracted from the image comprise features of a first type and features of a second type;

performing feature combination on the image features comprises combining a respective one of the features of the first type and a respective one of the features of the second type into a respective one of one-dimensional feature vectors.

7. The computer-implemented method of claim 6 , wherein the features of the first type are hue features, and the features of the second type are local binary pattern (LBP) texture features.

8. An apparatus for detecting a foreign object on a background object in an image, comprising:

a memory; and

one or more processors;

wherein the memory and the one or more processors are connected with each other; and

the memory stores computer-executable instructions for controlling the one or more processors to:

extract image features of the image based on image characteristics of the background object and a suspected foreign object;

detect a salient region in the image based on a human visual attention model;

generate a salient region mask based on detection of the salient region;

obtain the image features in a region surrounded by the salient region mask;

perform feature combination and feature vector length normalization on the image features of the region surrounded by the salient region mask to generate normalized feature vectors;

identify the suspected foreign object using an image classifier, wherein the image classifier uses the normalized feature vectors as input for identifying the suspected foreign object; and

determine a color consistency of the region surrounded by the salient region mask;

calculate variance of hue features of the region surrounded by the salient region mask; and

determine whether the variance of the hue features is greater than a first threshold value;

wherein the variance of the hue features being greater than the first threshold value indicates the color consistency of the region surrounded by the salient region mask being below a second threshold value.

9. The apparatus of claim 8 , wherein the memory further stores computer-executable instructions for controlling the one or more processors to segment the salient region into a plurality of sub-regions, based on a determination that the color consistency of the region surrounded by the salient region mask is below a threshold value; and

determine a contour of a refined salient region based on results of segmenting the salient region into the plurality of sub-regions.

10. The apparatus of claim 8 , wherein the memory further stores computer-executable instructions for controlling the one or more processors to output a classification label corresponding to the suspected foreign object and a relative location of the suspected foreign object on the background object.

11. The apparatus of claim 8 , wherein the background object is a photovoltaic panel.

12. The apparatus of claim 8 , further comprising an image capturing device configured to obtain the image having the foreign object on the background object.

13. The apparatus of claim 8 , wherein the image features extracted from the image comprise features of a first type and features of a second type;

wherein the memory stores computer-executable instructions for controlling the one or more processors to combine a respective one of the features of the first type and a respective one of the features of the second type into a respective one of one-dimensional feature vectors.

14. The apparatus of claim 13 , wherein the features of the first type are hue features, and the features of the second type are local binary pattern (LBP) texture features.

15. A computer-program product comprising a non-transitory tangible computer-readable medium having computer-readable instructions thereon, the computer-readable instructions being executable by a processor to cause the processor to perform:

extracting image features of an image based on image characteristics of a background object and a suspected foreign object in the image;

detecting a salient region in the image based on a human visual attention model;

generating a salient region mask based on detection of the salient region;

obtaining the image features in a region surrounded by the salient region mask;

performing feature combination and feature vector length normalization on the image features of the region surrounded by the salient region mask to generate normalized feature vectors;

identifying the suspected foreign object using an image classifier, wherein the image classifier uses the normalized feature vectors as input for identifying the suspected foreign object; and

determining a color consistency of the region surrounded by the salient region mask;

wherein determining the color consistency comprises:

calculating variance of hue features of the region surrounded by the salient region mask; and

determining whether the variance of the hue features is greater than a first threshold value;

wherein the variance of the hue features being greater than the first threshold value indicates the color consistency of the region surrounded by the salient region mask being below a second threshold value.

16. A computer-implemented method of detecting a foreign object on a background object in an image, comprising:

extracting image features of the image based on image characteristics of the background object and a suspected foreign object;

detecting a salient region in the image based on a human visual attention model;

generating a salient region mask based on detection of the salient region;

obtaining the image features in a region surrounded by the salient region mask;

performing feature combination and feature vector length normalization on the image features of the region surrounded by the salient region mask to generate normalized feature vectors;

identifying the suspected foreign object using an image classifier, wherein the image classifier uses the normalized feature vectors as input for identifying the suspected foreign object; and

pre-training an initial image classifier to obtain the image classifier;

wherein pre-training the initial image classifier comprises:

providing a training sample, a training classification label corresponding to the training sample, training image features corresponding to the training sample, and a training salient region mask corresponding to the training sample, the training image features comprising training features of a first type and training features of a second type;

performing feature combination and feature vector length normalization on the training image features of a region surrounded by the training salient region mask to generate training normalized feature vectors;

classifying the training sample based on the training normalized feature vectors to generate an initial classification label;

determining a degree of match between the initial classification label and the training classification label;

tuning the initial image classifier based on the degree of match;

providing the training image having a training foreign object on a training background object;

extracting training image features of the training image based on image characteristics of the training background object and the training foreign object;

detecting a training salient region in the training image based on the human visual attention model;

generating the training salient region mask based on detection of the training salient region; and

obtaining the training image features in a region surrounded by the training salient region mask.