Apparatus and method for detecting defect using deep learning-based surface inspection

1 . An apparatus for detecting defects using deep learning-based surface inspection, wherein an electronic device comprising at least one processor for detecting defects using deep learning-based surface inspection, comprises:

a normal image-based unsupervised training unit configured to, based on a normal image, train a reconstructive sub-network which allows an original image to be obtained through an image with noise inserted, and a discriminative sub-network which allows detection of a noise area; and

a defective image-based online supervised training unit configured to, based on a defective image, train a reconstructive sub-network which inputs an inspection target image to generate a flaw removal image, and a discriminative sub-network which extracts a flaw area image by comparing a difference between the inspection target image and the flaw removal image,

wherein the normal image-based unsupervised training unit comprises:

an anomaly generation unit configured to insert noise into a normal product image to generate a defective product image;

a reconstructive sub-network calculation unit configured to convert the defective product image into a normal product image; and

a discriminative sub-network calculation unit configured to extract a defective area by comparing a difference between the defective product image synthesized and the normal product image.

2 . The apparatus of claim 1 , wherein the anomaly generation unit comprises:

a Perlin noise generation unit configured to, when the normal product image is input, generate Perlin noise equal in size to the normal product image;

a noise area image extraction unit configured to extract a noise area image using a threshold; and

a data synthesis unit configured to synthesize the normal product image and noise data.

3 . The apparatus of claim 1 , wherein the reconstructive sub-network calculation unit comprises:

a flaw removal image generation unit configured to receive the defective product image synthesized in the anomaly generation unit as an input to output a flaw removal image of the same size as the defective product image through reconstructive sub-network calculation;

an image difference calculation unit configured to calculate a difference between the flaw removal image and the normal product image; and

an image concatenation unit configured to concatenate the flaw removal image and a defective image.

4 . The apparatus of claim 1 , wherein the discriminative sub-network calculation unit comprises:

a flaw area image generation unit configured to receive a defective product image and a normal product image concatenated as an input to generate a flaw area image of the same size as a product image through discriminative sub-network calculation;

an image difference calculation unit configured to calculate a difference between the flaw area image and a noise area image; and

a weight correction unit configured to correct weights of the reconstructive sub-network and the discriminative sub-network, using differences calculated from the reconstructive sub-network and the discriminative sub-network, respectively.

5 . An apparatus for detecting defects using deep learning-based surface inspection, wherein an electronic device comprising at least one processor for detecting defects using deep learning-based surface inspection, comprises:

a normal image-based unsupervised training unit configured to, based on a normal image, train a reconstructive sub-network which allows an original image to be obtained through an image with noise inserted, and a discriminative sub-network which allows detection of a noise area; and

a defective image-based online supervised training unit configured to, based on a defective image, train a reconstructive sub-network which inputs an inspection target image to generate a flaw removal image, and a discriminative sub-network which extracts a flaw area image by comparing a difference between the inspection target image and the flaw removal image,

wherein the defective image-based online supervised training unit comprises:

a reconstructive sub-network calculation unit configured to input the inspection target image to generate the flaw removal image;

a discriminative sub-network calculation unit configured to extract the flaw area image by comparing the difference between the inspection target image and the flaw removal image; and

a defective product detection and training unit configured to calculate whether the flaw area image is defective, detect a defective product and train based on a threshold to perform binarization of the flaw area image, and calculate a difference between a binarized image and the flaw area image to correct a weight of the discriminative sub-network.

6 . The apparatus of claim 5 , wherein the reconstructive sub-network calculation unit comprises:

a flaw removal image generation unit configured to receive the inspection target image as an input to generate the flaw removal image; and

an image concatenation unit configured to concatenate the inspection target image and the flaw removal image.

7 . The apparatus of claim 5 , wherein the discriminative sub-network calculation unit comprises:

a flaw area image generation unit configured to receive an inspection target image and a flaw removal image concatenated as an input to generate a flaw area image of the same size as a product image through discriminative sub-network calculation; and

a defect calculation unit configured to calculate whether the flaw area image is defective.

8 . The apparatus of claim 5 , wherein the defective product detection and training unit comprises:

an image binarization unit configured to detect a defective product and train based on a threshold to perform binarization of the flaw area image;

an image difference calculation unit configured to calculate a difference between a binarized image and the flaw area image; and

a weight correction unit configured to correct the weight of the discriminative sub-network based on the calculated difference between the binarized image and the flaw area image.

9 . A method for detecting defects using deep learning-based surface inspection, wherein an operation for detecting defects using deep learning-based surface inspection is performed in an electronic device comprising at least one processor, the method comprising:

a normal image-based unsupervised training step of, based on a normal image, training a reconstructive sub-network which allows an original image to be obtained through an image with noise inserted, and a discriminative sub-network which allows detection of a noise area; and

a defective image-based online supervised training step of, based on a defective image, training a reconstructive sub-network which inputs an inspection target image to generate a flaw removal image, and a discriminative sub-network which extracts a flaw area image by comparing a difference between the inspection target image and the flaw removal image,

wherein the normal image-based unsupervised training step comprises:

an anomaly generating step of inserting noise into a normal product image to generate a defective product image;

a reconstructive sub-network calculating step of converting the defective product image into a normal product image; and

a discriminative sub-network calculating step of extracting a defective area by comparing a difference between the defective product image synthesized and the normal product image.

10 . The method of claim 9 , wherein the anomaly generating step comprises:

a Perlin noise generating step of, when the normal product image is input, generating Perlin noise equal in size to the normal product image;

a noise area image extracting step of extracting a noise area image using a threshold; and

a data synthesizing step of synthesizing the normal product image and noise data.

11 . The method of claim 9 , wherein the reconstructive sub-network calculating step comprises:

a flaw removal image generating step of receiving the defective product image synthesized in the anomaly generating step as an input to output a flaw removal image of the same size as the defective product image through reconstructive sub-network calculation;

an image difference calculating step of calculating a difference between the flaw removal image and the normal product image; and

an image concatenating step of concatenating the flaw removal image and a defective image.

12 . The method of claim 9 , wherein the discriminative sub-network calculating step comprises:

a flaw area image generating step of receiving a defective product image and a normal product image concatenated as an input to generate a flaw area image of the same size as a product image through discriminative sub-network calculation;

an image difference calculating step of calculating a difference between the flaw area image and a noise area image; and

a weight correcting step of correcting weights of the reconstructive sub-network and the discriminative sub-network, using differences calculated from the reconstructive sub-network and the discriminative sub-network, respectively.

13 . A method for detecting defects using deep learning-based surface inspection, wherein an operation for detecting defects using deep learning-based surface inspection is performed in an electronic device comprising at least one processor, the method comprising:

a normal image-based unsupervised training step of, based on a normal image, training a reconstructive sub-network which allows an original image to be obtained through an image with noise inserted, and a discriminative sub-network which allows detection of a noise area; and

a defective image-based online supervised training step of, based on a defective image, training a reconstructive sub-network which inputs an inspection target image to generate a flaw removal image, and a discriminative sub-network which extracts a flaw area image by comparing a difference between the inspection target image and the flaw removal image,

wherein the defective image-based online supervised training step comprises:

a reconstructive sub-network calculating step of inputting the inspection target image to generate the flaw removal image;

a discriminative sub-network calculating step of extracting the flaw area image by comparing the difference between the inspection target image and the flaw removal image; and

a defective product detecting and training step of calculating whether the flaw area image is defective, detect a defective product and train based on a threshold to perform binarization of the flaw area image, and calculate a difference between a binarized image and the flaw area image to correct a weight of the discriminative sub-network.

14 . The method of claim 13 , wherein the reconstructive sub-network calculating step comprises:

a flaw removal image generating step of receiving the inspection target image as an input to generate the flaw removal image; and

an image concatenating step of concatenating the inspection target image and the flaw removal image.

15 . The method of claim 13 , wherein the discriminative sub-network calculating step comprises:

a flaw area image generating step of receiving an inspection target image and a flaw removal image concatenated as an input to generate a flaw area image of the same size as a product image through discriminative sub-network calculation; and

a defect calculating step of calculating whether the flaw area image is defective.

16 . The method of claim 13 , wherein the defective product detecting and training step comprises:

an image binarization step of detecting a defective product and train based on a threshold to perform binarization of the flaw area image;

an image difference calculating step of calculating a difference between a binarized image and the flaw area image; and

a weight correcting step of correcting the weight of the discriminative sub-network based on the calculated difference between the binarized image and the flaw area image.