Generating minority class defect detection data from visual inspection dataset using self-supervised defect generator
Methods, systems, and computer-readable storage media for a defect detection system that generates synthetic defect data that is representative of real-world defects in products. The synthetic defect data is included in training data for training a defect detection model. The defect detection model is deployed to a production environment to detect defects in products.
1 . A computer-implemented method for detecting defects in products, the method being executed by one or more processors and comprising:
generating a first set of context images from a set of positive images, each positive image representing one or more defects occurring in products, each context image in the first set of context images comprising at least one cropped area cropping out a defect represented in a respective positive image;
training a defect generation model using the set of context images, the defect generation model being trained to inpaint synthetic defects within cropped areas;
generating a second set of context images from a set of negative images, each negative image representing a product absent any defect, each context image in the second set of context images comprising at least one cropped area cropping representing a target defect location;
providing a set of synthetic defect data by processing the second set of context images through the defect generation model, for each context image in the second set of context images, the defect generation model inpainting a synthetic defect in the at least one cropped area to provide a synthetic defect image;
training a defect detection model using training data that comprises at least a portion of the set of synthetic defect data; and
executing defect detection of products by processing images of the products through the defect detection model.
2 . The method of claim 1 , wherein the context images in the first set of context images are generated based on a text embedding that is representative of text that indicates a target defect class.
3 . The method of claim 1 , wherein the defect generation model is trained based on a text embedding that is representative of text that indicates a target defect class.
4 . The method of claim 1 , wherein generating a second set of context images from a set of negative images comprises, for each negative image providing at least one bounding box indicating a location within the negative image that a defect is to be inpainted and cropping an area of the bounding box to provide the at least one cropped area.
5 . The method of claim 1 , wherein providing a set of synthetic defect data by processing the second set of context images through the defect generation model comprises providing a text embedding that is representative of text that indicates a target defect class to the defect generation model, the defect generation model inpainting a synthetic defect corresponding to the target defect class in each of the at least one cropped areas of the context images of the second set of context images.
6 . The method of claim 1 , wherein the set of synthetic defect data comprises synthetic defect images representative of multiple defect classes.
7 . The method of claim 1 , wherein the training data further comprises at least a portion of the set of negative images.
8 . A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for detecting defects in products, the operations comprising:
generating a first set of context images from a set of positive images, each positive image representing one or more defects occurring in products, each context image in the first set of context images comprising at least one cropped area cropping out a defect represented in a respective positive image;
training a defect generation model using the set of context images, the defect generation model being trained to inpaint synthetic defects within cropped areas;
generating a second set of context images from a set of negative images, each negative image representing a product absent any defect, each context image in the second set of context images comprising at least one cropped area cropping representing a target defect location;
providing a set of synthetic defect data by processing the second set of context images through the defect generation model, for each context image in the second set of context images, the defect generation model inpainting a synthetic defect in the at least one cropped area to provide a synthetic defect image;
training a defect detection model using training data that comprises at least a portion of the set of synthetic defect data; and
executing defect detection of products by processing images of the products through the defect detection model.
9 . The non-transitory computer-readable storage medium of claim 8 , wherein the context images in the first set of context images are generated based on a text embedding that is representative of text that indicates a target defect class.
10 . The non-transitory computer-readable storage medium of claim 8 , wherein the defect generation model is trained based on a text embedding that is representative of text that indicates a target defect class.
11 . The non-transitory computer-readable storage medium of claim 8 , wherein generating a second set of context images from a set of negative images comprises, for each negative image providing at least one bounding box indicating a location within the negative image that a defect is to be inpainted and cropping an area of the bounding box to provide the at least one cropped area.
12 . The non-transitory computer-readable storage medium of claim 8 , wherein providing a set of synthetic defect data by processing the second set of context images through the defect generation model comprises providing a text embedding that is representative of text that indicates a target defect class to the defect generation model, the defect generation model inpainting a synthetic defect corresponding to the target defect class in each of the at least one cropped areas of the context images of the second set of context images.
13 . The non-transitory computer-readable storage medium of claim 8 , wherein the set of synthetic defect data comprises synthetic defect images representative of multiple defect classes.
14 . The non-transitory computer-readable storage medium of claim 8 , wherein the training data further comprises at least a portion of the set of negative images.
15 . A system, comprising:
a computing device; and
a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for detecting defects in products, the operations comprising:
generating a first set of context images from a set of positive images, each positive image representing one or more defects occurring in products, each context image in the first set of context images comprising at least one cropped area cropping out a defect represented in a respective positive image;
training a defect generation model using the set of context images, the defect generation model being trained to inpaint synthetic defects within cropped areas;
generating a second set of context images from a set of negative images, each negative image representing a product absent any defect, each context image in the second set of context images comprising at least one cropped area cropping representing a target defect location;
providing a set of synthetic defect data by processing the second set of context images through the defect generation model, for each context image in the second set of context images, the defect generation model inpainting a synthetic defect in the at least one cropped area to provide a synthetic defect image;
training a defect detection model using training data that comprises at least a portion of the set of synthetic defect data; and
executing defect detection of products by processing images of the products through the defect detection model.
16 . The system of claim 15 , wherein the context images in the first set of context images are generated based on a text embedding that is representative of text that indicates a target defect class.
17 . The system of claim 15 , wherein the defect generation model is trained based on a text embedding that is representative of text that indicates a target defect class.
18 . The system of claim 15 , wherein generating a second set of context images from a set of negative images comprises, for each negative image providing at least one bounding box indicating a location within the negative image that a defect is to be inpainted and cropping an area of the bounding box to provide the at least one cropped area.
19 . The system of claim 15 , wherein providing a set of synthetic defect data by processing the second set of context images through the defect generation model comprises providing a text embedding that is representative of text that indicates a target defect class to the defect generation model, the defect generation model inpainting a synthetic defect corresponding to the target defect class in each of the at least one cropped areas of the context images of the second set of context images.
20 . The system of claim 15 , wherein the set of synthetic defect data comprises synthetic defect images representative of multiple defect classes.