Granted Patent B2

US 12,333,705 · App. 17/868,044 · Granted Jun 17, 2025

Inventors: Felix Rothmund (Munich, DE); Simran Agarwal (Munich, DE); Leslie Casas (Munich, DE); Keng Chai (Munich, DE); Markus Bößl (Munich, DE); Shweta Mahajan (Munich, DE); Jonathan Pirnay (Munich, DE); Jochen Riedisser (Munich, DE)

Assignee: Fujitsu Technology Solutions GmbH

G06T7/0008G06T7/0012G06V10/26G06V10/764G06V10/7747G06V10/82G06T2207/20081G06T2207/20084G06T2207/30004G06T2207/30108G06V2201/06

Abstract

Claims (34)

1. A method of analyzing a product, the method comprising:

receiving an image of the product;

performing an anomaly detection on the received image using an inpainting autoencoder, wherein the inpainting autoencoder comprises at least one first neural network trained based on a first set of training images, and the first set of training images comprises a plurality of training images each showing a corresponding defect-free product;

determining, using a binary classifier, whether or not a defect is present based on a result of the anomaly detection;

performing defect detection on the received image using a defect detector, wherein the defect detector comprises at least one third neural network trained based on at least one third set of training images, and the at least one third set of training images comprises a plurality of training images each showing a corresponding defective product; and

evaluating a result of the analysis of the received image based on a weighting of the results of the anomaly detection, the defect detection, and the binary classifier.

2. The method according to claim 1 , wherein to perform the anomaly detection, the received image is divided into at least two sub-images, the anomaly detection is performed on the individual sub-images, and the binary classifier is applied to the results of the anomaly detection of the individual sub-images.

3. The method according to claim 1 , wherein the defect detection further comprises determining a location of a detected defect in the received image and/or a type of the detected defect and/or a defect probability.

4. The method according to claim 1 , wherein, in determining, using the binary classifier, whether or not a defect is present based on a result of the anomaly detection, the binary classifier evaluates, based on a reconstruction error of a version of the received image reconstructed by means of the inpainting autoencoder, whether the product represented in the received image is to be evaluated as defective or defect-free.

5. The method according to claim 1 , wherein the binary classifier comprises at least a second neural network trained based on a second set of training images, and the second set of training images comprises results of the inpainting autoencoder; and

the method further comprises: generating an attention map of the at least one second neural network when applying the binary classifier.

6. A method of training an apparatus for analyzing a product, the method comprising:

creating a first set of training images, the first set of training images comprising a plurality of training images each showing a defect-free product;

training at least a first neural network of an inpainting autoencoder based on the first set of training images;

creating a binary classifier configured to identify whether or not a defect is present based on a result of an anomaly detection using the inpainting autoencoder;

creating at least one third set of training images, wherein the at least one third set of training images comprises a plurality of training images each showing a defective product, and further at least one defect is marked in each training image of the third set of training images;

training at least one third neural network of a defect identifier based on the at least one third set of training images; and

determining a weighting by which to evaluate results of an anomaly detection by the inpainting autoencoder, a defect detection by the defect detector, and an application of the binary classifier.

7. The method to according claim 6 , wherein creating the binary classifier comprises:

creating a second set of training images, wherein the second set of training images comprises results of the inpainting autoencoder; and

training at least a second neural network of the binary classifier based on the second set of training images.

8. The method according to claim 6 , further comprising:

creating a further third set of training images, wherein the further third set of training images comprises both a plurality of training images each showing a defective product and a plurality of training images each showing a defect-free product, and training the at least one third neural network is additionally performed based on the further third set of training images.

9. The method according to claim 6 , further comprising:

creating a still further third set of training images, wherein the still further third set of training images comprises a plurality of training images each showing a defect-free product detected as defective by the previously trained at least one third neural network, and wherein the training of the at least one third neural network is additionally performed based on the still further third set of training images.

10. The method according to claim 6 , wherein training the at least one first neural network of the inpainting autoencoder based on the first set of training images further comprises:

patchwise reconstructing a training image of the first set of training images;

adding, for a plurality of reconstructed patches, an edge region from a corresponding portion of the original training image; and

training the at least one first neural network of the inpainting autoencoder using the reconstructed patches of the training image, taking into account the added edge regions.

11. The method according to claim 1 , wherein the product is a medical product.

12. The method according to claim 6 , wherein the product is a medical product.

13. An apparatus that analyzes a product, wherein the apparatus is adapted to perform the method according to claim 1 .

14. The apparatus according to claim 13 , wherein the apparatus further comprises an imaging device that captures the image of the product; and/or the apparatus further comprises a control device adapted to cause a sorting out of a product detected as defective.

15. The apparatus according to claim 13 , adapted to be retrofitted to a semi-automatic inspection device.

Assignments (3)

Priority Claims (1)

Continuity (1)

References Cited (35)