Identifying anomaly location
A region of interest is extracted from a captured image of a physical object. An autoencoder model is applied to the extracted region of interest to reconstruct the region of interest. A location of an anomaly of the physical object within the extracted region of interest, if any, is identified based on the extracted and reconstructed regions of interest.
1 . A non-transitory computer-readable data storage medium storing program code executable by a processor to perform processing comprising:
extracting a region of interest from a captured image of a physical object, wherein the region of interest is a prespecified area within the captured image that is to be input to an autoencoder model;
applying the autoencoder model to the extracted region of interest to reconstruct the region of interest; and
identifying a location of an anomaly of the physical object within the extracted region of interest, if any, based on the extracted and reconstructed regions of interest, wherein identifying the location of the anomaly of the physical object within the extracted region of interest, if any, comprises generating a residual map between the extracted region of interest and the reconstructed region of interest.
2 . The non-transitory computer-readable data storage medium of claim 1 , wherein extracting the region of interest comprises:
aligning the captured image of the physical object against a reference image of another physical object of a same type as the physical object; and
cropping the aligned captured image based on a bounding box identifying a corresponding region of interest within the reference image,
wherein the cropped aligned captured image constitutes the region of interest.
3 . The non-transitory computer-readable data storage medium of claim 2 , wherein aligning the captured image against the reference image comprises calculating a transformation matrix that aligns the captured image to the reference image, and
wherein cropping the aligned captured image based on the bounding box comprises applying an inverse of the transformation matrix to the bounding box and cropping the captured image using the inverse-applied bounding box.
4 . The non-transitory computer-readable data storage medium of claim 1 , wherein extracting the region of interest comprises:
applying an object segmentation model to the captured image.
5 . The non-transitory computer-readable data storage medium of claim 4 , wherein the object segmentation model is a regional convolutional neural network (R-CNN) trained using a plurality of training images of other physical objects of a same type as the physical object and on which corresponding regions of interest have been preidentified.
6 . The non-transitory computer-readable data storage medium of claim 1 , wherein the autoencoder model is trained on training images of anomaly-free physical objects of a same type as the physical object.
7 . The non-transitory computer-readable data storage medium of claim 1 , wherein identifying the location of the anomaly of the physical object within the extracted region of interest, if any, further comprises:
removing any pixel of the residual map having a value less than a threshold.
8 . The non-transitory computer-readable data storage medium of claim 7 , wherein the threshold is a static threshold.
9 . The non-transitory computer-readable data storage medium of claim 7 , wherein the threshold is an adaptive threshold calculated as a mean of values of pixels of the residual map, plus a product of a parameter and a standard deviation of the values of the pixels of the residual map, the parameter governing reconstruction error severity that is considered anomalous.
10 . The non-transitory computer-readable data storage medium of claim 7 , wherein identifying the location of the anomaly of the physical object within the extracted region of interest, if any, further comprises:
after removing any pixel having a value less than the threshold, applying a morphological operation to the residual map.
11 . The non-transitory computer-readable data storage medium of claim 10 , wherein applying the morphological operation to the residual map comprises:
applying an opening morphological operation to denoise the residual map of extraneous pixels having values greater than the threshold; and
applying a closing morphological operation to connectively group discontiguous pixels having values greater than the threshold.
12 . The non-transitory computer-readable data storage medium of claim 7 , wherein identifying the location of the anomaly of the physical object within the extracted region of interest, if any, further comprises:
determining that the physical object includes the anomaly if the residual map includes any pixels having values greater than the threshold,
wherein the location of the anomaly of the physical object within the extracted region corresponds to locations of pixels having values greater than the threshold within the residual map.
13 . The non-transitory computer-readable data storage medium of claim 12 , wherein identifying the location of the anomaly of the physical object within the extracted region of interest, if any, further comprises:
determining that the physical object does not include any anomaly if the residual map does not include any pixels having values greater than the threshold.
14 . A method comprising:
training an autoencoder model using a plurality of training images of anomaly-free physical objects of a same type; and
using the autoencoder model to identify a location of an anomaly of a physical object of the same type as the anomaly-free physical objects within an extracted region of interest of a captured image of the physical object, wherein the region of interest is a prespecified area within the captured image that is to be input to the autoencoder model, wherein using the autoencoder model to identify the location of the anomaly of the physical object within the extracted region of interest comprises generating a residual map between the extracted region of interest and a reconstructed region of interest generated by the autoencoder model based on the captured image.
15 . A computing device comprising:
a processor; and
a memory storing instructions executable by the processor to:
preprocess an image of a physical object to crop the image to extract a region of interest, wherein the region of interest is a prespecified area within the captured image that is to be input to an autoencoder model;
apply the autoencoder model to the preprocessed image to generate a reconstructed image; and
identify a location of an anomaly of the physical object within the image, if any, based on a residual map between the preprocessed image and the reconstructed image.
16 . The method of claim 14 , wherein using the autoencoder model to identify the location of the anomaly of the physical object within the extracted region of interest further comprises:
removing any pixel of the residual map having a value less than a threshold.
17 . The method of claim 16 , wherein the threshold is an adaptive threshold calculated as a mean of values of pixels of the residual map, plus a product of a parameter and a standard deviation of the values of the pixels of the residual map, the parameter governing reconstruction error severity that is considered anomalous.
18 . The method of claim 16 , wherein using the autoencoder model to identify the location of the anomaly of the physical object within the extracted region of interest further comprises:
after removing any pixel having a value less than the threshold, applying a morphological operation to the residual map.
19 . The computing device of claim 15 , wherein the processor is to preprocess the image of the physical object by:
extracting a region of interest from the image of the physical object.
20 . The computing device of claim 19 , wherein the processor is to extract the region of interest by:
applying an object segmentation model to the image of the physical object.