IP Library › Granted Patent US 12,315,176
Granted Patent B2
US 12,315,176 · App. 17/720,195 · Granted May 27, 2025

Devices, systems, and methods for anomaly detection

Inventors: Bradley Scott Denney (Irvine, CA); Nikhil Krishnan (Bangalore, IN)
Assignee: Canon Virginia, Inc.
G06T7/337G06T7/0002G06T2207/20224
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,315,176
App. No.
17/720,195
Granted
May 27, 2025
Kind
B2
Abstract

Some devices, systems, and methods obtain training images; select a first reference image and a second reference image from the training images; generate a first set of aligned training images, wherein generating the first set of aligned training images includes aligning the training images to the first reference image; generate a first anomaly-detection model based on the first set of aligned training images; generate a second set of aligned training images, wherein generating the second set of aligned training images includes aligning the training images to the second reference image; and generate a second anomaly-detection model based on the second set of aligned training images.

Claims (80)

1. A device comprising:

one or more computer-readable storage media; and

one or more processors in communication with the one or more computer-readable storage media to cause the device to perform operations comprising:

obtaining training images;

selecting a first reference image and a second reference image from the training images;

generating a first set of aligned training images, wherein generating the first set of aligned training images includes aligning the training images to the first reference image;

generating a first anomaly-detection model based on the first set of aligned training images;

generating a second set of aligned training images, wherein generating the second set of aligned training images includes aligning the training images to the second reference image;

generating a second anomaly-detection model based on the second set of aligned training images;

obtaining a test image that is defined in a test image space;

aligning the test image to the first reference image, thereby generating a first aligned test image;

generating a first error map based on the first aligned test image and on the first anomaly-detection model;

aligning the first error map to an image space of a specified image, thereby generating a first aligned error map;

aligning the test image to the second reference image, thereby generating a second aligned test image;

generating a second error map based on the second aligned test image and on the second anomaly-detection model;

aligning the second error map to the image space of the specified image, thereby generating a second aligned error map; and

generating a composite error map based on the first aligned error map and on the second aligned error map.

2. The device of claim 1 , wherein the specified image is the test image and the image space of the specified image is the test image space.

3. The device of claim 1 , wherein the operations further comprise:

selecting a third reference image from the training images;

generating a third set of aligned training images, wherein generating the third set of aligned training images includes aligning the training images to the third reference image; and

generating a third anomaly-detection model based on the third set of aligned training images.

4. The device of claim 1 , wherein generating the first anomaly-detection model based on the first set of aligned training images includes:

selecting, for each target pixel of a plurality of target pixels in the training images, one or more respective associated pixels of the target pixel; and

calculating, for each target pixel of the plurality of target pixels, respective differences between a value of the target pixel and one or more derived values of the associated pixels of the target pixel.

5. The device of claim 4 , wherein each of the one or more derived values of the associated pixels is a value of one of the associated pixels.

6. The device of claim 4 , wherein each of the one or more derived values of the associated pixels is based on a plurality of values of the associated pixels.

7. The device of claim 4 , wherein generating the first anomaly-detection model based on the first set of aligned training images further includes:

calculating, for each target pixel of the plurality of target pixels, one or both of:

a centrality of the respective differences between the value of the target pixel and the one or more derived values of the associated pixels of the target pixel, and

a deviation of the respective differences between the value of the target pixel and the one or more derived values of the associated pixels of the target pixel.

8. A method comprising:

obtaining training images;

selecting a first reference image and a second reference image from the training images;

generating a first set of aligned training images, wherein generating the first set of aligned training images includes aligning the training images to the first reference image;

generating a first anomaly-detection model based on the first set of aligned training images;

generating a second set of aligned training images, wherein generating the second set of aligned training images includes aligning the training images to the second reference image;

generating a second anomaly-detection model based on the second set of aligned training images;

obtaining a test image that is defined in a test image space;

aligning the test image to the first reference image, thereby generating a first aligned test image;

generating a first error map based on the first aligned test image and on the first anomaly-detection model;

aligning the first error map to an image space of a specified image, thereby generating a first aligned error map;

aligning the test image to the second reference image, thereby generating a second aligned test image;

generating a second error map based on the second aligned test image and on the second anomaly-detection model;

aligning the second error map to the image space of the specified image, thereby generating a second aligned error map; and

generating a composite error map based on the first aligned error map and on the second aligned error map.

9. The method of claim 8 , wherein the specified image is the test image and the image space of the specified image is the test image space.

10. The method of claim 8 , wherein aligning the first error map to the image space of the specified image includes warping the first error map, and

wherein aligning the second error map to the image space of the specified image includes warping the second error map.

11. The method of claim 8 , wherein generating the first anomaly-detection model based on the first set of aligned training images includes:

selecting, for each target pixel of a plurality of target pixels in the training images, one or more respective associated pixels of the target pixel; and

calculating, for each target pixel of the plurality of target pixels, respective differences between a value of the target pixel and one or more derived values of the associated pixels of the target pixel.

12. The method of claim 11 , wherein each of the one or more derived values of the associated pixels is a value of one of the associated pixels.

13. The method of claim 11 , wherein each of the one or more derived values of the associated pixels is based on a plurality of values of the associated pixels.

14. The method of claim 11 , wherein generating the first anomaly-detection model based on the first set of aligned training images further includes:

calculating, for each target pixel of the plurality of target pixels, one or both of:

a centrality of the respective differences between the value of the target pixel and the one or more derived values of the associated pixels of the target pixel, and

a deviation of the respective differences between the value of the target pixel and the one or more derived values of the associated pixels of the target pixel.

15. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:

obtaining training images;

selecting a first reference image and a second reference image from the training images;

generating a first set of aligned training images, wherein generating the first set of aligned training images includes aligning the training images to the first reference image;

generating a first anomaly-detection model based on the first set of aligned training images;

generating a second set of aligned training images, wherein generating the second set of aligned training images includes aligning the training images to the second reference image;

generating a second anomaly-detection model based on the second set of aligned training images;

obtaining a test image that is defined in a test image space;

aligning the test image to the first reference image, thereby generating a first aligned test image;

generating a first error map based on the first aligned test image and on the first anomaly-detection model;

aligning the first error map to an image space of a specified image, thereby generating a first aligned error map;

aligning the test image to the second reference image, thereby generating a second aligned test image;

generating a second error map based on the second aligned test image and on the second anomaly-detection model;

aligning the second error map to the image space of the specified image, thereby generating a second aligned error map; and

generating a composite error map based on the first aligned error map and on the second aligned error map.

16. The one or more non-transitory computer-readable media of claim 15 , wherein the specified image is the test image and the image space of the specified image is the test image space.

17. The one or more non-transitory computer-readable media of claim 15 , wherein generating the composite error map based on the first aligned error map and on the second aligned error map includes quantizing values in the first aligned error map and values in the second aligned error map.

18. The one or more non-transitory computer-readable media of claim 15 , wherein generating the first anomaly-detection model based on the first set of aligned training images includes:

selecting, for each target pixel of a plurality of target pixels in the training images, one or more respective associated pixels of the target pixel; and

calculating, for each target pixel of the plurality of target pixels, respective differences between a value of the target pixel and one or more derived values of the associated pixels of the target pixel.

19. The one or more non-transitory computer-readable media of claim 18 , wherein each of the one or more derived values of the associated pixels is a value of one of the associated pixels.

20. The one or more non-transitory computer-readable media of claim 18 , wherein each of the one or more derived values of the associated pixels is based on a plurality of values of the associated pixels.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2022
From: DENNEY, BRADLEY SCOTT; KRISHNAN, NIKHIL
To: CANON VIRGINIA, INC.
Reel/Frame 060570/0878 →
Continuity (2)
Provisional Application 63174947 · Apr 14, 2021
Related Publication 20220335631A1 · Oct 20, 2022
References Cited (64)
US 6005978A · Garakani · 1999 [cited by applicant]
US 6320976B1 · Murthy · 2001 [cited by applicant]
US 7318005B1 · Smaragdis · 2008 [cited by applicant]
US 7440607B1 · Lin · 2008 [cited by applicant]
US 7715623B2 · Ling · 2010 [cited by applicant]
US 8107717B2 · Maeda · 2012 [cited by applicant]
US 8131107B2 · Sun · 2012 [cited by applicant]
US 8238635B2 · Can · 2012 [cited by applicant]
US 8345949B2 · Zhao · 2013 [cited by applicant]
US 8442301B2 · Dragovich · 2013 [cited by applicant]
US 8867787B2 · Singamsetti · 2014 [cited by applicant]
US 9170543B2 · Arakawa · 2015 [cited by applicant]
US 9256920B1 · Choudhury · 2016 [cited by applicant]
US 9378552B2 · Sato · 2016 [cited by applicant]
US 9400256B2 · Henderkott · 2016 [cited by applicant]
US 9405991B2 · Sharma · 2016 [cited by applicant]
US 9460490B2 · Choudhury · 2016 [cited by applicant]
US 9619755B2 · Wang · 2017 [cited by applicant]
US 10410084B2 · Huang · 2019 [cited by applicant]
US 10789697B2 · Krishnan · 2020 [cited by applicant]
US 10997462B2 · Krishnan · 2021 [cited by applicant]
US 10997712B2 · Cao · 2021 [cited by applicant]
US 20020088952A1 · Rao · 2002 [cited by examiner]
US 20060245635A1 · Ishikawa · 2006 [cited by applicant]
US 20080101686A1 · Sali · 2008 [cited by examiner]
US 20090238432A1 · Can · 2009 [cited by applicant]
US 20090279772A1 · Sun · 2009 [cited by applicant]
US 20130148987A1 · Arakawa · 2013 [cited by applicant]
US 20140270489A1 · Lim · 2014 [cited by applicant]
US 20140348415A1 · Bhattad · 2014 [cited by applicant]
US 20150227809A1 · Alpert · 2015 [cited by applicant]
US 20170140516A1 · Maher · 2017 [cited by applicant]
US 20180114092A1 · Huang · 2018 [cited by examiner]
US 20190066291A1 · Martin · 2019 [cited by examiner]
US 20190220965A1 · Cao · 2019 [cited by examiner]
US 20190285980A1 · Yoshikawa · 2019 [cited by applicant]
US 20190311224A1 · Krishnan · 2019 [cited by examiner]
US 20200151496A1 · Cao · 2020 [cited by applicant]
US 20210133989A1 · Bhattacharyya · 2021 [cited by examiner]
US 20210209418A1 · Badanes · 2021 [cited by examiner]
US 20220036525A1 · Xin · 2022 [cited by examiner]
US 20220222803A1 · Lin · 2022 [cited by examiner]
JP 2015108522A · 2015 [cited by applicant]
David W. J. Stein et al., Anomaly Detection from Hyperspectral Imagery, IEEE Signal Processing Magazine, Jan. 2002. [cited by applicant]
M. Haselmann et al., Pixel-Wise Defect Detection by CNNs without Manually Labeled Training Data, Applied Artificial Intelligence, Mar. 2019. [cited by applicant]
Rene Heideklang et al., Application of Data Fusion in Nondestructive Testing (NDT), Jul. 2013. [cited by applicant]
Xavier Gros et al., Pixel Level NDT Data Fusion, Jul. 1999. [cited by applicant]
J. Hassan et al., Welding Defect Detection and Classification Using Geometric Features, Dec. 2012. [cited by applicant]
Nick Kingsbury, Complex Wavelets for Shift Invariant Analysis and Filtering of Signals, May 2001. [cited by applicant]
Roumen Kountchev et al., Defects detection in X-ray images and photos, Feb. 2011. [cited by applicant]
V Lashkia, Defect detection in X-ray images using fuzzy reasoning, Image and Vision Computing, vol. 19, Issue 5, Apr. 2001, Abstract. [cited by applicant]
Shekhar B. Sastry, Computationally Efficient Methods for Shift-variant Image Restoration in Two and Three Dimensions, Sep. 2011. [cited by applicant]
Ian T. Young et al., Fundamentals of Image Processing, Jul. 2004. [cited by applicant]
Tian Yuan et al., Automatic Defect Detection in X-Ray Images Using Image Data Fusion, Dec. 2006. [cited by applicant]
Caixia Zheng et al., An improved method for object detection in astronomical images, Jun. 2015. [cited by applicant]
Richard Szeliski, Computer Vision: Algorithms and Applications, pp. i-73, Sep. 2010. [cited by applicant]
Richard Szeliski, Computer Vision: Algorithms and Applications, pp. 73-174, Sep. 2010. [cited by applicant]
Richard Szeliski, Computer Vision: Algorithms and Applications, pp. 174-266, Sep. 2010. [cited by applicant]
Richard Szeliski, Computer Vision: Algorithms and Applications, pp. 267-363, Sep. 2010. [cited by applicant]
Richard Szeliski, Computer Vision: Algorithms and Applications, pp. 363-466, Sep. 2010. [cited by applicant]
Richard Szeliski, Computer Vision: Algorithms and Applications, pp. 467-575, Sep. 2010. [cited by applicant]
Richard Szeliski, Computer Vision: Algorithms and Applications, pp. 577-679, Sep. 2010. [cited by applicant]
Richard Szeliski, Computer Vision: Algorithms and Applications, pp. 679-776, Sep. 2010. [cited by applicant]
Richard Szeliski, Computer Vision: Algorithms and Applications, pp. 777-789 and 933-957, Sep. 2010. [cited by applicant]