IP Library › Granted Patent US 11,748,865
Granted Patent B2
US 11,748,865 · App. 17/113,672 · Granted Sep 5, 2023

Hierarchical image decomposition for defect detection

Inventors: Florian Michael Scheidegger (Dietlikon, CH); Adelmo Cristiano Innocenza Malossi (Schonenberg, CH)
Assignee: International Business Machines Corporation
G06T7/0004G06T3/40G06T2207/20084G06T2207/30108
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,748,865
App. No.
17/113,672
Granted
Sep 5, 2023
Kind
B2
Abstract

Aspects of the present invention disclose a method, computer program product, and system for improving object detection in high-resolution images at inference time. The method includes one or more processors receiving a high-resolution image. The method further includes one or more processors decomposing the received image into hierarchically organized layers of images. Each layer comprises at least one image tile of the received image. Each of the image tiles have a corresponding resolution suitable to a baseline image recognition algorithm. The method further includes one or more processors applying the baseline algorithm to each of the image tiles of each layer. The method further includes one or more processors performing a result aggregation of results of the baseline algorithm applications to the image tiles of the layers.

Claims (53)

1. A method comprising:

receiving, by one or more processors, a high-resolution image;

decomposing, by one or more processors, the received image into hierarchically organized layers of images, wherein each layer comprises at least one image tile of the received image, and wherein each of the image tiles have a corresponding resolution suitable to a baseline image recognition algorithm;

applying, by one or more processors, the baseline algorithm to each of the image tiles of each layer; and

performing, by one or more processors, a result aggregation of results of the baseline algorithm applications to the image tiles of the layers, wherein performing the result aggregation of results of the baseline algorithm applications to the image tiles of the layers comprises:

aggregating, by one or more processors, results of the baseline algorithm per layer,

performing, by one or more processors, a pairwise layer comparison of results of the baseline algorithm to adjacent pairwise layers, and

performing, by one or more processors, a hierarchical aggregation of the baseline algorithm results depending on the pairwise layer comparison.

2. The method of claim 1 , wherein performing the result aggregation of results of the baseline algorithm applications to the image tiles of the layers further comprises:

extracting, by one or more processors, polygon encoding a shape of a recognized object; and

mapping, by one or more processors, local polygon coordinates used for the image tiles to global coordinates used for the image tile having the highest corresponding resolution, thereby compacting shapes of polygons of results of the baseline algorithm having a higher resolution, such that the compacted shapes are comparable to shapes in tiles having a lower resolution.

3. The method of claim 2 , further comprising:

eliminating, by one or more processors, overlapping areas between adjacent image tile borders of a respective layer; and

merging, by one or more processors, a detected partial object of the adjacent tiles into one detected object.

4. The method of claim 3 , wherein performing a pairwise layer comparison further comprises:

comparing, by one or more processors, compacted relating shapes of image tiles of adjacent layers; and

building, by one or more processors, an intersection of shapes based on the comparison of the compacted relating shapes, thereby building N intermediate image layers, wherein N is by 1 smaller than said number of hierarchically organized layers.

5. The method of claim 4 , wherein performing a hierarchical aggregation further comprises:

building, by one or more processors, a pixel-wise union of all N intermediate image layers, thereby building a final image of a resolution equivalent to the resolution of the received high-resolution image including a polygon encircling of the detected object.

6. The method of claim 1 , wherein the results of the baseline algorithm comprise at least one selected out of the group consisting of a class of a recognized item, a bounding box surrounding an identified object in an image tile, and a mask represented by a polygon encircling a shape of the recognized object in the image tile.

7. The method of claim 1 , wherein the baseline algorithm is a mask R-CNN (Region Based Convolutional Neural Network) algorithm or fast R-CNN algorithm.

8. The method of claim 1 , wherein the resolution suitable to a baseline image recognition algorithm is selected out of the group consisting of 224×224 pixels, 512×512 pixel, 800×800 pixels, and 1024×800 pixels.

9. The method of claim 1 , wherein the baseline algorithm is pre-trained such that a neural network model has been built for inference tasks for an object recognition.

10. The method of claim 1 , wherein the object to be detected is a material defect.

11. A computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising:

program instructions to receive a high-resolution image;

program instructions to decompose the received image into hierarchically organized layers of images, wherein each layer comprises at least one image tile of the received image, and wherein each of the image tiles have a corresponding resolution suitable to a baseline image recognition algorithm;

program instructions to apply the baseline algorithm to each of the image tiles of each layer; and

program instructions to perform a result aggregation of results of the baseline algorithm applications to the image tiles of the layers, wherein the program instructions to perform the result aggregation of results of the baseline algorithm applications to the image tiles of the layers comprise program instructions to:

aggregate results of the baseline algorithm per layer,

perform a pairwise layer comparison of results of the baseline algorithm to adjacent pairwise layers, and

perform a hierarchical aggregation of the baseline algorithm results depending on the pairwise layer comparison.

12. The computer program product of claim 11 , wherein the program instructions to perform the result aggregation of results of the baseline algorithm applications to the image tiles of the layers further comprise program instructions to:

extract polygon encoding a shape of a recognized object; and

map local polygon coordinates used for the image tiles to global coordinates used for the image tile having the highest corresponding resolution, thereby compacting shapes of polygons of results of the baseline algorithm having a higher resolution, such that the compacted shapes are comparable to shapes in tiles having a lower resolution.

13. The computer program product of claim 11 , wherein the baseline algorithm is a mask R-CNN (Region Based Convolutional Neural Network) algorithm or fast R-CNN algorithm.

14. A computer system comprising:

one or more computer processors;

one or more computer readable storage media; and

program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the program instructions comprising:

program instructions to receive a high-resolution image;

program instructions to decompose the received image into hierarchically organized layers of images, wherein each layer comprises at least one image tile of the received image, and wherein each of the image tiles have a corresponding resolution suitable to a baseline image recognition algorithm;

program instructions to apply the baseline algorithm to each of the image tiles of each layer; and

program instructions to perform a result aggregation of results of the baseline algorithm applications to the image tiles of the layers, wherein the program instructions to perform the result aggregation of results of the baseline algorithm applications to the image tiles of the layers comprise program instructions to:

aggregate results of the baseline algorithm per layer,

perform a pairwise layer comparison of results of the baseline algorithm to adjacent pairwise layers, and

perform a hierarchical aggregation of the baseline algorithm results depending on the pairwise layer comparison.

15. The computer system of claim 14 , wherein the program instructions to perform the result aggregation of results of the baseline algorithm applications to the image tiles of the layers further comprise program instructions to:

extract polygon encoding a shape of a recognized object; and

map local polygon coordinates used for the image tiles to global coordinates used for the image tile having the highest corresponding resolution, thereby compacting shapes of polygons of results of the baseline algorithm having a higher resolution, such that the compacted shapes are comparable to shapes in tiles having a lower resolution.

16. The computer system of claim 14 , wherein the baseline algorithm is a mask R-CNN (Region Based Convolutional Neural Network) algorithm or fast R-CNN algorithm.

17. The computer system of claim 14 , wherein the object to be detected is a material defect.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE DOCKET NUMBER. CORRECT DOCKET NUMBER IS P202005563US01 (NOT P2020556US01 AS PREVIOUSLY RECORDED). PREVIOUSLY RECORDED ON REEL 054566 FRAME 0290. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 14, 2020
From: SCHEIDEGGER, FLORIAN MICHAEL; MALOSSI, ADELMO CRISTIANO INNOCENZA
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 054729/0312 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2020
From: SCHEIDEGGER, FLORIAN MICHAEL; MALOSSI, ADELMO CRISTIANO INNOCENZA
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 054566/0290 →
Continuity (1)
Related Publication 20220180497A1 · Jun 9, 2022