Industrial defect recognition method and system, computing device, and storage medium
View Patent ↗A target region is first extracted from a to-be-recognized image, and then a defect coarse-selection region including an industrial defect is obtained from the target region. Through two times of region extraction, a proportion of a size of the industrial defect to a size of a background is increased, so that a probability of detecting a small-sized industrial defect can be increased. After the defect coarse-selection region including the industrial defect is detected, localization and size estimation are further performed on the industrial defect in the defect coarse-selection region.
1. A method, comprising:
obtaining a to-be-recognized image, wherein the to-be-recognized image is of an industrial product;
extracting, based on a target region detection model, at least one target region from the to-be-recognized image, wherein the at least one target region comprises a first industrial defect;
obtaining, based on the at least one target region and a defect detection model, a defect coarse-selection region and a first defect type of the first industrial defect, wherein the defect coarse-selection region is in a first target region of the at least one target region, and wherein the defect coarse-selection region comprises the first industrial defect; and
determining a size and a first location of the first industrial defect.
2. The method of according to claim 1 , wherein obtaining the defect coarse-selection region comprises:
using the at least one target region as an input of the defect detection model, wherein the defect detection model comprises an artificial intelligence (AI) model trained by using a defect training sample;
performing, based on the defect detection model, an inference to obtain a second location of the defect coarse-selection region in the target region and to obtain the first defect type; and
extracting, based on the second location, the defect coarse-selection region.
3. The method of to claim 1 , wherein determining the size and the first location comprises:
determining, based on a grayscale distribution in the defect coarse-selection region, the size and the first location; and
determining, based on the first location, a second location of the defect coarse-selection region in the first target region, and a third location of the first target region in the to-be-recognized image, a fourth location of the first industrial defect a location, in the to-be-recognized image.
4. The method of claim 3 , further comprising:
generating a detection report, wherein the detection report comprises the first defect type, the size, and the fourth location; and
providing, for a user, providing the detection report.
5. The method of claim 1 , wherein the first industrial defect comprises a weld seam defect, and wherein the first target region comprises a weld seam.
6. The method of claim 1 , further comprising:
obtaining a plurality of industrial training samples, wherein each industrial training sample corresponds to target region annotation information, and wherein the target region annotation information comprises location information of a multi-degree-of-freedom quadrilateral annotated in the corresponding industrial training sample and annotation category information indicating that the corresponding multi-degree-of-freedom quadrilateral is a second target region; and
training, based on the plurality of industrial training samples and the target region annotation information, a target region detection network to obtain the target region detection model.
7. The method of claim 1 , further comprising:
obtaining a plurality of defect training samples;
extracting, based on the target region detection model, a plurality of second target regions;
obtaining defect annotation information of a third target region that is in the plurality of second target regions and that comprises a second industrial defect, wherein the defect annotation information indicates a second defect type and a second location of the second industrial defect in the third target region; and
training, based on the third target region, training a defect detection network to obtain the defect detection model.
8. The method of claim 1 , wherein the target region detection model uses a multi-degree-of-freedom quadrilateral as a detection box, and wherein an angle between each edge of the multi-degree-of-freedom quadrilateral and each edge of the to-be-recognized image comprises any degree from 0 degrees to 180 degrees.
9. An apparatus, comprising:
a memory configured to store instructions; and
one or more processors coupled to the memory and configured to execute the instructions to:
obtain a to-be-recognized image, wherein the to-be-recognized image is of an industrial product;
extract, based on a target region detection model, at least one target region from the to-be-recognized image, wherein the at least one target region comprises a first industrial defect;
obtain, based on the at least one target region and a defect detection model, a defect coarse-selection region and a first defect type of the first industrial defect, wherein the defect coarse-selection region is in a first target region of the at least one target region, and wherein the defect coarse-selection region comprises the first industrial defect; and
determine a size and a first location of the first industrial defect.
10. The apparatus of claim 9 , wherein the one or more processors are further configured to execute the instructions to:
use the at least one target region as an input of the defect detection model, wherein the defect detection model comprises an artificial intelligence (AI) model trained by using a defect training sample;
perform, based on the defect detection model, inference to obtain a second location of the defect coarse-selection region in the target region and the first defect type; and
extract, based on the second location, the defect coarse-selection region.
11. The apparatus of claim 9 , wherein the one or more processors are further configured to execute the instructions to:
determine, based on a grayscale distribution in the defect coarse-selection region, the size of the first location; and
determine, based on the first location, a second location of the defect coarse-selection region in the first target region, and a third location of the first target region in the to-be-recognized image, a fourth location of the first industrial defect in the to-be-recognized image.
12. The apparatus of claim 11 , wherein the one or more processors are further configured to execute the instructions to:
generate a detection report, wherein the detection report comprises the first defect type, the size, and the fourth location; and
provide, for a user, the detection report.
13. The apparatus of claim 9 , wherein the first industrial defect comprises a weld seam defect, and wherein the first target region comprises a weld seam.
14. The apparatus of claim 9 , wherein the one or more processors are further configured to execute the instructions to:
obtain a plurality of industrial training samples, wherein each industrial training sample corresponds to target region annotation information, and wherein the target region annotation information comprises location information of a multi-degree-of-freedom quadrilateral annotated in the corresponding industrial training sample and annotation category information indicating that the corresponding multi-degree-of-freedom quadrilateral is a second target region; and
train, based on the plurality of industrial training samples and the target region annotation information, a target region detection network to obtain the target region detection model.
15. The apparatus of claim 9 , wherein the one or more processors are further configured to:
obtain a plurality of defect training samples;
extract, based on the target region detection model, a plurality of second target regions;
obtain defect annotation information of a third target region that is in the plurality of second target regions and that comprises a second industrial defect, wherein the defect annotation information indicates a second defect type and a second location of the second industrial defect in the third target region; and
train, based on the third target region, a defect detection network to obtain the defect detection model.
16. The apparatus of claim 9 , wherein the target region detection model uses a multi-degree-of-freedom quadrilateral as a detection box, and wherein an angle between each edge of the multi-degree-of-freedom quadrilateral and each edge of the to-be-recognized image comprises is of any degree from 0 degrees to 180 degrees.
17. A computer program product comprising instructions stored on a non-transitory computer-readable medium that, when executed by one or more processors, cause an apparatus to:
obtain a to-be-recognized image, wherein the to-be-recognized image is of an industrial product;
extract, based on a target region detection model, at least one target region from the to-be-recognized image, wherein the at least one target region comprises a first industrial defect;
obtain, based on the at least one target region and a defect detection model, a defect coarse-selection region and a first defect type of the first industrial defect, wherein the defect coarse-selection region is in a first target region of the at least one target region, and wherein the defect coarse-selection region comprises the first industrial defect; and
determine a size and a first location of the first industrial defect in the defect coarse-selection region.
18. The computer program product of claim 17 , wherein the one or more processors are further configured to execute the instructions to cause the apparatus to:
use the at least one target region as an input of the defect detection model, wherein the defect detection model comprises an artificial intelligence (AI) model trained by using a defect training sample;
perform, based on the defect detection model, inference to obtain a second location of the defect coarse-selection region in the target region and the first defect type; and
extract, based on the second location, the defect coarse-selection region.
19. The computer program product of claim 17 , wherein the one or more processors are further configured to execute the instructions to cause the apparatus to:
determine, based on a grayscale distribution in the defect coarse-selection region, the size and the first location; and
determine, based on the first location, a second location of the defect coarse-selection region in the first target region, and a third location of the first target region in the to-be-recognized image, a fourth location of the first industrial defect in the to-be-recognized image.
20. The computer program product of claim 19 , wherein the one or more processors are further configured to execute the instructions to cause the apparatus to:
generate a detection report, wherein the detection report comprises the first defect type, the size, and the fourth location; and
provide, for a user, the detection report.