Method and system for defect detection

1. A method for defect detection, the method comprising:

acquiring a two-dimensional (2D) picture of an object to be detected;

inputting the acquired 2D picture to a trained defect segmentation model to obtain a segmented 2D defect mask, wherein the defect segmentation model is trained based on a multi-level feature extraction instance segmentation network with intersection over union (IoU) thresholds being increased level by level, and the 2D defect mask comprises information about a defect type, a defect size, and a defect location of a segmented defect region; and

determining the segmented 2D defect mask based on a predefined defect rule to output a defect detection result,

wherein the multi-level feature extraction instance segmentation network is obtained by cascading at least three levels of instance segmentation networks.

2. The method of claim 1 , wherein the method further comprises:

acquiring a three-dimensional (3D) picture of the object to be detected;

preprocessing the acquired 3D picture to obtain an image with depth information of the object to be detected;

inputting the obtained image with depth information to the trained defect segmentation model to obtain a segmented 3D defect mask, wherein the 3D defect mask comprises information about a defect depth of a segmented defect region; and

performing, based on a predefined detect rule, fusion and determination on the segmented 2D defect mask and the segmented 3D defect mask to output a defect detection result.

3. The method of claim 2 , wherein the performing, based on a predefined defect rule, fusion and determination on the segmented 2D defect mask and the segmented 3D defect mask further comprises:

performing pixel-level alignment on the 2D picture and the 3D picture by using a coordinate transformation matrix between the acquired 2D picture and the acquired 3D picture, to obtain aligned pictures;

filling the segmented 2D defect mask and the segmented 3D defect mask into the aligned pictures; and

performing, based on the predefined defect rule, fusion and determination on the segmented 2D defect mask and the segmented 3D defect mask on the aligned pictures to output a defect detection result.

4. The method of claim 1 , wherein the multi-level feature extraction instance segmentation network is obtained by cascading three: levels of instance segmentation networks, wherein IoU thresholds for positive and negative sample sampling are set to 0.2 to 0.4 at the first level, are set to 0.3 to 0.45 at the second level, and are set to 0.5 to 0.7 at the third level.

5. The method of claim 4 , wherein the IoU thresholds are set to 0.3 at the first level, are set to 0.4 at the second level, and are set to 0.5 at the third level.

6. The method of claim 1 , wherein a defect mask output by the multi-level feature extraction instance segmentation network is obtained based on a weighted average of defect masks output by an instance segmentation network at each level.

7. The method of claim 1 , wherein the determining the segmented detect mask based on a predefined defect rule to output a defect detection result further comprises:

when a size or a depth of the segmented defect region is greater than a predefined threshold for a defect type of the defect region, outputting a defect class as a defect detection result.

8. The method of claim 1 , wherein the defect type comprises a pit defect and a protrusion defect, and the determining the segmented defect mask based on a predefined defect rule to output a defect detection result further comprises:

when the defect type of the defect region is a pit defect, if both the segmented 2D defect mask and the segmented 3D defect mask comprise the defect region, or the depth of the defect region is greater than a predefined threshold for the pit detect, outputting a detect class as the defect detection result; or

when the defect type of the defect region is a protrusion defect, if both the segmented 2D defect mask and the segmented 3D defect mask comprise the defect region, and the size and the depth of the defect region are greater than predefined thresholds for the protrusion defect, outputting a defect class as a defect detection result.

9. A non-transitory computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a computing device, cause the computing device to implement tine method for defect detection of claim 1 .

10. A system for defect detection, the system comprising circuitry configured to:

acquire a two-dimensional (2D) picture of an object to be detected;

input the acquired 2D picture to a trained defect segmentation model to obtain a segmented 2D defect mask, wherein the defect segmentation model is trained based on a multi-level feature extraction instance segmentation network with intersection over union (IoU) thresholds being increased level by level, and the 2D defect mask comprises information about a defect type, a defect size, and a defect location of a segmented defect region; and

determine the segmented 2D defect mask based on a predefined defect rule to output a defect detection result,

wherein the multi-level feature extraction instance segmentation network is obtained by cascading at least three levels of instance segmentation networks.

11. The system of claim 10 , wherein the circuitry is further configured to:

acquire a three-dimensional (3D) picture of the object to be detected;

preprocess the acquired 3D picture to obtain an image with depth information of the object to be detected; and

input the obtained image with depth information to the trained defect segmentation model to obtain a segmented 3D defect mask, wherein the 3D defect mask comprises information about a defect depth of a segmented defect region; and

perform, based on a predefined defect rule, fusion and determination on the segmented 2D defect mask and the segmented 3D defect mask to output a defect detection result.

12. The system of claim 11 , wherein the circuitry is further configured to:

perform pixel-level alignment on the 2D picture and the 3D picture by using a coordinate transformation matrix between the acquired 2D picture and the acquired 3D picture, to obtain aligned pictures;

fill the segmented 2D defect mask and the segmented 3D defect mask into the aligned pictures; and

perform, based on the predefined detect rule, fusion and determination on the segmented 2D defect mask and the segmented 3D defect mask on the aligned pictures to output a defect detection result.

13. The system of claim 10 , wherein the multi-level feature extraction instance segmentation network is obtained by cascading three levels of instance segmentation networks, wherein IoU thresholds for positive and negative sample sampling are set to 0.2 to 0.4 at the first level, are set to 0.3 to 0.45 at the second level, and are set to 0.5 to 0.7 at the third level.

14. The system of claim 13 , wherein the IoU thresholds are set to 0.3 at the first level, are set to 0.4 at the second level, and are set to 0.5 at the third level.

15. The system of claim 10 , wherein a defect mask output by the multi-level feature extraction instance segmentation network is obtained based on a weighted average of defect masks output by an instance segmentation network at each level.

16. The system of claim 10 , wherein the circuitry defect determination module is further configured to:

when a size or a depth of the segmented defect region is greater than a predefined threshold for a defect type of the defect region, output a defect class as a defect detection result.

17. The system of claim 10 , wherein the defect type comprises a pit defect and a protrusion defect, and the circuitry is further configured to:

when the defect type of the defect region is a pit defect, if both the segmented 2D defect mask and the segmented 3D defect mask comprise the detect region, or the depth of the defect region is greater than a predefined threshold for the pit defect, output a defect class as the defect detection result; or

when the defect type of the defect region is a protrusion defect, if both the segmented 2D defect mask and the segmented 3D defect mask comprise the defect region, and the size and the depth of the defect region are greater than predefined thresholds for the protrusion detect, output a defect class as a defect detection result.