Cross-scale defect detection method based on deep learning
A cross-scale defect detection method based on deep learning, including: (S 1 ) building a vision data acquisition system to acquire a surface image of a part to be processed; and building a defect dataset; (S 2 ) building a deep learning-based cross-scale defect detection model; and inputting the defect dataset obtained in the step (S 1 ) into the deep learning-based cross-scale defect detection model for model training; and (S 3 ) building a defect detection system according to the deep learning-based cross-scale defect detection model and the vision data acquisition system; and detecting a defect of the surface image of the part to be processed.
1 . A cross-scale defect detection method based on deep learning, comprising:
(S 1 ) building a vision data acquisition system to acquire a depth image and a visible image of a defect of a surface image of a part to be processed; and constructing a defect dataset;
(S 2 ) building a deep learning-based cross-scale defect detection model; and inputting the defect dataset constructed in step (S 1 ) into the deep learning-based cross-scale defect detection model for model training; and
(S 3 ) building a defect detection system according to the deep learning-based cross-scale defect detection model and the vision data acquisition system; and detecting, by the defect detection system, the defect of the surface image of the part to be processed;
wherein in step (S 2 ), the deep learning-based cross-scale defect detection model is operated through steps of:
(a) taking a defect data group comprising the depth image and the visible image as an input;
(b) extracting a feature of the depth image and a feature of the visible image by means of a bi-branch feature extraction network; and subjecting the feature of the depth image and the feature of the visible image respectively extracted by two branches of the bi-branch feature extraction network to weighted fusion through an attention mechanism; and
(c) subjecting the feature of the depth image and the feature of the visible image after weighted fusion to cross-scale feature fusion to reduce a channel dimension of a feature map F A2 of a first scale, a channel dimension of a feature map F A3 of a second scale, a channel dimension of a feature map F A4 of a third scale, a channel dimension of a feature map F A5 of a fourth scale and a channel dimension of a feature map F A6 acquired by a squeeze-and-excitation (SE) module from 256 to 64; and obtaining intermediate feature maps F B2 , F B3 , F B4 , F B5 and F B6 by using a 1×1 convolution;
performing up-sampling and/or down-sampling on the F B2 , F B3 , F B4 and F B5 followed by concatenating to obtain a 256-D feature map having the same spatial resolution with a corresponding scale; and
concatenating the F A4 with the 256-D feature map to achieve a cross-scale feature fusion of five scales, expressed as:
f
=
∑
n
N
sum
(
∑
i
∈
W
,
j
∈
H
F
n
(
x
i
,
y
j
)
)
+
F
A
4
;
wherein (x, y) is a pixel point of F n ; sum indicates summation; N is the number of feature maps; W is an image width; and H is an image height.
2 . The cross-scale defect detection method of claim 1 , wherein step (S 1 ) comprises:
(S 11 ) acquiring, by a depth sensor, the depth image of the defect of the surface image of the part to be processed, and acquiring, by a vision sensor, the visible image of the defect of the surface image of the part to be processed, wherein the depth image of the defect and the visible image of the defect are constructed as a defect data group;
(S 12 ) rotating, cutting, scaling and converting the depth image and the visible image acquired in step (S 11 ) to increase data of the surface image of the part to be processed for training; and
(S 13 ) annotating the defect of the surface image of the part to be processed by means of LabelImg to obtain the defect dataset.
3 . The cross-scale defect detection method of claim 1 , wherein each of the two branches of the bi-branch feature extraction network comprises a mix convolution branch and a squeeze-and-excitation (SE) branch;
wherein the mix convolution branch is configured to fuse multi-scale local information by using different receptive fields according to a convolution kernel size and a group size;
the SE branch is configured to distinguish a significance between different feature layers, and deepen semantic extraction and decoding through residual skip connection;
the group size G determines the number of different types of convolution kernels for a single input tensor; and
when G=1, mix convolution is equivalent to a normal depth convolution.
4 . The cross-scale defect detection method of claim 1 , wherein a fusion equation of the attention mechanism is expressed as:
F
=
∑
i
N
λ
1
F
di
+
λ
2
F
vi
;
wherein λ 1 is a weight of the feature of the depth image; λ 2 is a weight of the feature of the visible image; N is the number of layers of a feature map; F di is an i th -layer feature map of the depth image; and F vi is an i th -layer feature map of the visible image.
5 . The cross-scale defect detection method of claim 1 , wherein the step (S 3 ) comprises:
building the defect detection system according to the deep learning-based cross-scale defect detection model and the vision data acquisition system; and
outputting, by the defect detection system, defect detection results in real time and saving the defect detection results in a form comprising a defect image and a table comprising a defect location;
wherein the vision data acquisition system is configured to acquire data; and the deep learning-based cross-scale defect detection model is configured for defect detection and detection result output.