Method for identifying binary dot matrix code on mold surface
The present disclosure relates to a process for identifying a binary dot matrix code on a mold surface, comprising: capturing a plurality of binary dot matrix code images of the mold surface; inputting the acquisition data into the YOLOv8 network model for training; recapturing the images to be tested and inputting them into the upgraded YOLOv8 network model for detection and output; calculating positional coordinates of vertices at the outermost periphery of the four bounding boxes based on the output results to obtain the distorted quadrilateral, which is converted into the square pattern by the perspective transformation; inputting the square pattern into the upgraded YOLOv8 network model for detection and outputting the results, and calculating the counterclockwise rotation angle θ; obtaining the standard pattern by rotating the square pattern by the angle θ; dividing the standard pattern into the image blocks with the same size in equal portions, and identifying the circular code points of the image blocks, concatenating the identification results to obtain the binary-encoded sequence data; and decoding the binary-encoded sequence data based on binary-encoded decoding rules, and deriving corresponding binary-encoded character information.
1 . A method for identifying a binary dot matrix code on a mold surface, comprising:
step S 1 , data acquisition: capturing a plurality of binary dot matrix code images of the mold surface by a camera, wherein a fiducial marker is arranged in each of four corners of each of the plurality of binary dot matrix code images, and a training set is constructed through annotating bounding boxes for each fiducial marker;
step S 2 , model training: inputting data from the training set into a You Only Look Once version 8 (YOLOv8) network model to train the YOLOv8 network model to obtain optimal weight data of the YOLOv8 network model, and obtaining a trained and upgraded YOLOv8 network model by loading the optimal weight data into the YOLOv8 network model again;
step S 3 , fiducial marker detection: capturing binary dot matrix code images of a mold surface to be tested by the camera to form a test set, inputting images of the test set to the trained and upgraded YOLOv8 network model, and outputting corner points of four bounding boxes of data of the test set by model training, each of the four bounding boxes is provided with 4 corner points, respectively, totaling 16 corner points;
step S 4 , calculating peripheral vertices: filtering the 16 corner points by a filtering strategy, and calculating positional coordinates of vertices at an outermost periphery of the four bounding boxes;
step S 5 , perspective correction: obtaining a distorted quadrilateral based on the positional coordinates of the vertices at the outermost periphery of the four bounding boxes, and converting the distorted quadrilateral into a standard L×L square pattern by perspective transformation;
step S 6 , calculating a counterclockwise rotation angle: inputting the square pattern obtained in the step S 5 into the trained and upgraded YOLOv8 network model of the step S 2 for detection, and outputting four corner points of bounding boxes corresponding to a fiducial marker of the square pattern;
obtaining a coordinate of a center point G of the bounding boxes based on the four corner points of the bounding boxes;
comparing the coordinate of the center point G of the bounding boxes with a coordinate of a center point
O
(
L
2
,
L
2
)
of the square pattern obtained in the step S 5 to obtain a counterclockwise rotation angle θ of the square pattern;
step S 7 , rotational correction: rotating the square pattern of the step S 5 counterclockwise by an angle θ by perspective transformation to align the square pattern to a correct position to obtain a standard pattern of binary dot matrix code images of the test set;
step S 8 , code point identification: dividing the standard pattern into image blocks with a same size in equal portions based on a dimension of a binary code dots; and inputting the image blocks into the trained and upgraded YOLOv8 network model of the step S 2 sequentially in the order of top to bottom and left to right for identification;
in response to determining that a code point exists in the image blocks, outputting 1, and in response to determining that there is no code point in the image blocks, outputting 0;
concatenating identification results to obtain binary-encoded sequence data; and
step S 9 , decoding: decoding the binary-encoded sequence data based on binary-encoded decoding rules and deriving corresponding binary-encoded character information.
2 . The method of claim 1 , wherein the filtering strategy is categorized into two types, and the filtering strategy includes a first filtering strategy and a second filtering strategy, wherein:
a filtering process of the first filtering strategy is as follows:
step S 4 - a 1 : constructing the 16 corner points of the four bounding boxes output by the YOLOv8 network model in the Step S 3 into a set P,
wherein each corner point is denoted as a ij , a coordinate of each corner point is denoted as (x ij , y ij ), i denotes numbers of the bounding boxes, which takes a range of i∈={1,2,3,4}, and j denotes the jth point of the four corner points of the bounding boxes, which takes a range of j∈{1,2,3,4};
step S 4 - a 2 : coordinate information of four corner points of a first bounding box including:
a coordinate of a 11 being denoted as (x 11 , y 11 )), a coordinate of a 12 being denoted as (x 12 , y 12 ), a coordinate of a 13 being denoted as (x 13 , y 13 ), a coordinate of a 14 being denoted as (x 14 , y 14 );
step S 4 - a 3 : assuming that positional coordinates of vertices at the outermost periphery of the four bounding boxes are Q 1 (x Q1 , y Q1 ), Q 2 (x Q2 , y Q2 ), Q 3 (x Q3 , y Q3 ), Q 4 (x Q4 , y Q4 ), and Q 1 is located at a topmost part of a binary dot matrix code pattern, Q 2 is located at a bottom part of the binary dot matrix code pattern, Q 3 is located at a leftmost part of the binary dot matrix code pattern, and Q 4 is located at a rightmost part of the binary dot matrix code pattern, coordinates of individual vertices being calculated as follows:
Q 1 is located at the topmost part of the binary dot matrix code pattern, which with a vertical coordinate y Q1 is a point with a smallest vertical coordinate in the set P, as shown in equations (1) and (2) below:
y
Q
1
=
min
(
y
ij
)
,
i
,
j
∈
[
1
,
4
]
(
1
)
Q
1
=
{
(
x
ij
,
y
ij
)
∈
P
,
y
ij
=
y
Q
1
}
,
i
,
j
∈
[
1
,
4
]
(
2
)
Q 2 is located at the bottom part of the binary dot matrix code pattern, which with a longitudinal coordinate y Q2 is a point with a largest longitudinal coordinate in the set P, as shown in equations (3) and (4) below:
y
Q
2
=
max
(
y
ij
)
,
i
,
j
∈
[
1
,
4
]
(
3
)
Q
2
=
{
(
x
ij
,
y
ij
)
∈
P
,
y
ij
=
y
Q
2
}
,
i
,
j
∈
[
1
,
4
]
(
4
)
Q 3 is located at the leftmost part of the binary dot matrix code pattern, which with a transverse coordinate x Q3 is a smallest transverse coordinate in the set P, as shown in equations (5) and (6) below:
x
Q
3
=
min
(
x
ij
)
,
i
,
j
∈
[
1
,
4
]
(
5
)
Q
3
=
{
(
x
ij
,
y
ij
)
∈
P
,
x
ij
=
x
Q
3
}
,
i
,
j
∈
[
1
,
4
]
(
6
)
Q 4 is located at the rightmost part of the binary dot matrix code pattern, which a transverse coordinate x Q4 is a largest transverse coordinate in the set P, as shown in equations (7) and (8) below:
x
Q
4
=
max
(
x
ij
)
,
i
,
j
∈
[
1
,
4
]
(
7
)
Q
4
=
{
(
x
ij
,
y
ij
)
∈
P
,
x
ij
=
x
Q
4
}
,
i
,
j
∈
[
1
,
4
]
.
(
8
)
3 . The method of claim 2 , wherein a filtering process of the second filtering strategy is as follows:
constructing the 16 corner points of the four bounding boxes outputted by the YOLOv8 network model in the step S 3 into the set W, wherein each coordinate of each point w m is (x m , y m );
wherein for any point w m , a distance between the point w m from the other 15 corner points w n (m≠n) is defined as d (w m , w n ), which is calculated using a Euclidean distance equation (9):
d
(
w
m
,
w
n
)
=
(
x
m
-
x
n
)
2
+
(
y
m
-
y
n
)
2
,
m
,
n
∈
[
1
,
1
6
]
,
m
≠
n
(
9
)
calculating a sum of distances between the point w m and the other 15 corner points using a following equation (10):
S
w
m
=
∑
n
=
1
,
m
≠
n
1
6
d
(
w
m
,
w
n
)
,
m
,
n
∈
[
1
,
1
6
]
(
10
)
by comparing all S w m , selecting points corresponding to a first 4 largest S w m as 4 peripheral vertices of the binary dot matrix code, wherein an expression equation is as follows:
assuming that all S w m are sorted in descending order to obtain a sorted sequence as follows (11):
S
w
1
≥
S
w
2
≥
…
≥
S
w
16
(
11
)
selecting a set S w m corresponding to points of the first 4 largest P′ as:
P
′
=
{
w
1
,
w
2
,
w
3
,
w
4
}
.
(
12
)
4 . The method of claim 1 , wherein a transformation equation of the perspective transformation is a transformation process of converting a pixel coordinate (u, v) to a point on a three-dimensional world coordinate system, and transforming the point to another pixel coordinate (x′, y′),
a transformation equation (13) of the perspective transformation is as follows:
[
x
′
y
′
z
′
]
=
[
b
11
b
12
b
13
b
21
b
22
b
23
b
31
b
32
b
33
]
[
u
v
1
]
(
13
)
wherein a coordinate of an original image in a two-dimensional plane is (u, v), (x′, y′, z′) is a coordinate of a converted three-dimensional world;
the perspective transformation matrix
[
b
11
b
12
b
13
b
21
b
22
b
23
b
31
b
32
b
33
]
is split into 3 parts,
(
b
11
b
12
b
21
b
22
)
is configured as linear transformation,
(
b
13
b
23
)
is configured as the perspective transformation, (b 31 , b 32 ) is configured as translation operation.
5 . The method of claim 3 , wherein in the perspective transformation, a pixel coordinate (u, v) is transformed to a point on the three-dimensional world coordinate system, and transformed to a new two-dimensional plane to obtain a corrected coordinate of (x, y), and equations (14) and (15) is shown as follows:
x
=
x
′
z
′
=
b
11
u
+
b
21
v
+
b
31
b
13
u
+
b
23
v
+
b
33
(
14
)
y
=
y
′
z
′
=
b
12
u
+
b
22
v
+
b
32
b
13
u
+
b
23
v
+
b
33
.
(
15
)
6 . The method of claim 1 , wherein the obtaining the counterclockwise rotation angle θ of the square pattern in the step S 6 includes:
step S 6 - 1 : assuming that the four corner points of the bounding boxes corresponding to the fiducial marker of the square pattern are denoted as T r and coordinates are denoted as (x r , y r ), wherein r ∈[1,4], coordinates of the center point G (x G , y G ) of the bounding boxes and the center point G of the bounding boxes is calculating by equations (16) and (17):
x
G
=
∑
r
=
1
4
x
r
4
;
(
16
)
y
G
=
∑
r
=
1
4
y
r
4
;
(
17
)
step S 6 - 2 : determining the counterclockwise rotation angle θ of the square pattern by comparing the coordinate of the center point G (x G , y G ) of the bounding boxes with a coordinate of a center point
O
(
L
2
,
L
2
)
of the square pattern of the step S 5 , wherein a corresponding relationship is as follows:
θ
=
{
0
,
x
G
>
L
2
and
y
G
>
L
2
90
,
x
G
<
L
2
and
y
G
>
L
2
180
,
x
G
<
L
2
and
y
G
<
L
2
270
,
x
G
>
L
2
and
y
G
<
L
2
.
(
18
)
7 . The method of claim 6 , wherein the obtaining the standard pattern of binary dot matrix code images of the test set in the step S 7 includes:
assuming that a point U(x u , y u ) on the binary dot matrix code pattern after the perspective correction in the step S 5 is rotated counterclockwise by an angle θ;
wherein a rotation process is as follows:
calculating a coordinate (x c , y c ) of the point U(x u , y u ) with respect to the center point
O
(
L
2
,
L
2
)
:
x
c
=
x
u
-
L
2
(
19
)
y
c
=
y
u
-
L
2
(
20
)
applying a rotation matrix R to rotate (x c , y c ) to obtain a coordinate (x h , y h ):
R
=
[
cos
(
θ
)
-
sin
(
θ
)
sin
(
θ
)
cos
(
θ
)
]
(
21
)
[
x
h
y
h
]
=
R
·
[
x
c
y
c
]
(
22
)
translating (x h , y h ) back to an original position to obtain a final rotation point U′(x new , y new ):
x
new
=
x
h
+
L
2
(
23
)
y
new
=
y
h
+
L
2
(
24
)
obtaining a coordinate by rotating the point about the center point by the θ:
x
new
=
cos
(
θ
)
·
(
x
u
=
L
2
)
-
sin
(
θ
)
·
(
y
u
-
L
2
)
+
L
2
(
25
)
y
new
=
sin
(
θ
)
·
(
x
u
=
L
2
)
-
cos
(
θ
)
·
(
y
u
-
L
2
)
+
L
2
.
(
26
)
8 . The method of claim 1 , wherein the code point identification in the step S 8 includes:
step S 8 - 1 , dividing a binary dot matrix code standard pattern into the image blocks: based on the dimension of the binary code dots, dividing the binary dot matrix code standard pattern rotated to the correct position in the step S 7 into the image blocks with the same size in equal portions; and
step S 8 - 2 , image blocks identification code point: identifying circular binary code points of the image blocks sequentially in the order of top to bottom and left to right by using the trained and upgraded YOLOv8 network model in the step S 2 , in response to determining that a code point exists in the image blocks, outputting 1, and in response to determining that there is no code point in the image blocks, outputting 0, and identifying and obtaining a binary coding sequence 01.