Single-frame fringe pattern analysis method based on multi-scale generative adversarial network
The invention discloses a single-frame fringe pattern analysis method based on multi-scale generative adversarial network. A multi-scale generative adversarial neural network model is constructed and a comprehensive loss function is applied. Next, training data are collected to train the multi-scale generative adversarial network. During the prediction, a fringe pattern is fed into the trained multi-scale network where the generator outputs the sine term, cosine term, and the modulation image of the input pattern. Finally, the arctangent function is applied to compute the phase. When the network is trained, the parameters of the network do not need to manually tune during the calculation. Since the input of the neural network is only a single fringe pattern, the invention provides an efficient and high-precision phase calculation method for moving objects.
1 . A single-frame fringe pattern analysis method based on multi-scale generative adversarial network, comprising the steps of:
constructing a multi-scale generative adversarial neural network model including a multi-scale image generator and an image discriminator;
constructing a comprehensive loss function L for the multi-scale generative adversarial neural network;
collecting training data to train the multi-scale generative adversarial network;
feeding a single fringe pattern into the trained multi-scale network during prediction, wherein the multi-scale image generator outputs a sine term, a cosine term, and a modulation image of an input pattern; and
applying an arctangent function to compute a phase using the sine term and the cosine term,
wherein the multi-scale image generator comprises four data processing paths including data processing path (1), data processing path (2), data processing path (3) and data processing path (4) with a same structure,
wherein:
an input of the data processing path (4) is I 4 (x, y) with an image size of
1
8
H
×
1
8
W
;
an input of the data processing path (3) includes I 3 (x, y) and an output of the data processing path (4) comprising a sine term, a cosine term and a modulation image upsampled to a size of
1
4
H
×
1
4
W
;
an input of data processing path (2) includes I 2 (x, y) and an output of the data processing path (3) comprising a sine term, a cosine term and a modulation image upsampled to a size of
1
2
H
×
1
2
W
;
an input of data processing path (1) includes I 1 (x, y) and an output of the data processing path (2) comprising a sine term, a cosine term and a modulation image upsampled to a size of H×W.
2 . The method of claim 1 , wherein each data processing path consists of a convolutional layer, 4 residual blocks, and a convolutional layer that is linearly activated.
3 . The method of claim 2 , wherein the comprehensive loss function L is expressed as:
L
=
α
L
image
+
β
L
GAN
,
where α and β are weights, L image is the loss function regarding an image content, and L GAN is an adversarial loss function.
4 . The method of claim 1 , wherein the image discriminator comprises six residual blocks and a fully connected layer connected sequentially, and wherein the fully connected layer is activated by a sigmoid function.
5 . The method of claim 4 , wherein the comprehensive loss function L is expressed as:
L
=
α
L
image
+
β
L
GAN
,
where α and β are weights, L image is the loss function regarding an image content, and L GAN is an adversarial loss function.
6 . The method of claim 1 , wherein the comprehensive loss function L is expressed as:
L
=
α
L
image
+
β
L
GAN
,
where α and β are weights, L image is the loss function regarding an image content, and L GAN is an adversarial loss function.
7 . The method of claim 6 , wherein the image content loss L image is expressed as:
L
image
=
γ
L
f
+
η
L
m
,
where γ and η are weights, L f represents loss of sine and cosine terms, L m represents loss of the modulation image.
8 . The method of claim 7 , wherein the loss of the sine and cosine terms L f is expressed as:
L
f
=
∑
s
=
1
4
L
s
f
,
and
L
s
f
=
1
H
s
×
W
s
[
G
sin
s
-
P
sin
s
2
+
G
cos
s
-
P
cos
s
2
]
,
where S represents different scales of the multi-scale image generator; L f calculates a sum of errors of output sine term and cosine term at four different scales; H s is a height of the image at the scale S, and W s is a width of the image at the scale S; G is a ground truth data and P represents predicted results generated by the multi-scale image generator; subscript sin indicates sine term and subscript cos indicates cosine term, and
wherein the loss of the modulation image L m is written as:
L
m
=
∑
s
=
1
4
L
s
m
,
and
L
s
m
=
1
H
s
×
W
s
G
mod
s
-
P
mod
s
2
,
where subscript mod indicates the modulation image.
9 . The method of claim 6 , the adversarial loss function L GAN is expressed as:
L
GAN
=
E
T
~
p
(
T
)
[
log
d
(
T
)
]
+
E
I
~
p
(
I
)
[
log
(
1
-
d
(
g
(
I
)
)
)
]
,
where E represents expectation; I is an input fringe image; T is a ground-truth data comprising ground-truth sine term, cosine term, and modulation degree diagram corresponding to the input fringe image; p represents probability distribution; g represents the multi-scale image generator; g(I) represents estimated sine term, cosine term, modulation degree diagram by the multi-scale image generator; and d is the image discriminator.
10 . The method of claim 1 , wherein the step of collecting training data to train the multi-scale generative adversarial network comprises the steps of:
measuring fringe images I t (x, y) of v different scenes, and taking seven phase-shifting fringe images for each scene;
calculating a ground-truth sine term M t (x, y), a ground-truth cosine term D t (x, y), and a ground-truth modulation image B t (x, y) with a 7-step phase-shifting method;
feeding the fringe image I t (x, y) into the multi-scale image generator to output the sine term M(x, y), cosine term D(x, y), and modulation image B(x, y);
inputting to the image discriminator a group of sine terms, cosine terms, and modulation images alternately extracted from a ground-truth dataset and a dataset generated by the multi-scale image generator; and
outputting a probability value between 0 and 1 from the image discriminator, wherein the training process terminates when an adversarial loss reaches a preset threshold.
11 . The method of claim 10 , wherein the phase calculation comprises substituting the estimated sine term M(x, y) and cosine term D(x, y) into the arctangent function to obtain a phase φ(x, y) according to a formula of
ϕ
(
x
,
y
)
=
arc
tan
M
(
x
,
y
)
D
(
x
,
y
)
.