Method and apparatus for reconstructing medical image
The present application provides a method and an apparatus for reconstructing a medical image, and a method and an apparatus for training a medical image reconstruction network. The method for training a medical image reconstruction network includes: performing feature coding extraction on a real image sample to obtain a feature coding vector of the real image sample; performing, through an image reconstruction network, image reconstruction based on the feature coding vector to obtain a first image, and performing image reconstruction based on a first hidden layer vector of the real image sample to obtain a second image; and performing, through an image discrimination network, image discrimination on the real image sample, the first image, and the second image, and optimizing the image reconstruction network according to an image discrimination result.
1. A method for training a medical image reconstruction network, being applied to a server, and the method comprising:
performing feature coding extraction on a real image sample to obtain a feature coding vector of the real image sample;
performing, through an image reconstruction network, image reconstruction based on the feature coding vector to obtain a first image, and performing image reconstruction based on a first hidden layer vector of the real image sample to obtain a second image; and
performing, through an image discrimination network, image discrimination on the real image sample, the first image, and the second image to obtain a loss function of the image discrimination network, a structural similarity metric loss function, and a perceptual metric loss function, and optimizing the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function;
wherein
the step of optimizing the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function comprises:
performing adversarial training on the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function, which comprises:
determining a second loss function of the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function, updating a network parameter of the image reconstruction network by a gradient descent method, and training the image reconstruction network;
wherein, the second loss function is as follows:
L G =−E z e [D ( G ( z e ))]− E z r [D ( G ( z r ))]+λ 1 L SSIM ( G ( z r ), x real )+λ 2 L perceptual ( G ( z r ), x real ),
L SSIM =E z e [C ( z e )]− E z r [C ( z r )],
L
Perceptual
(
G
(
z
r
)
,
x
real
)
=
G
(
z
r
)
-
x
real
2
2
+
∑
h
=
1
H
∑
w
=
1
W
ϕ
(
G
(
z
r
)
)
-
ϕ
(
G
(
z
r
)
)
2
,
and
L D =E z e [D ( G ( z e ))]+ E z r [D ( G ( z r ))]−2 E x real [D ( x real )],
in which, L G represents the second loss function, z e represents the feature coding vector, z r represents the first hidden layer vector, C represents an image coding network, D represents the image discrimination network, G represents the image reconstruction network, E represents a mathematical expectation, L SSIM represents the structural similarity metric loss function, L perceptual represents the perceptual metric loss function, X real represents a real image, λ 1 and λ 2 represent weight coefficients, Φ represents a Gram matrix, and L D represents the loss function of the image discrimination network.
2. The method for training a medical image reconstruction network according to claim 1 , wherein the step of performing feature coding extraction on the real image sample to obtain the feature coding vector of the real image sample comprises:
performing, through an image coding network, feature extraction on the real image sample to obtain the feature coding vector of the real image sample.
3. The method for training a medical image reconstruction network according to claim 2 , wherein the step of performing, through the image coding network, feature extraction on the real image sample to obtain the feature coding vector of the real image sample comprises:
performing hierarchical feature extraction on the real image sample through a plurality of three-dimensional convolutional layers of the image coding network; and
processing an extracted feature through a linear function to obtain the feature coding vector of the real image sample.
4. The method for training a medical image reconstruction network according to claim 3 , wherein the linear function is a piecewise linear function.
5. The method for training a medical image reconstruction network according to claim 4 , wherein the piecewise linear function is a ReLU function.
6. The method for training a medical image reconstruction network according to claim 1 , wherein the step of performing, through the image reconstruction network, image reconstruction based on the feature coding vector to obtain the first image, and performing image reconstruction based on the first hidden layer vector of the real image sample to obtain the second image, comprises:
inputting the feature coding vector and the first hidden layer vector into the image reconstruction network to obtain the first image and the second image; wherein convolutional layers of the image reconstruction network are three-dimensional separable convolutional layers with nearest neighbor upsampling.
7. A method for reconstructing a medical image, being applied to a server, and the method comprising:
acquiring a second hidden layer vector of an image to be reconstructed; and
performing, through a trained image reconstruction network, image reconstruction on the image to be reconstructed;
wherein
the trained image reconstruction network is obtained by a method for training a medical image reconstruction network, and the method for training the medical image reconstruction network comprises:
performing feature coding extraction on a real image sample to obtain a feature coding vector of the real image sample;
performing, through an image reconstruction network, image reconstruction based on the feature coding vector to obtain a first image, and performing image reconstruction based on a first hidden layer vector of the real image sample to obtain a second image; and
performing, through an image discrimination network, image discrimination on the real image sample, the first image, and the second image to obtain a loss function of the image discrimination network, a structural similarity metric loss function, and a perceptual metric loss function, and optimizing the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function;
wherein
the step of optimizing the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function comprises:
performing adversarial training on the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function, which comprises:
determining a second loss function of the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function, updating a network parameter of the image reconstruction network by a gradient descent method, and training the image reconstruction network;
wherein, the second loss function is as follows:
L G =−E z e [D ( G ( z e ))]− E z r [D ( G ( z r ))]+λ 1 L SSIM ( G ( z r ), x real )+λ 2 L perceptual ( G ( z r ), x real ),
L SSIM =E z e [C ( z e )]− E z r [C ( z r )],
L
Perceptual
(
G
(
z
r
)
,
x
real
)
=
G
(
z
r
)
-
x
real
2
2
+
∑
h
=
1
H
∑
w
=
1
W
ϕ
(
G
(
z
r
)
)
-
ϕ
(
G
(
z
r
)
)
2
,
and
L D =E z e [D ( G ( z e ))]+ E z r [D ( G ( z r ))]−2 E x real [D ( x real )],
in which, L G represents the second loss function, z e represents the feature coding vector, z r represents the first hidden layer vector, C represents an image coding network, D represents the image discrimination network, G represents the image reconstruction network, E represents a mathematical expectation, L SSIM represents the structural similarity metric loss function, L perceptual represents the perceptual metric loss function, X real represents a real image, λ 1 and λ 2 represent weight coefficients, Φ represents a Gram matrix, and L D represents the loss function of the image discrimination network.
8. A terminal device, comprising: a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor; wherein the processor is configured to implement the following steps when executing the computer-readable instructions:
performing feature coding extraction on a real image sample to obtain a feature coding vector of the real image sample;
performing, through an image reconstruction network, image reconstruction based on the feature coding vector to obtain a first image, and performing image reconstruction based on a first hidden layer vector of the real image sample to obtain a second image; and
performing, through an image discrimination network, image discrimination on the real image sample, the first image, and the second image to obtain a loss function of the image discrimination network, a structural similarity metric loss function, and a perceptual metric loss function, and optimizing the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function;
wherein
the step of optimizing the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function comprises:
performing adversarial training on the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function, which comprises:
determining a second loss function of the image reconstruction network according to the loss function of the image discrimination network, the structural similarity metric loss function, and the perceptual metric loss function, updating a network parameter of the image reconstruction network by a gradient descent method, and training the image reconstruction network;
wherein, the second loss function is as follows:
L G =−E z e [D ( G ( z e ))]− E z r [D ( G ( z r ))]+λ 1 L SSIM ( G ( z r ), x real )+λ 2 L perceptual ( G ( z r ), x real ),
L SSIM =E z e [C ( z e )]− E z r [C ( z r )],
L
Perceptual
(
G
(
z
r
)
,
x
real
)
=
G
(
z
r
)
-
x
real
2
2
+
∑
h
=
1
H
∑
w
=
1
W
ϕ
(
G
(
z
r
)
)
-
ϕ
(
G
(
z
r
)
)
2
,
and
L D =E z e [D ( G ( z e ))]+ E z r [D ( G ( z r ))]−2 E x real [D ( x real )],
in which, L G represents the second loss function, z e represents the feature coding vector, z r represents the first hidden layer vector, C represents an image coding network, D represents the image discrimination network, G represents the image reconstruction network, E represents a mathematical expectation, L SSIM represents the structural similarity metric loss function, L perceptual represents the perceptual metric loss function, X real represents a real image, λ 1 and λ 2 represent weight coefficients, Φ represents a Gram matrix, and L D represents the loss function of the image discrimination network.
9. The terminal device according to claim 8 , wherein the step of performing feature coding extraction on the real image sample to obtain the feature coding vector of the real image sample comprises:
performing, through an image coding network, feature extraction on the real image sample to obtain the feature coding vector of the real image sample.
10. The terminal device according to claim 9 , wherein the step of performing, through the image coding network, feature extraction on the real image sample to obtain the feature coding vector of the real image sample comprises:
performing hierarchical feature extraction on the real image sample through a plurality of three-dimensional convolutional layers of the image coding network; and
processing an extracted feature through a linear function to obtain the feature coding vector of the real image sample.
11. The terminal device according to claim 10 , wherein the linear function is a ReLU function.