CT image generation method and apparatus, computer device, and computer-readable storage medium

1. A Computed Tomography (CT) image generation method, performed by a computer device, the method comprising:

obtaining a first X-ray image and a second X-ray image, the first X-ray image and the second X-ray image being X-ray images acquired for a target object from two orthogonal viewing angles;

performing, by using a generator, three-dimensional reconstruction based on the first X-ray image and the second X-ray image to obtain a three-dimensional model of the target object, the generator comprising an encoder and a decoder, the encode including a first encoding unit and a second encoding unit and the decoder including a first decoding unit, a second decoding unit, and a fusion decoding unit, wherein performing three-dimensional reconstruction comprises:

decoding, by using the first decoding unit, first encoding information from the first encoding unit to obtain first decoding information;

decoding, by using the second decoding unit, second encoding information from the second encoding unit, to obtain second decoding information; and

performing, via the fusion decoding unit, three-dimensional fusion by using the first decoding information and the second decoding information as data of different viewing angles in a three-dimensional space to obtain the three-dimensional model of the target object; and

obtaining a CT image of the target object according to the three-dimensional model of the target object.

2. The method according to claim 1 , wherein:

the generator comprises an encoder and a decoder; and

performing, by using the generator, the three-dimensional reconstruction based on the first X-ray image and the second X-ray image to obtain the three-dimensional model of the target object comprises:

separately encoding, by using the encoder, the first X-ray image and the second X-ray image to obtain the first encoding information and the second encoding information; and

performing, by using the decoder, three-dimensional reconstruction decoding on the first encoding information and the second encoding information to obtain the three-dimensional model of the target object.

3. The method according to claim 2 , wherein:

separately encoding, by using the encoder, the first X-ray image and the second X-ray image to obtain the first encoding information and the second encoding information comprises:

performing, by using the first encoding unit, two-dimensional encoding on the first X-ray image to obtain the first encoding information; and

performing, by using the second encoding unit, two-dimensional encoding on the second X-ray image to obtain the second encoding information.

4. The method according to claim 3 , further comprising:

providing output of the first encoding unit to the first decoding unit through a connection-A, the connection-A being used for converting the first encoding information in a two-dimensional form into encoding information in a three-dimensional form.

5. The method according to claim 4 , wherein performing, by using the generator, three-dimensional reconstruction based on the first X-ray image and the second X-ray image to obtain the three-dimensional model of the target object comprises:

expanding, by using the connection-A, the first encoding information in a two-dimensional form into a first one-dimensional vector;

stretching the first one-dimensional vector into a second one-dimensional vector; and

rearranging the second one-dimensional vector into the encoding information in the three-dimensional form.

6. The method according to claim 3 , wherein the first decoding unit comprises a neural network of n+2 decoding layers, the n+2 decoding layers comprising:

an up-sampling layer, n cascaded up-convolutional layers, and a three-dimensional convolutional layer, the up-sampling layer being connected to the first up-convolutional layer, the n th up-convolutional layer being connected to the three-dimensional convolutional layer, n being a positive integer,

wherein the first decoding unit further comprises n connections-C, a first input terminal of an i th connection-C of the n connections-C being connected to an output terminal of an i th decoding layer of the n+2 decoding layers in the first decoding unit, a second input terminal of the i th connection-C being connected to an output terminal of an (i+1) th encoding layer in the first encoding unit, and an output terminal of the i th connection-C of the n connections-C being connected to an input terminal of an (i+1) th decoding layer of the n+2 decoding layers in the first decoding unit, wherein the i th connection-C is used for performing weighted summation on three-dimensional decoding information inputted from the first input terminal and three-dimensional encoding information inputted from the second input terminal, and providing a result of the weighted summation as an input of the (i+1) th decoding layer,

wherein decoding, by using the first decoding unit, the first encoding information to obtain the first decoding information comprises:

performing, by using the i th connection-C after transforming the three-dimensional decoding information inputted from the first input terminal and the three-dimensional encoding information inputted from the second input terminal into the same three-dimensional space, weighted summation on the three-dimensional decoding information and the three-dimensional encoding information in the three-dimensional space; and

providing the result of the weighted summation as the input of the (i+1) th decoding layer.

7. The method according to claim 6 , wherein the second input terminal of the i th connection-C is further connected to an output terminal of the (i+1) th encoding layer in the first encoding unit through an i th connection-B, wherein the method of claim 6 further comprises converting, by the i th connection-B, two-dimensional encoding information outputted by the (i+1) th encoding layer into three-dimensional encoding information.

8. The method according to claim 7 , further comprising expanding, by the i th connection-B, the two-dimensional encoding information outputted by the (i+1) th encoding layer into m layers in a vertical dimension, and determining expanded two-dimensional encoding information of m layers as the three-dimensional encoding information, m being a positive integer.

9. The method according to claim 3 , wherein the fusion decoding unit comprises a neural network of n+2 fusion decoding layers, the n+2 fusion decoding layers comprising:

an up-sampling layer, n cascaded up-convolutional layers, and a three-dimensional convolutional layer, an output terminal of the up-sampling layer being connected to the first up-convolutional layer, the n th up-convolutional layer being connected to the three-dimensional convolutional layer, and n being a positive integer,

wherein the fusion decoding unit further comprises a connection-C, a first input terminal of the connection-C being connected to an input terminal of the first decoding unit, a second input terminal of the connection-C being connected to an input terminal of the second decoding unit, and an output terminal of the connection-C being connected to an input terminal of the up-sampling layer,

wherein the method according to claim 3 further comprises performing weighted summation, by the connection-C is used, on three-dimensional encoding information inputted from the first input terminal and three-dimensional encoding information inputted from the second input terminal, and using a result of the summation as an input of a next fusion decoding layer.

10. The method according to claim 1 , further comprising training the generator based on a generative adversarial network by using a loss function of the generator comprising at least one of:

an adversarial loss;

the adversarial loss and a reconstruction loss;

the adversarial loss and a projection loss; or

the adversarial loss, the reconstruction loss, and the projection loss, wherein

the adversarial loss is used for representing a semantic loss between the three-dimensional model reconstructed by the generator and a sample CT image, the reconstruction loss is used for representing a pixel-level discrepancy loss between the three-dimensional model reconstructed by the generator and the sample CT image, and the projection loss is used for representing a discrepancy loss between the three-dimensional model reconstructed by the generator and the sample CT image on at least one projection plane.

11. The method according to claim 10 , further comprising:

obtaining a training dataset, the training dataset comprising at least two training samples;

setting a discriminator corresponding to the generator;

fixing a first neural network parameter in the generator at an i th training stage, and optimizing a second neural network parameter in the discriminator by using a training sample in the training dataset;

fixing the second neural network parameter in the discriminator at an (i+1) th training stage, and optimizing the first neural network parameter in the generator by using the other training sample in the training dataset; and

performing the foregoing two training stages alternately, until the first neural network parameter and the second neural network parameter steadily converge.

12. The method according to claim 11 , wherein obtaining the training dataset comprises:

obtaining at least two real CT images;

generating a corresponding first virtual X-ray image and a corresponding second virtual X-ray image for each of the at least two real CT images; and

designating each of the at least two real CT images, the corresponding first virtual X-ray images, and the corresponding second virtual X-ray images as one of the at least two training samples.

13. A computer device, comprising a processor and a memory, the memory storing at least one program, the at least one program, when loaded and executed by the processor, being configured to perform the method of claim 1 .

14. A non-transitory computer-readable storage medium, storing at least one program, the at least one program, when loaded and executed by a processor, being configured to perform the method of claim 1 .

15. A three-dimensional image synthetic method, performed by a computer device, the method comprising:

obtaining a first cross-sectional image and a second cross-sectional image, the first cross-sectional image and the second cross-sectional image being images acquired for a target object from two orthogonal viewing angles;

obtaining an encoder and a decoder, the encoder comprising a first encoding unit and a second encoding unit, and the decoder comprising a first decoding unit, a second decoding unit, and a fusion decoding unit;

performing, by using the first encoding unit, two-dimensional encoding on the first cross-sectional image to obtain first encoding information;

decoding, by using the first decoding unit, the first encoding information to obtain first decoding information;

performing, by using the second encoding unit, two-dimensional encoding on the second cross-sectional image to obtain second encoding information;

decoding, by using the second decoding unit, the second encoding information, to obtain second decoding information; and

performing, by using the fusion decoding unit, three-dimensional fusion by using the first decoding information and the second decoding information as data of different viewing angles in a three-dimensional space to obtain a three-dimensional image of the target object.

16. A three-dimensional image processing apparatus, comprising:

an encoder, configured to perform, in parallel, two-dimensional encoding on a first cross-sectional image to obtain first encoding information and two-dimensional encoding on a second cross-sectional image to obtain second encoding information, the first cross-sectional image and the second cross-sectional image being images acquired for a target object from two orthogonal viewing angles; and

a decoder in communication with the encoder and being configured to:

decode, in parallel, the first encoding information to obtain first decoding information and the second encoding information to obtain second decoding information; and

perform three-dimensional fusion by using the first decoding information and the second decoding information as data of different viewing angles in a three-dimensional space to obtain a three-dimensional image of the target object.

17. The three dimensional image processing apparatus of claim 16 , wherein the encoder comprising a first encoding unit and a second encoding unit, configured to perform, in parallel, the two-dimensional encoding on the first cross-sectional image and the two-dimensional encoding on the second cross-sectional image, wherein the first encoding unit comprises a neural network of n+2 encoding layers, the n+2 encoding layers comprising:

a two-dimensional convolutional layer, n cascaded densely-connected layer, and a pooling layer, the two-dimensional convolutional layer being connected to the first densely-connected layer, the n th densely-connected layer being connected to the pooling layer, and n being a positive integer.

18. The three dimensional image processing apparatus of claim 16 , wherein the decoder comprises a first decoding unit and a second decoding unit, configured to decode, in parallel, the first encoding information and the second encoding information, wherein the first decoding unit comprises a neural network of n+2 decoding layers, the n+2 decoding layers comprising:

an up-sampling layer, n cascaded up-convolutional layers, and a three-dimensional convolutional layer, the up-sampling layer being connected to the first up-convolutional layer, the n th up-convolutional layer being connected to the three-dimensional convolutional layer, n being a positive integer.

19. The three dimensional image processing apparatus of claim 16 , wherein the decoder comprises a fusion decoding unit, configured to perform the three-dimensional fusion, wherein the fusion decoding unit comprises a neural network of n+2 fusion decoding layers, the n+2 fusion decoding layers comprising:

an up-sampling layer, n cascaded up-convolutional layers, and a three-dimensional convolutional layer, an output terminal of the up-sampling layer being connected to the first up-convolutional layer, the n th up-convolutional layer being connected to the three-dimensional convolutional layer, and n being a positive integer.

20. The three dimensional image processing apparatus of claim 19 , wherein the fusion decoding unit further comprises n compound connections-C, each of the compound connections-C comprising a first connection-C and a second connection-C, wherein

a first input terminal of an i th first connection-C is connected to an output terminal of an (i+1) th decoding layer in the first decoding unit, a second input terminal of the i th first connection-C is connected to an output terminal of an (i+1) th decoding layer in the second decoding unit, an output terminal of the i th first connection-C is connected to a first input terminal of an i th second connection-C, a second input terminal of the i th second connection-C is connected to an output terminal of an i th fusion decoding layer in the fusion decoding unit, and an output terminal of the i th second connection-C is connected to an input terminal of an (i+1) th fusion decoding layer in the fusion decoding unit.