Three-dimensional medical image generation
An embodiment of the invention may include a method, computer program product and computer system for three-dimensional medical image generation. The method, computer program product and computer system may include computing device which may receive a first three-dimensional medical image of a first patient from a first period of time, a two-dimensional medical image of the first patient from a second period of time and a plurality of three-dimensional medical images for a plurality of second patients. The computing device may input the three-dimensional medical image of the first patient, the two-dimensional medical image of the first patient and the plurality of three-dimensional medical images for a plurality of second patients into a generative adversarial network (GAN). The computing device may generate a synthetic three-dimensional medical image for the first patient based on the two-dimensional medical image from the second period of time utilizing the GAN.
1. A method for three-dimensional medical image generation, the method comprising:
receiving a first three-dimensional medical image, that was taken from a first patient at a first period of time;
receiving a two-dimensional medical image, that was taken from the first patient at a second period of time;
receiving a plurality of three-dimensional medical images for a plurality of second patients;
inputting the first three-dimensional medical image of the first patient, the two-dimensional medical image of the first patient, and the plurality of three-dimensional medical images for the plurality of second patients into a generative adversarial network (GAN); and
generating a synthetic three-dimensional medical image for the first patient based on the two-dimensional medical image from the second period of time utilizing the GAN.
2. The method of claim 1 , further comprising:
analyzing the synthetic three-dimensional medical image of the first patient for one or more differences compared to the first three-dimensional medical image of the first patient; and
annotating the three-dimensional medical image of the first patient with indicators indicating the differences between the first three-dimensional medical image of the first patient and the synthetic three-dimensional image of the first patient.
3. The method of claim 2 , further comprising:
generating a report with a textual description of the differences; and
displaying at least one of the synthetic three-dimensional medical image of the first patient, the report, and the annotated three-dimensional medical image of the first patient to a user via a user interface.
4. The method of claim 1 , wherein the first three-dimensional medical image of the first patient is a digital breast tomosynthesis (DBT) image.
5. The method of claim 1 , wherein the two-dimensional medical image is a full-field digital mammography (FFDM) image.
6. The method of claim 1 , wherein the plurality of three-dimensional images for the plurality of second patients are digital breast tomosynthesis (DBT) images.
7. The method of claim 1 , wherein generating the synthetic three-dimensional medical image for the first patient based on the two-dimensional medical image from the second period of time utilizing the GAN further comprises:
inputting the synthetic three-dimensional medical image into a first discriminator of the GAN, wherein the first discriminator tries to discriminate between the synthetic three-dimensional medical image and the plurality of three-dimensional medical images from the plurality of second patients; and
inputting the synthetic three-dimensional medical image into a second discriminator of the GAN, wherein the second discriminator creates projected two-dimensional medical images corresponding to the synthetic three-dimensional medical image, wherein the second discriminator tries to discriminate between the projected two-dimensional medical images corresponding to the synthetic three-dimensional medical image and the two-dimensional medical image of the first patient from the second period of time.
8. A computer program product for three-dimensional medical image generation, the computer program product comprising:
a computer-readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, wherein the program instructions are executable by a computer to cause the computer to perform a method, the method comprising:
receiving a first three-dimensional medical image, that was taken from a first patient at a first period of time;
receiving a two-dimensional medical image, that was taken from the first patient at a second period of time;
receiving a plurality of three-dimensional medical images for a plurality of second patients;
inputting the first three-dimensional medical image of the first patient, the two-dimensional medical image of the first patient and the plurality of three-dimensional medical images for the plurality of second patients into a generative adversarial network (GAN); and
generating a synthetic three-dimensional medical image for the first patient based on the two-dimensional medical image from the second period of time utilizing the GAN.
9. The computer program product of claim 8 , further comprising:
analyzing the synthetic three-dimensional medical image of the first patient for one or more differences compared to the first three-dimensional medical image of the first patient; and
annotating the three-dimensional medical image of the first patient with indicators indicating the differences between the first three-dimensional medical image of the first patient and the synthetic three-dimensional image of the first patient.
10. The computer program product of claim 9 , further comprising:
generating a report with a textual description of the differences; and
displaying at least one of the synthetic three-dimensional medical image of the first patient, the report, and the annotated three-dimensional medical image of the first patient to a user via a user interface.
11. The computer program product of claim 8 , wherein the first three-dimensional medical image of the first patient is a digital breast tomosynthesis (DBT) image.
12. The computer program product of claim 8 , wherein the two-dimensional medical image is a full-field digital mammography (FFDM) image.
13. The computer program product of claim 8 , wherein the plurality of three-dimensional images for the plurality of second patients are digital breast tomosynthesis (DBT) images.
14. The computer program product of claim 8 , wherein generating the synthetic three-dimensional medical image for the first patient based on the two-dimensional medical image from the second period of time utilizing the GAN further comprises:
inputting the synthetic three-dimensional medical image into a first discriminator of the GAN, wherein the first discriminator tries to discriminate between the synthetic three-dimensional medical image and the plurality of three-dimensional medical images from the plurality of second patients; and
inputting the synthetic three-dimensional medical image into a second discriminator of the GAN, wherein the second discriminator creates projected two-dimensional medical images corresponding to the synthetic three-dimensional medical image, wherein the second discriminator tries to discriminate between the projected two-dimensional medical images corresponding to the synthetic three-dimensional medical image and the two-dimensional medical image of the first patient from the second period of time.
15. A system for three-dimensional medical image generation, the system comprising:
a computer system comprising: a processor, a computer readable storage medium, and program instructions stored on the computer readable storage medium that are executable by the processor to cause the computer system to:
receive a first three-dimensional medical image, that was taken from a first patient at a first period of time;
receive a two-dimensional medical image, that was taken from the first patient at a second period of time;
receive a plurality of three-dimensional medical images for a plurality of second patients;
input the first three-dimensional medical image of the first patient, the two-dimensional medical image of the first patient and the plurality of three-dimensional medical images for the plurality of second patients into a generative adversarial network (GAN); and
generate a synthetic three-dimensional medical image for the first patient based on the two-dimensional medical image from the second period of time utilizing the GAN.
16. The system of claim 15 , further comprising program instructions to:
analyze the synthetic three-dimensional medical image of the first patient for one or more differences compared to the first three-dimensional medical image of the first patient; and
annotate the three-dimensional medical image of the first patient with indicators indicating the differences between the first three-dimensional medical image of the first patient and the synthetic three-dimensional image of the first patient.
17. The system of claim 16 , further comprising program instructions to:
generate a report with a textual description of the differences; and
display at least one of the second three-dimensional medical image of the first patient, the report, and the annotated three-dimensional medical image of the first patient to a user via a user interface.
18. The system of claim 15 , wherein the first three-dimensional medical image of the first patient and the plurality of three-dimensional images for the plurality of second patients are digital breast tomosynthesis (DBT) images.
19. The system of claim 15 , wherein the two-dimensional medical image is a full-field digital mammography (FFDM) image.
20. The system of claim 15 , wherein the program instructions to generate the synthetic three-dimensional medical image for the first patient based on the two-dimensional medical image from the second period of time utilizing the GAN further comprise program instructions to:
input the synthetic three-dimensional medical image into a first discriminator of the GAN, wherein the first discriminator tries to discriminate between the synthetic three-dimensional medical image and the plurality of three-dimensional medical images from the plurality of second patients; and
input the synthetic three-dimensional medical image into a second discriminator of the GAN, wherein the second discriminator creates projected two-dimensional medical images corresponding to the synthetic three-dimensional medical image, wherein the second discriminator tries to discriminate between the projected two-dimensional medical images corresponding to the synthetic three-dimensional medical image and the two-dimensional medical image of the first patient from the second period of time.