Medical image synthesis for motion correction using generative adversarial networks
A computer system is configured to remove motion artifacts in medical images using a generative adversarial network (GAN). The computer system instantiates the GAN having one or more generative network(s) and one or more discriminative network(s) that are pitted against each other to train a generative model and a discriminative model. The training uses a training dataset including a plurality of medical images that are previously classified as without significant motion artifacts for diagnostic purposes. The discriminative model is trained to classify medical images as real or artificial. The generative model is trained to enhance the quality of a medical image and remove motion artifacts by producing a medical image directly from a post-contrast image, without using a pre-contrast mask.
1. A computer system for removing motion artifacts in medical images using a generative adversarial network (GAN) comprising:
one or more processors; and
one or more computer-readable media having stored thereon executable instructions that when executed by the one or more processors configure the computer system to perform at least the following:
instantiate the generative adversarial network (GAN) having one or more generative network(s) and one or more discriminative network(s) that are pitted against each other to train a generative model and a discriminative model,
wherein the training of the generative network(s) and the discriminative network(s) uses a training dataset comprising a first plurality of medical images that are previously classified as without significant motion artifacts for diagnostic purposes;
wherein the discriminative model is trained to classify medical images as artificially generated or real; and
wherein the generative model is trained to enhance the quality of a medical image and remove motion artifacts by producing an image without the use of a pre-contrast mask.
2. The computer system of claim 1 , the computer system further configured to:
receive a new medical image having significant motion artifacts; and
enhance quality of the new medical image by the generative model, such that the enhanced medical image no longer has significant motion artifacts.
3. The computer system of claim 2 , the computer system further configured to:
determine whether the new medical image has significant motion artifacts; and
in response to the determination of having significant motion artifacts, cause the generative model to enhance the first medical image by substantially removing the motion artifacts in the new medical image.
4. The computer system of claim 1 , the computer system further configured to modify the discriminative model and generative model by transfer learning, comprising:
continue to train the previously trained discriminative model and the generative model using a second plurality of medical images that are different from the first plurality of medical images, wherein the second plurality of medical images are also previously classified as without significant motion artifacts.
5. The computer system of claim 1 , the computer system further configured to:
for each of the first plurality of medical images, when the medical image is not with a predetermined size, transform the image into one or more images of the predetermined size; and
use the medical images having the predetermined size as the training dataset.
6. The computer system of claim 5 , wherein for each of the plurality of images, when the image is greater than the predetermined size, split the image into a plurality of images having the predetermined size.
7. The computer system of claim 1 , wherein the training dataset includes a plurality of medical images that have been confirmed by human experts as without significant motion artifacts.
8. The computer system of claim 5 , the computer system further configured to:
receive a medical image having significant motion artifacts; and
cause the generative model to enhance quality of the medical image by substantially removing the motion artifacts, such that the enhanced medical image no longer has significant motion artifacts.
9. The computer system of claim 5 , the computer system further configured to:
receive a contrast image prior to subtraction of a pre-contrast image; and
enhance the contrast image without having to subtract a pre-contrast image, such that the discriminative model classifies the enhanced image as a real medical image without significant motion artifact.
10. The computer system of claim 1 , wherein each of the one or more discriminative network(s) is a convolutional network.
11. The computer system of claim 10 , wherein each of the convolutional network(s) includes a receptive field having a predetermined size.
12. The computer system of claim 11 , wherein a size of a receptive field of each of the convolutional network(s) is selective from 1×1, 70×70, or 286×286.
13. The computer system of claim 10 , wherein at least one of the convolutional network(s) includes U-Net convolutional network(s).
14. The computer system of claim 1 , wherein:
the GAN has an objective represented by an objective function,
the discriminative network(s) is configured to maximize the objective; and
the generative network(s) is configured to minimize the objective.
15. The computer system of claim 14 , wherein the objective includes a distance loss.
16. The computer system of claim 14 , wherein the objective includes a distance loss plus an adversarial loss.
17. The computer system of claim 1 , wherein the GAN is one of the following: (1) pixelGAN, (2) patchGAN, or (3) imageGAN.
18. The computer system of claim 17 , wherein the GAN is a patchGAN, in which the discriminative network classifies a digital medical image as real or artificial when each patch having a predetermined size in the digital medical image has no significant motion artifacts.
19. A method for removing motion artifacts in medical images using a generative adversarial network, the method comprising:
instantiating the generative adversarial network (GAN) having one or more generative network(s) and one or more discriminative network(s) that are pitted against each other to train a generative model and a discriminative model,
wherein the training of the generative network(s) and the discriminative network(s) uses a training dataset comprising a first plurality of medical images that are previously classified as without significant motion artifacts for diagnostic purposes;
wherein the discriminative model is trained to classify medical images as artificially generated or real; and
wherein the generative model is trained to enhance the quality of a medical image and remove motion artifacts by producing an image without the use of a pre-contrast mask.
20. The method of claim 19 , further comprising:
receive a first medical image having significant motion artifacts; and
enhance quality of the first medical image by the generative model, such that the enhanced medical image no longer has significant motion artifacts.