Generative adversarial network medical image generation for training of a classifier
Mechanisms are provided to implement a machine learning training model. The machine learning training model trains an image generator of a generative adversarial network (GAN) to generate medical images approximating actual medical images. The machine learning training model augments a set of training medical images to include one or more generated medical images generated by the image generator of the GAN. The machine learning training model trains a machine learning model based on the augmented set of training medical images to identify anomalies in medical images. The trained machine learning model is applied to new medical image inputs to classify the medical images as having an anomaly or not.
1. A method, in a data processing system comprising a processor and a memory, the memory comprising instructions that are executed by the processor to configure the processor to implement a machine learning training model, the method comprising:
training, by the machine learning training model, an image generator of a generative adversarial network (GAN) to generate medical images approximating actual medical images;
augmenting, by the machine learning training model, a set of training medical images to include one or more generated medical images generated by the image generator of the GAN;
training, by the machine learning training model, a machine learning model based on the augmented set of training medical images to identify anomalies in medical images; and
applying the trained machine learning model to new medical image inputs to classify the medical images as having an anomaly or not, wherein training the machine learning model comprises training a discriminator of the GAN based on training data, input to the discriminator, comprising actual labeled medical image data, actual unlabeled medical image data, and generated medical image data generated by the image generator of the GAN.
2. The method of claim 1 , wherein training a generator of the GAN to generate medical images approximating actual medical images, further comprises:
receiving, by the image generator of the GAN, at least one medical image showing an anatomical structure;
inputting, by a noise generator, a noise input to the image generator;
generating, by the image generator, at least one generated image based on a combination of the at least one medical image and the noise input;
providing the at least one medical image and the at least one generated image to the discriminator of the machine learning training image;
analyzing, by the discriminator, the at least one medical image and the at least one generated image to label the at least one medical image and the at least one generated image as either being an actual medical image or a generated medical image; and
modifying an operational parameter of the generator based on the results of analyzing the at least one medical image and the at least one generated image by the discriminator.
3. The method of claim 1 , wherein the GAN is trained to classify medical images with regard to whether or not the medical images contain an abnormality.
4. The method of claim 1 , wherein the discriminator is trained, using an adversarial training technique based on the augmented set of training medical images, to discriminate between medical images showing anomalies and medical images showing normal medical conditions.
5. The method of claim 1 , wherein training the image generator comprises training the image generator based on both labeled training medical images and non-labeled medical images, and wherein the training is based on a feedback obtained from an output of the discriminator.
6. The method of claim 1 , wherein the generated medical image data is generated by the image generator of the GAN based on input noise, and wherein training, by the machine learning training model, the image generator to generate medical images approximating actual medical images comprises:
training the discriminator of the GAN to separately classify generated medical images from actual medical images and separately classifying actual medical images showing a normal medical condition from actual medical images showing an abnormal medical condition; and
training the image generator to generate medical images that cause the discriminator to fail to correctly separately classify the generated medical images from actual medical images for a predetermined ratio of instances indicating convergence of the training of the GAN.
7. The method of claim 1 , wherein the discriminator of the GAN is a convolutional neural network with K+1 output nodes, where K is a number of classes of abnormalities that the machine learning model is configured to identify in input medical images and the K+1 class is a class indicating an input medical image to be a generated medical image from the image generator, and wherein the discriminator outputs, for each input medical image, a corresponding vector output having K+1 values indicating into which class the discriminator classifies the input medical image.
8. The method of claim 1 , wherein the discriminator is a convolutional neural network with 2K output nodes, where K is a number of classes of abnormalities that the machine learning model is configured to identify in input medical images and for each class in the K number of classes, there is a separate output node indicating whether the input medical image is a generated medical image from the image generator, or an actual medical image, and wherein the discriminator outputs, for each input medical image, a corresponding vector output having 2K values indicating into which class the discriminator classifies the input medical image.
9. The method of claim 1 , further comprising:
configuring a loss function of a discriminator of the GAN to include error components for each of actual labeled medical images, actual unlabeled medical images, and generated medical images, wherein training the image generator of the GAN to generate medical images approximating actual medical images comprises minimizing the loss function of the discriminator.
10. The method of claim 1 , wherein the generator is one of a plurality of generators, and wherein at least one generator in the plurality of generators is trained to generate medical images approximating actual medical images having a normal medical condition depicted, and wherein at least one other generator in the plurality of generators is trained to generate medical images approximating actual medical images having an abnormal medical condition depicted.
11. The method of claim 1 , wherein the discriminator generates a classification output that classifies input image data into one of a plurality of classifications, the plurality of classifications comprising a first classification for real-normal image data indicating the input image data to be actual image data representing a normal medical condition, at least one second classification for real-abnormal image data indicating the input image data to be actual image data representing a corresponding abnormal medical condition, and a third classification for generated image data indicating the input image data to be image data generated by the image generator.
12. A non-transitory computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to implement a machine learning training model that operates to: train an image generator of a generative adversarial network (GAN) to generate medical images approximating actual medical images; augment a set of training medical images to include one or more generated medical images generated by the image generator of the GAN; train a machine learning model based on the augmented set of training medical images to identify anomalies in medical images; and apply the trained machine learning model to new medical image inputs to classify the medical images as having an anomaly or not, wherein training the machine learning model comprises training a discriminator of the GAN based on training data, input to the discriminator, comprising actual labeled medical image data, actual unlabeled medical image data, and generated medical me data generated by the image generator of the GAN.
13. The non-transitory computer program product of claim 12 , wherein the computer readable program further causes the machine learning training model to train the generator of the GAN to generate medical images approximating actual medical images, at least by: receiving, by the image generator of the GAN, at least one medical image showing an anatomical structure; inputting, by a noise generator, a noise input to the image generator; generating, by the image generator, at least one generated image based on a combination of the at least one medical image and the noise input; providing the at least one medical image and the at least one generated image to the discriminator of the machine learning training image; analyzing, by the discriminator, the at least one medical image and the at least one generated image to label the at least one medical image and the at least one generated image as either being an actual medical image or a generated medical image; and modifying an operational parameter of the generator based on the results of analyzing the at least one medical image and the at least one generated image by the discriminator.
14. The non-transitory computer program product of claim 12 , wherein the discriminator is trained, using an adversarial training technique based on the augmented set of training medical images, to discriminate between medical images showing anomalies and medical images showing normal medical conditions.
15. The non-transitory computer program product of claim 12 , wherein the computer readable program further causes the machine learning training model to train the image generator at least by training the image generator based on both labeled training medical images and non-labeled medical images, and wherein the training is based on a feedback obtained from an output of the discriminator.
16. The non-transitory computer program product of claim 12 , wherein the generated medical image data is generated by the image generator of the GAN based on input noise, and wherein the computer readable program further causes the machine learning training model to train the image generator to generate medical images approximating actual medical images at least by: training the discriminator of the GAN to separately classify generated medical images from actual medical images and separately classifying actual medical images showing a normal medical condition from actual medical images showing an abnormal medical condition; and training the image generator to generate medical images that cause the discriminator to fail to correctly separately classify the generated medical images from actual medical images for a predetermined ratio of instances indicating convergence of the training of the GAN.
17. The non-transitory computer program product of claim 12 , wherein the discriminator of the GAN is a convolutional neural network with K+1 output nodes, where K is a number of classes of abnormalities that the machine learning model is configured to identify in input medical images and the K+1 class is a class indicating an input medical image to be a generated medical image from the image generator, and wherein the discriminator outputs, for each input medical image, a corresponding vector output having K+1 values indicating into which class the discriminator classifies the input medical image.
18. The non-transitory computer program product of claim 12 , wherein the discriminator is a convolutional neural network with 2K output nodes, where K is a number of classes of abnormalities that the machine learning model is configured to identify in input medical images and for each class in the K number of classes, there is a separate output node indicating whether the input medical image is a generated medical image from the image generator, or an actual medical image, and wherein the discriminator outputs, for each input medical image, a corresponding vector output having 2K values indicating into which class the discriminator classifies the input medical image.
19. The non-transitory computer program product of claim 12 , wherein the computer readable program further causes the machine learning training model to: configure a loss function of a discriminator of the GAN to include error components for each of actual labeled medical images, actual unlabeled medical images, and generated medical images, wherein training the image generator of the GAN to generate medical images approximating actual medical images comprises minimizing the loss function of the discriminator.
20. An apparatus comprising:
at least one processor; and
at least one memory coupled to the at least one processor, wherein the at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement a machine learning training model that operates to:
train an image generator of a generative adversarial network (GAN) to generate medical images approximating actual medical images;
augment a set of training medical images to include one or more generated medical images generated by the image generator of the GAN;
train a machine learning model based on the augmented set of training medical images to identify anomalies in medical images; and
apply the trained machine learning model to new medical image inputs to classify the medical images as having an anomaly or not, wherein training the machine learning model comprises training a discriminator of the GAN based on training data, input to the discriminator, comprising actual labeled medical image data, actual unlabeled medical image data, and generated medical image data generated by the image generator of the GAN.