Adversarial and dual inverse deep learning networks for medical image analysis

1. A method for automatically performing a medical image analysis task on a medical image of a patient, comprising:

receiving a medical image of a patient;

inputting the medical image to a trained deep neural network; and

automatically estimating an output model that provides a result of a target medical image analysis task on the input medical image using the trained deep neural network, wherein the trained deep neural network is trained in a discriminative adversarial network based on a minimax objective function comprising 1) a first cost term related to classification, by a discriminator network of the discriminative adversarial network, of ground truth output models, 2) a second cost term related to classification, by the discriminator network, of estimated output models estimated by an estimator network of the discriminative adversarial network from input training images, and 3) a third cost term computed using a cost function that calculates an error between the ground truth output models and the estimated output models.

2. The method of claim 1 , wherein the trained deep neural network is the estimator network that directly maps the output model from the input medical image.

3. The method of claim 2 , wherein the estimator network and the discriminator network of the discriminative adversarial network are trained together based on a set of training samples including the input training images and the ground truth output models to learn parameters for the estimator network and the discriminator network that optimize the minimax objective function over the set of training samples.

4. The method of claim 1 , further comprising:

training the estimator network and the discriminator network to learn parameters to optimize the minimax objective function by repeating the following training operations for a plurality of iterations:

with the parameters of the estimator network fixed, learning the parameters of the discriminator network to optimize the minimax objective function; and

with the parameters of the discriminator network fixed, learning the parameters of the estimator network to optimize the minimax objective function.

5. The method of claim 4 , wherein:

learning the parameters of the discriminator network to optimize the minimax objective function comprises adjusting the parameters of the discriminator network to maximize probability scores computed by the discriminator network for the ground truth output models and to minimize probability scores computed by the discriminator for the estimated output models estimated by the estimator network from the input training images over a set of training samples; and

learning the parameters of the estimator network to optimize the minimax objective function comprises adjusting the parameters of the estimator network to minimize the error between the ground truth output models and the estimated output models estimated by the estimator network and to maximize the probability scores computed by the discriminator for the estimated models estimated by the estimator network over the set of training samples.

6. An apparatus for automatically performing a medical image analysis task on a medical image of a patient, comprising:

means for receiving a medical image of a patient;

means for inputting the medical image to a trained deep neural network; and

means for automatically estimating an output model that provides a result of a target medical image analysis task on the input medical image using the trained deep neural network, wherein the trained deep neural network is trained in a discriminative adversarial network based on a minimax objective function comprising 1) a first cost term related to classification, by a discriminator network of the discriminative adversarial network, of ground truth output models, 2) a second cost term related to classification, by the discriminator network, of estimated output models estimated by an estimator network of the discriminative adversarial network from input training images, and 3) a third cost term computed using a cost function that calculates an error between the ground truth output models and the estimated output models.

7. The apparatus of claim 6 , wherein the trained deep neural network is the estimator network that directly maps the output model from the input medical image.

8. The apparatus of claim 7 , wherein the estimator network and the discriminator network of the discriminative adversarial network are trained together based on a set of training samples including the input training images and the ground truth output models to learn parameters for the estimator network and the discriminator network that optimize the minimax objective function over the set of training samples.

9. The apparatus of claim 6 , further comprising:

training the estimator network and the discriminator network to learn parameters to optimize the minimax objective function by repeating the following training operations for a plurality of iterations:

with the parameters of the estimator network fixed, learning the parameters of the discriminator network to optimize the minimax objective function; and

with the parameters of the discriminator network fixed, learning the parameters of the estimator network to optimize the minimax objective function.

10. The apparatus of claim 9 , wherein:

learning the parameters of the discriminator network to optimize the minimax objective function comprises adjusting the parameters of the discriminator network to maximize probability scores computed by the discriminator network for the ground truth output models and to minimize probability scores computed by the discriminator for the estimated output models estimated by the estimator network from the input training images over a set of training samples; and

learning the parameters of the estimator network to optimize the minimax objective function comprises adjusting the parameters of the estimator network to minimize the error between the ground truth output models and the estimated output models estimated by the estimator network and to maximize the probability scores computed by the discriminator for the estimated models estimated by the estimator network over the set of training samples.

11. A non-transitory computer readable medium storing computer program instructions for automatically performing a medical image analysis task on a medical image of a patient, the computer program instructions when executed by a processor cause the processor to perform operations comprising:

receiving a medical image of a patient;

inputting the medical image to a trained deep neural network; and

automatically estimating an output model that provides a result of a target medical image analysis task on the input medical image using the trained deep neural network, wherein the trained deep neural network is trained in a discriminative adversarial network based on a minimax objective function comprising 1) a first cost term related to classification, by a discriminator network of the discriminative adversarial network, of ground truth output models, 2) a second cost term related to classification, by the discriminator network, of estimated output models estimated by an estimator network of the discriminative adversarial network from input training images, and 3) a third cost term computed using a cost function that calculates an error between the ground truth output models and the estimated output models.

12. The non-transitory computer readable medium of claim 11 , wherein the trained deep neural network is the estimator network that directly maps the output model from the input medical image.

13. The non-transitory computer readable medium of claim 12 , wherein the estimator network and the discriminator network of the discriminative adversarial network are trained together based on a set of training samples including the input training images and the ground truth output models to learn parameters for the estimator network and the discriminator network that optimize the minimax objective function over the set of training samples.

14. The non-transitory computer readable medium of claim 11 , wherein the operations further comprise:

training the estimator network and the discriminator network to learn parameters to optimize the minimax objective function by repeating the following training operations for a plurality of iterations:

with the parameters of the estimator network fixed, learning the parameters of the discriminator network to optimize the minimax objective function; and

with the parameters of the discriminator network fixed, learning the parameters of the estimator network to optimize the minimax objective function.

15. The non-transitory computer readable medium of claim 14 , wherein:

learning the parameters of the discriminator network to optimize the minimax objective function comprises adjusting the parameters of the discriminator network to maximize probability scores computed by the discriminator network for the ground truth output models and to minimize probability scores computed by the discriminator for the estimated output models estimated by the estimator network from the input training images over a set of training samples; and

learning the parameters of the estimator network to optimize the minimax objective function comprises adjusting the parameters of the estimator network to minimize the error between the ground truth output models and the estimated output models estimated by the estimator network and to maximize the probability scores computed by the discriminator for the estimated models estimated by the estimator network over the set of training samples.