Method and system for automatic multiple lesion annotation of medical images
A method includes receiving, from a patient, an image having a visible lesion, modifying the image to appear as if the lesion were not present, thereby forming a second image, generating a delineation of the abnormality using a difference between the first and second images, and tagging the segmented lesions.
1. A method comprising annotating medical images in a database of medical images, said database including first and second sets of images, said first set consisting of images that are known to have lesions, and said second set consisting of images that are known to be free of lesions, wherein annotating said medical images comprises:
using at least images that are stored in said database to determine values of first parameters and values of second parameters, said first parameters being for configuring a first image analysis component and said second parameters being for configuring a second image analysis component,
wherein said first image analysis component is configured to produce an output based on an input image from said first set, said output including a synthetic normal image and lesion data, said synthetic normal image having been formed by modifying said input image to reduce an amount of data indicative of a lesion in said input image, said lesion data characterizing said lesion,
wherein said second image analysis component is configured to discriminate between images from said second set and said synthetic normal image, and
wherein determining values of said parameters results in parameters that hinder said second image analysis component in said discrimination and that also promote an ability to reconstruct said input image from said output of said first image analysis component, and
processing each image of said first set using said first image analysis component to generate annotated images according to said lesion data output by said first image analysis component with said first image as input, said first image analysis component having been configured with said values of said first parameters.
2. The method of claim 1 , wherein said synthetic normal image is one of a plurality of synthetic normal images and wherein said method further comprises using a computer-implemented training controller to determine values of first and second parameters for a parameterized discriminator that distinguishes real normal images, which are from normal patients, from said synthetic normal images, wherein said controller determines said values of said first parameters to reduce an aggregate measure of discriminability between real normal images and said synthetic normal images and said controller selects said values of said second parameters to increase said discriminability.
3. The method of claim 1 , further comprising suppressing modification of said input image at points in said input image that are outside said lesion.
4. The method of claim 3 , wherein suppressing said modification comprises comparing said first image to said second image.
5. The method of claim 1 , further comprising obtaining said medical images from a set of images obtained by computerized tomography.
6. The method of claim 5 , wherein the first image consists of a first region, which includes the lesion, and a second region, which excludes the lesion, and wherein the first and second regions differ in intensity.
7. The method of claim 1 , further comprising obtaining said first set of images by magnetic resonance imaging.
8. The method of claim 7 , wherein said first set includes a first image that consists of a first region, which includes the lesion, and a second region, which excludes the lesion, and wherein the first and second regions differ in intensity.
9. The method of claim 1 , wherein determining values of said parameters comprises updating said values during the course of a plurality of iterations, wherein updating said values includes processing an image from said first set, using said first image analysis component to output a corresponding synthetic normal image and corresponding lesion data and, providing said synthetic normal image to said second image analysis component to predict whether said synthetic normal image is from said second set or a synthetic normal image, wherein said second image analysis component mistakenly identifies said synthetic normal image as being an image from said second set.
10. The method of claim 1 , further comprising processing said synthetic normal image and said lesion data to produce a synthetic abnormal image and determining an extent to which said synthetic abnormal image differs from said abnormal image that was used as a basis for generating said synthetic normal image.
11. The method of claim 1 , wherein said lesion data characterizes a spatial extent of a region occupied by said lesion.
12. The method of any of claim 1 , wherein said lesion data comprises image data in the region.
13. The method of any of claim 1 , wherein the first image processing component and the second image processing component each comprises an artificial neural network.
14. The method of claim 13 , wherein determining the values of the parameters comprises performing an adversarial training procedure using a combined metric representing an overall ability to discriminate between the images of the normal set of images and the images formed by the first image analysis component and an ability to reconstruct images from the abnormal set from the outputs of the first image analysis component.
15. The method of claim 1 , wherein said input image is an image of a patient having abnormal lesions, wherein said method further comprises receiving an age-specific template created by averaging normalized images over a population of healthy subject images in a certain age range and generating a delineation of a lesion using a statistical voxel comparison between a normalized image without abnormality or disease and without lesions.
16. The method of claim 15 , further comprising segmenting the lesion, using two encoder-decoder networks to determine parameter values of transformation parameters for anatomical normalization and selecting the values of the transformation parameters to maximize a similarity measure between a warped image in a warped space and a source image in a source space, selecting a threshold value of the segmented lesion generated by analyzing statistical voxel comparison between the normalized images of patient with lesions and the normalized images of patients without lesions, and selecting the parameters of the inverse transformation to transform the lesion segment on the warped space into said source space.
17. The method of claim 1 , further comprising using an encoder-decoder segmentation network training controller to determine values for the parameters of a parameterized segmentation module for delineating lesions from patients and selecting parameter values to reduce a pixel-level loss function between annotated images from initial seed data sets and the predicted images from a segmentation module and selecting updated annotation data sets by applying a likelihood value between representative density functions of seed annotation data and annotated lesions predicted from a previously trained segmentation module, wherein said loss function is selected from the group consisting of a cross-entropy and a Dice similarity coefficient.
18. The method of claim 1 , wherein using said first image analysis component to generate said annotated images comprises segmenting and tagging abnormal lesions.
19. The method of claim 1 , wherein using said first image analysis component to generate said annotated images comprises implementing multi-tasking annotation.