System and method for detecting anomalies in images
Anomalies in images are detected. A generative network and/or an autoencoder (“G/A-Network”), a Siamese network, a first training-dataset of normal images and a second training-dataset of abnormal images are provided. The G/A-network is trained to produce latent data from input images and output images from the latent data, wherein the training is performed with images of the first training-dataset, wherein a loss function is used for training at least at the beginning of training, and the loss function enhances the similarity of the input images and respective output images. The Siamese network is trained to generate similarity measures between input images and respective output images, wherein the training is performed with images of the first training-dataset and the second training-dataset in that images of both training-datasets are used as input images for the G/A-network and output images of the G/A-network are compared with their respective input images by the Siamese network.
1. A method for training a network for detecting anomalies in medical images, the method comprising:
providing a generative network,
providing a Siamese network,
providing a first training dataset comprising medical images of a healthy organ or tissue,
providing a second training-dataset comprising medical images of a non-healthy organ or tissue,
training the generative network to produce latent data from input medical images and output medical images from the latent data, wherein the training is performed with images of the medical images of the healthy organ or tissue, wherein a loss function is used for training at least at a beginning of training, the loss function enhancing a similarity of respective input images and respective output images, and
training the Siamese network to generate similarity measures between respective input images and respective output images of the trained generative network, wherein the training is performed with medical images of the healthy organ or tissue and medical images of the non-healthy organ or tissue that are used as input to the trained generative network, wherein a respective similarity measure below a similarity threshold indicates that the trained generative network input a medical image of the non-healthy organ or tissue and wherein a similarity measure at or above the similarity threshold indicates the trained generative network input a medical image of the healthy organ or tissue, wherein the Siamese network is further configured to normalize the similarity measures,
inputting an input medical image into the trained generative network,
outputting, by the trained generative network an output medical image,
generating, by the trained Siamese network, a similarity measure for the input medical image and the output medical image, and
classifying, by a classification network, a pathology in the input medical image when the similarity measure is below the similarity threshold.
2. The method according to claim 1 , wherein training of the generative network comprises comparing one of the input images with a corresponding one of the output images generated by the generative network using a reconstruction loss function or a perceptual loss function.
3. The method according to claim 2 , wherein the generative network is trained by the comparison of data, wherein the training is assisted by a second Siamese network trained to replace a reconstruction loss function or a perceptual loss function.
4. The method according to claim 1 , wherein the Siamese network is trained to generate similarity measures between the input images and the respective output images directly by comparing the images or indirectly by comparing a latent space generated by the generative network.
5. The method according to claim 1 , wherein the generative network is a generative adversarial network (GAN) or a variable autoencoder (VAE).
6. The method according to claim 1 , wherein the Siamese network is trained to identify the similarity threshold.
7. The method according to claim 1 , wherein the normalization is based on a validation dataset.
8. The method according to claim 1 , wherein the classification network is trained by using at least images of the second training-dataset generated by the generative network from the second training-dataset.
9. The method according to claim 1 , wherein the Siamese network is trained to generate a spatially resolved similarity measure of images, and wherein the classifying is based on an area in an image and the respective similarity measure of the area.
10. The method according to claim 1 , wherein the training of the generative network and the Siamese network is an end-to-end training or wherein results generated by the Siamese network are used to further train the generative network or wherein results generated by the generative network are used to further train the Siamese network.
11. A method for detecting anomalies in images, the method comprising:
inputting an input medical image of an organ or tissue multiple times into a trained generative network, wherein the generative network to produce latent data from a medical image and output an image from the latent data, wherein the generative network is trained with images of healthy images of the organ or tissue, wherein a loss function is used for training at least at a beginning of training the generative network, the loss function enhancing a similarity of the input images and respective output images,
outputting, by the trained generative network a plurality of output medical image,
generating, by a trained Siamese network, a similarity measure for each of plurality of the input image and the output medical image, wherein the Siamese network is trained by inputting healthy and non-healthy medical images of the organ or tissue to the generative network and comparing the output images of the generative network with their respective input images, wherein a similarity measure below a similarity threshold indicates that the trained generative network input a non-healthy medical image of the organ or tissue and wherein a similarity measure at or above the similarity threshold indicates the trained generative network input a healthy medical image of the organ or tissue, wherein a probability distribution is generated for the plurality of similarity measures, and
classifying a pathology in the input medical image when probability distribution for the plurality of similarity measures is below the similarity threshold.
12. An imaging system comprising:
a processor configured to execute a computer program;
a memory configured to store the computer program, the computer program comprising instructions to:
acquire a medical image of an organ or tissue,
apply a generative network trained to produce latent data from the medical image and output an output image from the latent data, wherein the training of the generative network was performed with healthy medical images of an organ or tissue, wherein a loss function was used for training, the loss function being a similarity of input healthy medical images and respective output images,
apply a Siamese network to generate a similarity measure between the medical image and the output image, wherein the training of the Siamese network comprises inputting healthy and non-healthy medical images of the organ or tissue to the generative network and comparing the output images of the generative network with their respective input images, wherein a respective similarity measure below a similarity threshold indicates that the trained generative network input a non-healthy medical image of the organ or tissue and wherein a similarity measure at or above the similarity threshold indicates the trained generative network input a healthy medical image of the organ or tissue, wherein the Siamese network is further configured to normalize the similarity measure,
classify a pathology in the input medical image of the organ or tissue when the similarity measures is below the similarity threshold.