Methods and systems for flexible denoising of images using disentangled feature representation field
A system and method are provided for denoising images. A standard image module is configured to generate a standard anatomy feature and a standard noise feature from a standard image and reconstruct the standard image from the standard anatomy feature and the standard noise feature. A reduced quality image module is configured to generate a reduced quality anatomy feature and a reduced quality noise feature from a reduced quality image, and reconstruct the reduced quality image from the reduced quality anatomy feature and the reduced quality noise feature. A loss calculation module is provided for calculating loss metrics at least partially based on a comparison between 1) the reconstructed standard image and the standard image, and 2) the reconstructed reduced quality image and the reduced quality image. Upon providing the standard image module with the reduced quality anatomy feature, the standard image module outputs a reconstructed standard transfer image.
1 . A system for denoising medical images, comprising:
a standard image module configured to generate a standard anatomy feature and a standard noise feature from a standard image, and reconstruct the standard image from the standard anatomy feature and the standard noise feature;
a reduced quality image module configured to generate a reduced quality anatomy feature and a reduced quality noise feature from a reduced quality image, and reconstruct the reduced quality image from the reduced quality anatomy feature and the reduced quality noise feature;
a loss calculation module configured to calculate loss metrics comprising (i) a first reconstruction loss based at least in part on a comparison between the reconstructed standard image and the standard image; and (ii) a second reconstruction loss based at least in part on a comparison between the reconstructed reduced quality image and the reduced quality image, wherein the first reconstruction loss and the second reconstruction loss are computed independently of each other and are not cross-compared;
wherein the loss metrics are used for tuning the standard image module and the reduced quality image module using machine learning, wherein when the standard image module is provided with the reduced quality anatomy feature, the standard image module outputs a reconstructed standard transfer image that comprises the reduced quality anatomy feature and a noise level lower than that represented by the reduced quality noise feature, and wherein the loss calculation module comprises a standard anatomy encoder and is further configured to calculate a loss metric for the standard anatomy encoder based on a comparison between the standard anatomy feature and segmentation labels for the standard image.
2 . The system of claim 1 , wherein:
the standard image module comprises a standard noise encoder, and a standard generator, and wherein upon receipt of the standard image, the standard anatomy encoder outputs the standard anatomy feature, the standard noise encoder outputs the standard noise feature, and the standard generator reconstructs the standard image from the standard anatomy feature and the standard noise feature;
the reduced quality image module comprises a reduced quality anatomy encoder, a reduced quality noise encoder, and a reduced quality generator, and wherein upon receipt of the reduced quality image, the reduced quality anatomy encoder outputs the reduced quality anatomy feature, the reduced quality noise encoder outputs the reduced quality noise feature, and the reduced quality generator reconstructs the reduced quality image from the reduced quality anatomy feature and the reduced quality noise feature;
wherein the loss calculation module calculates a loss metric for the standard generator at least partially based on the comparison between the reconstructed standard image and the standard image;
wherein the loss calculation module calculates a loss metric for the reduced quality generator at least partially based on the comparison between the reconstructed reduced quality image and the reduced quality image;
and
wherein the loss calculation module calculates a loss metric for the reduced quality anatomy encoder based on a comparison with the output of the standard anatomy encoder.
3 . The system of claim 2 , wherein the loss metric for the reduced quality anatomy encoder is an adversarial loss metric.
4 . The system of claim 1 further comprising a segmentation network, and wherein a segmentation mask for the reconstructed standard image is evaluated based on a comparison with segmentation labels for the standard image.
5 . The system of claim 4 , wherein when the reduced quality generator is provided with a standard anatomy feature, the reduced quality generator outputs a reconstructed reduced quality transfer image that comprises the standard anatomy features and a noise level higher than that represented by the standard noise feature, and wherein a segmentation mask for the reduced quality transfer image is evaluated based on a comparison with the segmentation labels for the standard image.
6 . The system of claim 1 , wherein a loss metric for the standard transfer image is evaluated based on a comparison with a standard image reconstruction, and wherein the loss metric for the standard transfer image is an adversarial loss metric.
7 . The system of claim 1 wherein the standard image module and the reduced quality image module are trained simultaneously.
8 . The system of claim 1 , wherein the standard image module is trained prior to the training of the reduced quality image module, and wherein values for variables developed while training the standard image module are held constant during training of the reduced quality image module.
9 . The system of claim 8 , wherein after training the standard image module and the reduced quality image module, the system further trains the standard generator while holding constant values for the standard anatomy encoder, the standard noise encoder, and the reduced quality anatomy encoder.
10 . The system of claim 1 , wherein the standard anatomy feature and the reduced quality anatomy feature each correspond to a single anatomical structure.
11 . A method for denoising medical images, comprising:
generating, by a standard image module, a standard anatomy feature and a standard noise feature from a standard image;
reconstructing, by a standard image module, the standard image from the standard anatomy feature and the standard noise feature;
generating, by a reduced quality image module, a reduced quality anatomy feature and a reduced quality noise feature from a reduced quality image;
reconstructing, by the reduced quality image module, the reduced quality image from the reduced quality anatomy feature and the reduced quality noise feature; and
calculating, by a loss calculation module, loss metrics comprising (i) a first reconstruction loss based at least in part on a comparison between the reconstructed standard image and the standard image; and (ii) a second reconstruction loss based at least in part on a comparison between the reconstructed reduced quality image and the reduced quality image, wherein the first reconstruction loss and the second reconstruction loss are computed independently of each other and are not cross-compared;
wherein the loss metrics are used for tuning the standard image module and the reduced quality image module using machine learning, wherein when the standard image module is provided with the reduced quality anatomy feature, the standard image module outputs a reconstructed standard transfer image that comprises the reduced quality anatomy feature and a noise level lower than that represented by the reduced quality noise feature, and wherein the loss calculation module comprises a standard anatomy encoder and is further configured to calculate a loss metric for the standard anatomy encoder based on a comparison between the standard anatomy feature and segmentation labels for the standard image.
12 . The method of claim 11 , wherein:
the standard image module comprises a standard noise encoder, and a standard generator, and wherein upon receipt of the standard image, the standard anatomy encoder outputs the standard anatomy feature, the standard noise encoder outputs the standard noise feature, and the standard generator reconstructs the standard image from the standard anatomy feature and the standard noise feature;
the reduced quality image module comprises a reduced quality anatomy encoder, a reduced quality noise encoder, and a reduced quality generator, and wherein upon receipt of the reduced quality image, the reduced quality anatomy encoder outputs the reduced quality anatomy feature, the reduced quality noise encoder outputs the reduced quality noise feature, and the reduced quality generator reconstructs the reduced quality image from the reduced quality anatomy feature and the reduced quality noise feature;
wherein the loss calculation module calculates a loss metric for the standard generator at least partially based on the comparison between the reconstructed standard image and the standard image;
wherein the loss calculation module calculates a loss metric for the reduced quality generator at least partially based on the comparison between the reconstructed reduced quality image and the reduced quality image;
and
wherein the loss calculation module calculates a loss metric for the reduced quality anatomy encoder based on a comparison with the output of the standard anatomy encoder.
13 . The method of claim 12 , wherein the loss metric for the reduced quality anatomy encoder is an adversarial loss metric.
14 . The method of claim 11 further comprising generating a segmentation mask for the reconstructed standard image at a segmentation network and evaluating the segmentation mask based on a comparison with segmentation labels for the standard image.
15 . The method of claim 14 wherein, upon providing the reduced quality generator with a standard anatomy feature, the reduced quality generator outputs a reconstructed reduced quality transfer image, the reduced quality transfer image including the standard anatomy features and a noise level higher than that represented by the standard noise feature, and wherein a segmentation mask for the reduced quality transfer image is evaluated based on a comparison with the segmentation labels for the standard image.
16 . The method of claim 11 , wherein a loss metric for the standard transfer image is evaluated based on a comparison with a standard image reconstruction, and wherein the loss metric for the standard transfer image is an adversarial loss metric.
17 . The method of claim 11 wherein the standard image module and the reduced quality image module are trained simultaneously.
18 . The method of claim 11 , wherein the standard image module is trained prior to the training of the reduced quality image module, and wherein values for variables developed while training the standard image module are held constant during training of the reduced quality image module.
19 . The method of claim 18 , wherein after training the standard image module and the reduced quality image module, the method further trains the standard generator while holding constant values for the standard anatomy encoder, the standard noise encoder, and the reduced quality anatomy encoder.
20 . The method of claim 11 , wherein the standard anatomy feature and the reduced quality anatomy feature each correspond to a single anatomical structure.