Masking model for visualizing uncertainty in image-to-image reconstruction
Methods and systems of processing medical image data are presented herein. Some methods may include the steps of: receiving image data representing at least one medical image; generating a reconstructed image based on the image data; generating a mask from the image data and the reconstructed image using a pre-trained machine learning model; and applying the mask to the reconstructed image to generate a masked reconstructed image. The mask may include at least three uncertainty levels for a plurality of pixels in the reconstructed image.
1 . A method of processing medical image data using a processor, the method comprising:
receiving image data representing a medical image;
generating a reconstructed image based on the image data;
generating a mask from the image data and the reconstructed image using a pre-trained machine learning model, wherein the mask comprises a plurality of pixels, each pixel of the plurality of pixels in the mask corresponding to a corresponding pixel of a plurality of pixels in the reconstructed image in a manner that the each pixel of the plurality of pixels in the mask has a corresponding uncertainty value selected from at least three uncertainty levels, the uncertainty value for the each pixel of the plurality of pixels in the mask being based on a distance between the corresponding pixel of the plurality of pixels in the reconstructed image and a corresponding pixel of the image data; and
applying the mask to the reconstructed image to generate a masked reconstructed image.
2 . The method of claim 1 , wherein each of the at least three uncertainty levels is a value in a range of 0 to 1.
3 . The method of claim 1 , wherein generating the reconstructed image comprises colorization of the image data.
4 . The method of claim 1 , wherein generating the reconstructed image comprises identifying and estimating missing parts of the received image data.
5 . The method of claim 1 , wherein the image data represents an image of a first resolution, wherein the reconstructed image represents an image of a second resolution, and wherein the second resolution is higher than the first resolution.
6 . The method of claim 1 , wherein receiving the image data comprises collecting the image data from a medical device.
7 . The method of claim 6 , wherein the medical device is at least one of: an endoscope, a magnetic resonance imaging device, ultrasound device, and X-ray device.
8 . The method of claim 1 , wherein the pre-trained machine learning model is trained on a series of images to generate a mask for each image such that the distance between each masked reconstructed image and a corresponding true image is less than a first threshold with probability greater than a second threshold.
9 . The method of claim 1 , wherein the mask represents a visual representation of confidence that portions of the reconstructed image are accurate.
10 . The method of claim 1 , further comprising displaying the mask and the masked reconstructed image on a display.
11 . A system for processing medical image data comprising:
an input interface configured to receive image data;
a memory configured to store a plurality of processor-executable instructions, the memory comprising:
an image-to-image model;
a masking model, wherein the masking model comprises a pre-trained machine learning model; and
a processor configured to execute the plurality of processor-executable instructions to perform operations including:
receiving image data representing a medical image;
generating a reconstructed image based on the image data using the image-to-image model;
generating a mask from the image data and the reconstructed image using the masking model, wherein the mask comprises a plurality of pixels, each pixel of the plurality of pixels in the mask corresponding to a corresponding pixel of a plurality of pixels in the reconstructed image in a manner that the each pixel of the plurality of pixels in the mask has a corresponding uncertainty value selected from at least three uncertainty levels, the uncertainty value for the each pixel of the plurality of pixels in the mask being based on a distance between the corresponding pixel of the plurality of pixels in the reconstructed image and a corresponding pixel of the image data; and
generating a masked reconstructed image based on the reconstructed image and the mask.
12 . The system of claim 11 , wherein each of the at least three uncertainty levels is a value in a range of 0 to 1.
13 . The system of claim 11 , wherein the pre-trained machine learning model is trained on a series of images to generate a mask for each image such that the distance between each masked reconstructed image and a corresponding true image is less than a first threshold with probability greater than a second threshold.
14 . The system of claim 11 , wherein the mask represents a visual representation of confidence that portions of the reconstructed image are accurate.
15 . The system of claim 11 , further comprising displaying the mask and the masked reconstructed image on a display.
16 . A non-transitory processor-readable storage medium storing a plurality of processor-executable instructions for processing medical image data, the instructions being executed by a processor to perform operations comprising:
receiving image data representing at least one medical image;
generating a reconstructed image based on the image data;
generating a mask from the image data and the reconstructed image using a pre-trained machine learning model, wherein the mask comprises a plurality of pixels, each pixel of the plurality of pixels in the mask corresponding to a corresponding pixel of a plurality of pixels in the reconstructed image in a manner that the each pixel of the plurality of pixels in the mask has a corresponding uncertainty value selected from at least three uncertainty levels, the uncertainty value for the each pixel of the plurality of pixels in the mask being based on a distance between the corresponding pixel of the plurality of pixels in the reconstructed image and a corresponding pixel of the image data; and
generating a masked reconstructed image based on the reconstructed image and the mask.
17 . The non-transitory processor-readable storage medium of claim 16 , wherein each of the at least three uncertainty levels is a value in a range of 0 to 1.
18 . The non-transitory processor-readable storage medium of claim 16 , wherein the pre-trained machine learning model is trained on a series of images to generate a mask for each image such that the distance between each masked reconstructed image and a corresponding true image is less than a first threshold with probability greater than a second threshold.
19 . The non-transitory processor-readable storage medium of claim 16 , wherein the mask represents a visual representation of confidence that portions of the reconstructed image are accurate.
20 . The non-transitory processor-readable storage medium of claim 16 , further comprising displaying the mask and the masked reconstructed image on a display.