Uncertainty estimation in medical imaging
For uncertainty estimation for a machine-learned model prediction in medical imaging, a distribution in latent space is sampled to determine uncertainty in machine-learned model prediction. For example, a segmentation is output by a machine-learned model in response to input of multi-parametric data. A latent space generated by the machine-learned model is used to estimate the uncertainty of the segmentation, such as a segmentation of a prostate lesion. The model of Kohl, et al., may be used as the variational auto encoder generates a latent space representing the distribution of the training data, which latent space may be used to determine uncertainty.
1. A method for lesion segmentation for a medical imager, the method comprising:
acquiring, by the medical imager, multi-parametric data representing a patient;
generating, by a machine-learned network, a segmentation of a lesion represented in the multi-parametric data, the segmentation generated in response to input of the multi-parametric data to the machine-learned network, wherein the machine-learned network comprises a U-Net with convolutional layers and an encoder, and wherein generating comprises inputting the multi-parametric data to both the U-Net and the encoder and feeding a sample from a latent space at an output of the encoder to one of the convolutional layers of the U-Net, the U-Net generating the segmentation based in part on the input of the multi-parametric data and in part on the sample from the latent space;
determining an uncertainty of the segmentation, the uncertainty determined using the machine-learned network; and
displaying the segmentation and the uncertainty of the segmentation.
2. The method of claim 1 wherein the medical imager comprises a magnetic resonance (MR) imager, and wherein acquiring comprises acquiring the multi-parametric data as T2-weighted data, apparent diffusion coefficient data, and diffusion-weighted data.
3. The method of claim 2 wherein acquiring comprises acquiring the multi-parametric data representing a prostate region of the patient, and wherein generating comprises generating the segmentation of the lesion as a prostate lesion, the segmentation of the prostate lesion generated in response to input of the T2-weighted data, apparent diffusion coefficient data, diffusion-weighted data, and a prostate mask.
4. The method of claim 1 wherein the U-Net comprises a residual U-Net and the encoder comprises a conditional variational auto encoder, and wherein generating comprises concatenating the sample from the latent space with a last activation map of the residual U-Net.
5. The method of claim 1 wherein generating comprises generating the segmentation and additional segmentations, the additional segmentations generated with different samples from the latent space.
6. The method of claim 5 wherein generating comprises generating the segmentation and additional segmentations with the different samples being randomly sampled from the latent space.
7. The method of claim 6 wherein determining the uncertainty comprises calculating a variance of the segmentation and additional segmentations, the variance being a heat map representing the uncertainty.
8. The method of claim 1 wherein the sample comprises a mean in the latent space, wherein generating the segmentation comprises generating the segmentation based in part on the mean in the latent space, and wherein determining the uncertainty comprises determining a variation in the latent space and generating a heat map by the U-Net based in part on the variation.
9. The method of claim 1 wherein generating comprises generating the segmentation and additional segmentations by the machine-learned network, and wherein determining the uncertainty comprises determining a variance of the segmentation and additional segmentations.
10. The method of claim 1 wherein displaying comprises displaying the segmentation as a mean of multiple predictions by the machine-learned network and displaying the uncertainty as a heatmap by spatial location of the segmentation.
11. A medical system for uncertainty estimation, the medical system comprising:
an imager configured to capture scan data representing a patient;
an image processor configured to apply the scan data to a machine-learned network, the machine-learned network configured to output a classification, detection, and/or segmentation in response to the application, the image processor further configured to determine an uncertainty of the classification, detection, and/or segmentation with random samples from a latent space, wherein the machine-learned network comprises a U-Net and an encoder separate from the U-Net, the encoder configured to generate the latent space, wherein the U-Net is configured to output a plurality of the classification, detection, and/or segmentation in response to the application of the scan data representing the patient using the random samples and wherein the uncertainty is a variance of the plurality of the classification, detection, and/or segmentation; and
a display configured to display the classification, detection, and/or segmentation and to display the uncertainty.
12. The medical system of claim 11 wherein the imager comprises a magnetic resonance (MR) imager, wherein the scan data comprises multi-parametric MR data, wherein the machine-learned network is configured to output the segmentation as a segmentation of a lesion, and wherein the uncertainty is a heat map with greater uncertainty for some locations than others represented in the heat map.
13. The medical system of claim 11 wherein the display of the classification, detection, and/or segmentation is display of a mean of the plurality.
14. A medical system for uncertainty estimation, the medical system comprising:
an imager configured to capture scan data representing a patient;
an image processor configured to apply the scan data to a machine-learned network, the machine-learned network configured to output a classification, detection, and/or segmentation in response to the application, the image processor further configured to determine an uncertainty of the classification, detection, and/or segmentation with random samples from a latent space, wherein the machine-learned network comprises a U-Net and an encoder separate from the U-Net, the encoder configured to generate the latent space, wherein the uncertainty comprises the output based on a variance of the random samples; and
a display configured to display the classification, detection, and/or segmentation and to display the uncertainty.