Medical image segmentation with uncertainty estimation
Systems and methods for generating a segmentation mask of an anatomical structure, along with a measure of uncertainty of the segmentation mask, are provided. In accordance with one or more embodiments, a plurality of candidate segmentation masks of an anatomical structure is generated from an input medical image using one or more trained machine learning networks. A final segmentation mask of the anatomical structure is determined based on the plurality of candidate segmentation masks. A measure of uncertainty associated with the final segmentation mask is determined based on the plurality of candidate segmentation masks. The final segmentation mask and/or the measure of uncertainty are output.
1. A method comprising:
generating a plurality of candidate segmentation masks of an anatomical structure from an input medical image using one or more trained machine learning networks;
determining a final segmentation mask of the anatomical structure by calculating a mean of activation values of corresponding pixels across the plurality of candidate segmentation masks; and
determining a measure of uncertainty associated with the final segmentation mask by calculating a variance of values of corresponding pixels across the plurality of candidate segmentation masks to generate a variance map.
2. The method of claim 1 , wherein generating a plurality of candidate segmentation masks of an anatomical structure from an input medical image using one or more trained machine learning networks comprises:
sampling a plurality of samples from a prior distribution, the prior distribution being a probability distribution of segmentation variations of the anatomical structure in the input medical image; and
for each respective sample of the plurality of samples, generating a candidate segmentation mask based on the respective sample.
3. The method of claim 1 , wherein the one or more trained machine learning networks comprise a plurality of different trained machine learning networks, and generating a plurality of candidate segmentation masks of an anatomical structure from an input medical image using one or more trained machine learning networks comprises:
for each respective trained machine learning network of the plurality of different trained machine learning networks, generating a candidate segmentation mask of the anatomical structure from the input medical image using the respective trained machine learning network.
4. The method of claim 1 , wherein determining a measure of uncertainty associated with the final segmentation mask by calculating a variance of values of corresponding pixels across the plurality of candidate segmentation masks to generate a variance map comprises:
determining a probability associated with each pixel that may represent a boundary of the final segmentation mask.
5. The method of claim 4 , wherein determining a measure of uncertainty associated with the final segmentation mask by calculating a variance of values of corresponding pixels across the plurality of candidate segmentation masks to generate a variance map comprises:
averaging the probability associated with each pixel that may represent the boundary of the final segmentation mask.
6. The method of claim 4 , further comprising:
comparing the probability associated with each pixel that may represent the boundary of the final segmentation mask with a threshold;
assigning a color to each pixel that may represent the boundary of the final segmentation mask based on the comparing; and
overlaying the color for each pixel that may represent the boundary of the final segmentation mask on the input medical image.
7. The method of claim 1 , further comprising:
requesting user input for the final segmentation mask based on the measure of uncertainty.
8. The method of claim 1 , further comprising:
detecting anomalies in the input medical image based on the measure of uncertainty.
9. The method of claim 1 , further comprising:
identifying training data for further training the one or more trained machine learning networks based on the measure of uncertainty.
10. An apparatus, comprising:
means for generating a plurality of candidate segmentation masks of an anatomical structure from an input medical image using one or more trained machine learning networks;
means for determining a final segmentation mask of the anatomical structure by calculating a mean of activation values of corresponding pixels across the plurality of candidate segmentation masks; and
means for determining a measure of uncertainty associated with the final segmentation mask by calculating a variance of values of corresponding pixels across the plurality of candidate segmentation masks to generate a variance map.
11. The apparatus of claim 10 , wherein the means for generating a plurality of candidate segmentation masks of an anatomical structure from an input medical image using one or more trained machine learning networks comprises:
means for sampling a plurality of samples from a prior distribution, the prior distribution being a probability distribution of segmentation variations of the anatomical structure in the input medical image; and
means for generating, for each respective sample of the plurality of samples, a candidate segmentation mask based on the respective sample.
12. The apparatus of claim 10 , wherein the one or more trained machine learning networks comprise a plurality of different trained machine learning networks, and the means for generating a plurality of candidate segmentation masks of an anatomical structure from an input medical image using one or more trained machine learning networks comprises:
means for generating, for each respective trained machine learning network of the plurality of different trained machine learning networks, a candidate segmentation mask of the anatomical structure from the input medical image using the respective trained machine learning network.
13. A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
generating a plurality of candidate segmentation masks of an anatomical structure from an input medical image using one or more trained machine learning networks;
determining a final segmentation mask of the anatomical structure by calculating a mean of activation values of corresponding pixels across the plurality of candidate segmentation masks; and
determining a measure of uncertainty associated with the final segmentation mask by calculating a variance of values of corresponding pixels across the plurality of candidate segmentation masks to generate a variance map.
14. The non-transitory computer readable medium of claim 13 , wherein determining a measure of uncertainty associated with the final segmentation mask by calculating a variance of values of corresponding pixels across the plurality of candidate segmentation masks to generate a variance map comprises:
determining a probability associated with each pixel that may represent a boundary of the final segmentation mask.
15. The non-transitory computer readable medium of claim 14 , wherein determining a measure of uncertainty associated with the final segmentation mask by calculating a variance of values of corresponding pixels across the plurality of candidate segmentation masks to generate a variance map comprises:
averaging the probability associated with each pixel that may represent the boundary of the final segmentation mask.
16. The non-transitory computer readable medium of claim 13 , the operations further comprising:
requesting user input for the final segmentation mask based on the measure of uncertainty.