Method and system for artificial intelligence based medical image segmentation

1. A method for autonomous artificial intelligence based medical image segmentation, comprising:

receiving a medical image of a patient;

automatically determining a current segmentation context based on the medical image by inputting the medical image to a trained deep neural network that automatically extracts learned image characteristics from the medical image;

automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms including a plurality of different deep-learning based segmentation algorithms based on the current segmentation context using the trained deep neural network, wherein the trained deep neural network predicts a best segmentation algorithm or combination of segmentation algorithms from the plurality of segmentation algorithms for performing segmentation of a target anatomical structure in the medical image based on the extracted learned image characteristics; and

segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm.

2. The method of claim 1 , wherein the steps of automatically determining the current segmentation context based on the medical image and selecting the at least one segmentation algorithm from the plurality of segmentation algorithms based on the current segmentation context are performed by a software-based trained master segmentation artificial agent running on a computer system.

3. The method of claim 1 , wherein automatically determining a current segmentation context based on the medical image further comprises:

automatically identifying the target anatomical structure based on a view of the medical image currently being visualized on a display device by automatically detecting what anatomical structure is currently being visualized on the display device using a trained machine learning based classifier.

4. The method of claim 3 , wherein automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms comprises:

automatically selecting, by the trained deep neural network, the at least one segmentation algorithm from multiple segmentation algorithms specialized for segmentation of the automatically identified target anatomical structure based on the extracted learned image characteristics.

5. The method of claim 1 , wherein the trained deep neural network is trained based on medical image training samples and synthetic training samples generated at a plurality if noise levels from the medical image training samples.

6. The method of claim 1 , wherein automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms comprises selecting multiple segmentation algorithms from the plurality of segmentation algorithms based on the current segmentation context, and segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:

segmenting the target anatomical structure in the medical image using each of the selected multiple segmentation algorithms; and

fusing segmentation results from the selected multiple segmentation algorithms to generate final segmentation result for the target anatomical structure in the medical image.

7. The method of claim 1 , wherein the plurality of different deep-learning based segmentation algorithms each have a respective deep network architecture trained for a specific segmentation task.

8. The method of claim 1 , wherein segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:

segmenting the target anatomical structure in the medical image using the trained deep neural network with one or more integrated priors.

9. The method of claim 1 , wherein the selected at least one segmentation algorithm comprises a deep reinforcement learning based segmentation algorithm, and segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:

iteratively adjusting a statistical shape model representing a shape of the target anatomical structure in the medical image by selecting, at each iteration, an action corresponding to an adjustment of a parameter of the statistical shape model based on action values calculated using deep neural network trained using deep reinforcement learning.

10. The method of claim 1 , wherein segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:

generating a plurality of Gaussian distributed probability maps at different scales from the medical image using a trained deep image-to-image network;

combining the plurality of Gaussian probability maps at the different scales into a combined probability map; and

extracting a segmented boundary of the target anatomical structure from the medical image based on the combined probability map.

11. The method of claim 1 , wherein the selected at least one segmentation algorithm comprises a deep learning partial inference based segmentation algorithm, and segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:

generating an initial segmentation mask for the target anatomical structure from the medical image using a trained initial convolutional encoder decoder (CED);

constructing a unified tensor combining a current segmentation mask and the medical image; and

generating an updated segmentation mask for the target anatomical structure using a trained partial inference CED.

12. The method of claim 1 , wherein segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:

fitting an active shape model to a shape of the target anatomical structure in the medical image by using a recurrent neural network (RNN) to adjust locations of points on the active shape model.

13. An apparatus for autonomous artificial intelligence based medical image segmentation, comprising:

means for receiving a medical image of a patient;

means for automatically determining a current segmentation context based on the medical image by inputting the medical image to a trained deep neural network that automatically extracts learned image characteristics from the medical image;

means for automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms including a plurality of different deep-learning based segmentation algorithms based on the current segmentation context using the trained deep neural network, wherein the trained deep neural network predicts a best segmentation algorithm or combination of segmentation algorithms from the plurality of segmentation algorithms for performing segmentation of a target anatomical structure in the medical image based on the extracted learned image characteristics; and

means for segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm.

14. The apparatus of claim 13 , wherein the means for automatically determining a current segmentation context based on the medical image further comprises:

means for automatically identifying the target anatomical structure based on a view of the medical image currently being visualized on a display device by automatically detecting what anatomical structure is currently being visualized on the display device using a trained machine learning based classifier.

15. The apparatus of claim 14 , wherein the means for automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms comprises:

means for automatically selecting, by the trained deep neural network, the at least one segmentation algorithm from multiple segmentation algorithms specialized for segmentation of the automatically identified target anatomical structure based on the extracted learned image characteristics.

16. The apparatus of claim 13 , wherein the means for automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms comprises means for selecting multiple segmentation algorithms from the plurality of segmentation algorithms based on the current segmentation context, and the means for segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:

means for segmenting the target anatomical structure in the medical image using each of the selected multiple segmentation algorithms; and

means for fusing segmentation results from the selected multiple segmentation algorithms to generate final segmentation result for the target anatomical structure in the medical image.

17. A non-transitory computer readable medium storing computer program instructions for autonomous artificial intelligence based medical image segmentation, the computer program instructions when executed by a processor perform operations comprising:

receiving a medical image of a patient;

automatically determining a current segmentation context based on the medical image by inputting the medical image to a trained deep neural network that automatically extracts learned image characteristics from the medical image;

automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms including a plurality of different deep-learning based segmentation algorithms based on the current segmentation context using the trained deep neural network, wherein the trained deep neural network predicts a best segmentation algorithm or combination of segmentation algorithms from the plurality of segmentation algorithms for performing segmentation of a target anatomical structure in the medical image based on the extracted learned image characteristics; and

segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm.

18. The non-transitory computer readable medium of claim 17 , wherein automatically determining a current segmentation context based on the medical image further comprises:

automatically identifying the target anatomical structure based on a view of the medical image currently being visualized on a display device by automatically detecting what anatomical structure is currently being visualized on the display device using a trained machine learning based classifier.

19. The non-transitory computer readable medium of claim 18 , wherein automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms comprises:

automatically selecting, by the trained deep neural network, the at least one segmentation algorithm from multiple segmentation algorithms specialized for segmentation of the automatically identified target anatomical structure based on the extracted learned image characteristics.

20. The non-transitory computer readable medium of claim 17 , wherein automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms comprises selecting multiple segmentation algorithms from the plurality of segmentation algorithms based on the current segmentation context, and segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:

segmenting the target anatomical structure in the medical image using each of the selected multiple segmentation algorithms; and

fusing segmentation results from the selected multiple segmentation algorithms to generate final segmentation result for the target anatomical structure in the medical image.

21. The non-transitory computer readable medium of claim 17 , wherein the plurality of different deep-learning based segmentation algorithms each have a respective deep network architecture trained for a specific segmentation task.