Automated segmentation utilizing fully convolutional networks
Systems and methods for automated segmentation of anatomical structures (e.g., heart). Convolutional neural networks (CNNs) may be employed to autonomously segment parts of an anatomical structure represented by image data, such as 3D MRI data. The CNN utilizes two paths, a contracting path and an expanding path. In at least some implementations, the expanding path includes fewer convolution operations than the contracting path. Systems and methods also autonomously calculate an image intensity threshold that differentiates blood from papillary and trabeculae muscles in the interior of an endocardium contour, and autonomously apply the image intensity threshold to define a contour or mask that describes the boundary of the papillary and trabeculae muscles. Systems and methods also calculate contours or masks delineating the endocardium and epicardium using the trained CNN model, and anatomically localize pathologies or functional characteristics of the myocardial muscle using the calculated contours or masks.
1. A machine learning system, comprising:
at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data, medical imaging data of the heart, and a trained convolutional neural network (CNN) model; and
at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, in operation, the at least one processor:
calculates contours or masks delineating the endocardium and epicardium of the heart in the medical imaging data using the trained CNN model; and
anatomically localizes pathologies or functional characteristics of the myocardial muscle using the calculated contours or masks.
2. The machine learning system of claim 1 wherein the at least one processor calculates the ventricular insertion points at which the right ventricular wall attaches to the left ventricle.
3. The machine learning system of claim 2 wherein the at least one processor calculates the ventricular insertion points based on the proximity of contours or masks delineating the left ventricle epicardium to one or both of the right ventricle endocardium or the right ventricle epicardium.
4. The machine learning system of claim 3 wherein the at least one processor calculates the ventricular insertion points in one or more two-dimensional cardiac images based on the two points in the cardiac image in which the left ventricle epicardium boundary diverges from one or both of the right ventricle endocardium boundary or the right ventricle epicardium boundary.
5. The machine learning system of claim 2 wherein the at least one processor calculates the ventricular insertion points based on the intersection between acquired long axis views of the left ventricle and the delineation of the left ventricle epicardium.
6. The machine learning system of claim 5 wherein the at least one processor calculates at least one ventricular insertion point based on the intersection between the left ventricle epicardium contour and the left heart 3-chamber long axis plane.
7. The machine learning system of claim 5 wherein the at least one processor calculates at least one ventricular insertion point based on the intersection between the left ventricle epicardium contour and the left heart 4-chamber long axis plane.
8. The machine learning system of claim 5 wherein the at least one processor calculates at least one ventricular insertion point based on the intersection between the left heart 3-chamber long axis plane and one or both of the right ventricle epicardium contour or the right ventricle endocardium contour.
9. The machine learning system of claim 5 wherein the at least one processor calculates at least one ventricular insertion point based on the intersection between the left heart 4-chamber long axis plane and one or both of the right ventricle epicardium contour or the right ventricle endocardium contour.
10. The machine learning system of claim 1 wherein the at least one processor allows a user to manually delineate the location of one or more of the ventricular insertion points.
11. The machine learning system of claim 1 wherein the at least one processor uses a combination of contours and ventricular insertion points to present the anatomical location of pathologies or functional characteristics of the myocardial muscle in a standardized format.
12. The machine learning system of claim 11 wherein the standardized format is one or both of a 16- or 17-segment model of the myocardial muscle.
13. The machine learning system of claim 1 wherein the medical imaging data of the heart is one or more of functional cardiac images, myocardial delayed enhancement images or myocardial perfusion images.
14. The machine learning system of claim 13 wherein the medical imaging data of the heart is cardiac magnetic resonance images.
15. The machine learning system of claim 1 wherein the trained CNN model has been trained on annotated cardiac images of the same type as those for which the trained CNN model will be used for inference.
16. The machine learning system of claim 15 wherein the trained CNN model has been trained on one or more of functional cardiac images, myocardial delayed enhancement images or myocardial perfusion images.
17. The machine learning system of claim 16 wherein the data on which the trained CNN model has been trained are cardiac magnetic resonance images.
18. The machine learning system of claim 1 wherein the trained CNN model has been trained on annotated cardiac images of a different type than those for which the trained CNN model will be used for inference.
19. The machine learning system of claim 18 wherein the trained CNN model has been trained on one or more of functional cardiac images, myocardial delayed enhancement images or myocardial perfusion images.
20. The machine learning system of claim 19 wherein the data on which the trained CNN model has been trained are cardiac magnetic resonance images.
21. The machine learning system of claim 1 wherein the at least one processor fine tunes the trained CNN model on data of the same type for which the CNN model will be used for inference.
22. The machine learning system of claim 21 wherein, to fine tune the trained CNN model, the at least one processor retrains some or all of the layers of the trained CNN model.
23. The machine learning system of claim 1 wherein the at least one processor applies postprocessing to the contours or masks delineating the endocardium and epicardium of the heart to minimize the amount of non-myocardial tissue that is present in the region of the heart identified as myocardium.
24. The machine learning system of claim 23 wherein, to postprocess the contours or masks, the at least one processor applies morphological operations to the region of the heart identified as myocardium to reduce its area.
25. The machine learning system of claim 24 wherein the morphological operations include one or more of erosion or dilation.
26. The machine learning system of claim 23 wherein, to postprocess the contours or masks, the at least one processor modifies the threshold applied to probability maps predicted by the trained CNN model to only identify pixels of myocardium for which the trained CNN model expresses a probability above a threshold that the pixels are part of the myocardium.
27. The machine learning system of claim 26 wherein the threshold by which probability map values are converted to class labels is greater than 0.5.
28. The machine learning system of claim 23 wherein, to postprocess the contours or masks, the at least one processor shifts vertices of contours that delineate the myocardium towards or away from the center of the ventricle of the heart to reduce the identified area of myocardium.
29. The machine learning system of claim 1 wherein the pathologies or functional characteristics of the myocardial muscle include one or more of myocardial scarring, myocardial infarction, coronary stenosis, or perfusion characteristics.