Automated segmentation utilizing fully convolutional networks
Systems and methods for automated segmentation of anatomical structures (e.g., heart). Convolutional neural networks (CNNs) may be employed to autonomously segment parts of an anatomical structure represented by image data, such as 3D MRI data. The CNN utilizes two paths, a contracting path and an expanding path. In at least some implementations, the expanding path includes fewer convolution operations than the contracting path. Systems and methods also autonomously calculate an image intensity threshold that differentiates blood from papillary and trabeculae muscles in the interior of an endocardium contour, and autonomously apply the image intensity threshold to define a contour or mask that describes the boundary of the papillary and trabeculae muscles. Systems and methods also calculate contours or masks delineating the endocardium and epicardium using the trained CNN model, and anatomically localize pathologies or functional characteristics of the myocardial muscle using the calculated contours or masks.
1. A machine learning system, comprising:
at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data; and
at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, the at least one processor:
receives learning data comprising a plurality of batches of labeled image sets, each image set comprising image data representative of an anatomical structure, and each image set including at least one label which identifies the region of a particular part of the anatomical structure depicted in each image of the image set;
trains a fully convolutional neural network (CNN) model to segment at least one part of the anatomical structure utilizing the received learning data; and
stores the trained CNN model in the at least one nontransitory processor-readable storage medium of the machine learning system, wherein the CNN model comprises a contracting path and an expanding path, the contracting path includes a number of convolutional layers and a number of pooling layers, each pooling layer preceded by at least one convolutional layer, and the expanding path includes a number of convolutional layers and a number of upsampling layers, each upsampling layer preceded by at least one convolutional layer and comprising a fixed upsampling operation without a learned kernel and a 2×2 convolution operation with a learned kernel, the 2×2 convolution operation immediately preceded by the fixed upsampling operation and immediately succeeded by a concatenation of feature maps via a skip connection from the contracting path.
2. The machine learning system of claim 1 wherein the contracting path includes first and second parallel paths, one of the first and second parallel paths includes a number of convolutional layers and a number of pooling layers, each pooling layer preceded by at least one convolutional layer, and the other of the first and second parallel paths solely includes zero or more pooling layers.
3. The machine learning system of claim 1 wherein initial layers of the contracting path downsample the learning data, and layers subsequent to the initial layers contain a higher ratio of convolutional to downsampling operations than the initial layers.
4. The machine learning system of claim 1 wherein the expanding path includes fewer convolution operations than the contracting path.
5. The machine learning system of claim 1 wherein residual connections exist between each pair of layers that operate at the same spatial scale.
6. The machine learning system of claim 1 wherein the convolutions include dense M×M convolutions, where 1≤M≤11, cascaded N×1 and 1×N convolutions, where 3≤N≤11, and dilated convolutions.
7. The machine learning system of claim 1 wherein the CNN model includes skip connections between layers in the contracting path and the expanding path where the image sizes of those layers are compatible.
8. The machine learning system of claim 7 wherein the skip connections include concatenating feature maps of the CNN model.
9. The machine learning system of claim 7 wherein the skip connections are residual connections that add or subtract values of feature maps of the CNN model.
10. A method operating a machine learning system comprising at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data, and at least one processor communicably coupled to the at least one nontransitory processor-readable storage medium, the method comprising:
receiving, by the at least one processor, learning data comprising a plurality of batches of labeled image sets, each image set comprising image data representative of an anatomical structure, and each image set including at least one label which identifies the region of a particular part of the anatomical structure depicted in each image of the image set;
training, by the at least one processor, a fully convolutional neural network (CNN) model to segment at least one part of the anatomical structure utilizing the received learning data; and
storing, by the at least one processor, the trained CNN model in the at least one nontransitory processor-readable storage medium of the machine learning system, wherein training the CNN model comprises training a CNN model comprising a contracting path and an expanding path, the contracting path includes a number of convolutional layers and a number of pooling layers, each pooling layer preceded by at least one convolutional layer, and the expanding path includes a number of convolutional layers and a number of upsampling layers, each upsampling layer preceded by at least one convolutional layer and comprising a fixed upsampling operation without a learned kernel and a convolution operation with a learned kernel, the convolution operation immediately preceded by the fixed upsampling operation and succeeded by a concatenation of feature maps via a skip connection from the contracting path.
11. The method of claim 10 wherein training the CNN model comprises training a CNN model comprising a contracting path, and the contracting path includes first and second parallel paths, one of the first and second parallel paths includes a number of convolutional layers and a number of pooling layers, each pooling layer preceded by at least one convolutional layer, and the other of the first and second parallel paths solely includes zero or more pooling layers.
12. The method of claim 10 wherein training the CNN model comprises training a CNN model comprising a contracting path, and initial layers of the contracting path downsample the learning data, and layers subsequent to the initial layers contain a higher ratio of convolutional to downsampling operations than the initial layers.
13. The method of claim 10 wherein training the CNN model comprises training a CNN model comprising an expanding path which includes fewer convolution operations than the contracting path.
14. The method of claim 10 wherein training the CNN model comprises training a CNN model comprising residual connections between each pair of layers that operate at the same spatial scale.
15. The method of claim 10 wherein training the CNN model comprises training a CNN model wherein the convolutions include dense M×M convolutions, where 1≤M≤11, cascaded N×1 and 1×N convolutions, where 3≤N≤11, and dilated convolutions.
16. The method of claim 10 wherein training the CNN model comprises training a CNN model comprising skip connections between layers in the contracting path and the expanding path where the image sizes of those layers are compatible.
17. The method of claim 16 wherein training the CNN model comprises training a CNN model comprising skip connections, the skip connections include concatenating feature maps of the CNN model.
18. The method of claim 16 wherein training the CNN model comprises training a CNN model comprising skip connections, the skip connections are residual connections which add or subtract values of feature maps of the CNN model.