Vascular segmentation using fully convolutional and recurrent neural networks
Systems and methods are provided for automatic segmentation of a vessel. A sequence of image slices containing a vessel is acquired. Features maps are generated for each of the image slices using a trained fully convolutional neural network. A trained bi-directional recurrent neural network generates a segmented image based on the feature maps.
1. A method for segmenting a plurality of images, the method comprising:
acquiring, by an image processor, the plurality of images, wherein the plurality of images comprise sequential image slices of a vessel along a centerline of the vessel, wherein each sequential image slice is centered at a respective sampling point and aligned orthogonally with a tangent direction to the centerline at the respective sampling point;
generating, by the image processor, a feature map for each of the plurality of images using a machine-trained fully convolutional network; and
generating, by the image processor, segmented images for each of the plurality of images based on the feature maps for the plurality of images using a bi-directional long short-term memory network comprising at least a forward direction long short-term memory layer and a backward direction long short-term memory layer that each input a respective feature map, wherein the outputs of the forward direction long short-term memory layer and the backward direction long short-term memory layer are averaged to generate a segmented image, wherein each of the segmented images includes labels for different types of tissues, obstructions, calcifications, and foreign objects.
2. The method of claim 1 , further comprising:
generating, by the image processor, a segmented model of the vessel using the segmented images.
3. The method of claim 2 , further comprising:
detecting, by the image processor, one or more vascular abnormalities in the vessel based on the segmented model; and
classifying, by the image processor, a type of the one or more vascular abnormalities in the vessel.
4. The method of claim 1 , further comprising:
acquiring, by the image processor, one or more geometric features of the centerline;
wherein the machine-trained fully convolutional network uses the one or more geometric features as inputs.
5. The method of claim 4 , wherein the one or more geometric features comprise a curvature of the vessel.
6. The method of claim 1 , wherein the respective sampling points are uniformly distributed along the centerline.
7. A method for segmenting a three-dimensional image, the method comprising:
acquiring, by an image processor, the three-dimensional image;
identifying a centerline of a vessel through the three-dimensional image;
acquiring, by the image processor, a plurality of image slices at sampling points along the centerline, wherein each image slice is centered at a respective sampling point and aligned orthogonally with a tangent direction to the centerline at the respective sampling point;
generating, by the image processor, with a machine-trained fully convolutional network, feature maps for each of the plurality of images slices;
generating, by the image processor, segmented images for each of the plurality of images slices based on the feature maps with a bi-directional long short-term memory network comprising at least a forward direction long short-term memory layer and a backward direction long short-term memory layer that each input a respective feature map, wherein the outputs of the forward direction long short-term memory layer and the backward direction long short-term memory layer are averaged to generate the segmented images, each of the segmented images including labels for different types of tissues, obstructions, calcifications, and foreign objects; and
generating, by the image processor, a model from the segmented images.
8. The method of claim 7 , wherein the machine-trained fully convolutional network includes a down sampling region and an up-sampling region.
9. An imaging system for segmenting images for medical applications, the imaging system comprising:
a medical image scanner configured to acquire a sequential series of two-dimensional cross-sectional images along a centerline of a vessel, wherein each two-dimensional cross-sectional image is centered at a respective sampling point and aligned orthogonally with a tangent direction to the centerline at the respective sampling point; and
an image processor configured as a machine-trained fully convolutional network, the machine-trained fully convolutional network configured to generate a feature map for each of the plurality of two dimensional cross-sectional images, the image processor further configured as a bi-directional long short-term memory network comprising at least a forward direction long short-term memory layer and a backward direction long short-term memory layer that each input a respective feature map, wherein the outputs of the forward direction long short-term memory layer and the backward direction long short-term memory layer are averaged to generate segmented images for each of the two-dimensional cross-sectional images based on the feature maps, each of the segmented images including labels for different types of tissues, obstructions, calcifications, and foreign objects.
10. The imaging system of claim 9 , further comprising:
a display configured to display the segmented images.
11. The imaging system of claim 9 , wherein the medical image scanner is configured to acquire the series of two-dimensional cross-sectional images at uniform points along the centerline of the vessel.
12. The imaging system of claim 11 , wherein the image processor is further configured to generate a model of the vessel from the segmented images.
13. The imaging system of claim 11 , wherein the machine-trained recurrent neural network is further configured to identify one or more types of plaque based on the features maps.
14. The imaging system of claim 9 , wherein the medical image scanner is configured to acquire a plurality of series of two-dimensional cross-sectional image over a period of time.