Deep learning cardiac segmentation and motion visualization
Devices, systems, and methods for automated segmentation and slicing of cardiac computed tomography (CT) images are described. An example method includes receiving a first plurality of input image frames associated with a cardiac CT operation, each of the first plurality of input image frames comprising a representation of two or more chambers of a heart, and performing, using a convolutional neural network, a segmentation operation and a slicing operation on each of the first plurality of input image frames to generate each of a plurality of output image frames comprising results of the segmentation operation and the slicing operation, wherein the segmentation operation comprises identifying volumes of each of the two or more chambers of the heart based on blood volumes, wherein the slicing operation comprises identifying one or more features of the heart in at least one predefined plane in a coordinate system associated with the cardiac CT operation.
1 . A method for automated segmentation and slicing of cardiac computed tomography (CT) images, comprising:
receiving a first plurality of input image frames associated with a cardiac CT operation, each of the first plurality of input image frames comprising a representation of two or more chambers of a heart; and
performing, using a convolutional neural network (CNN), a segmentation operation and a re-slicing operation on each of the first plurality of input image frames to generate each of a plurality of output image frames comprising results of the segmentation operation and the re-slicing operation,
wherein the CNN comprises a modified U-Net architecture that includes:
a down-sampling path comprising a plurality of down-sampling steps,
a fully-connected layer connected to an output of a last max-pooling operation in the down-sampling path, and
an up-sampling path comprising a plurality of up-sampling steps,
wherein the segmentation operation comprises identifying volumes of each of the two or more chambers of the heart based on blood volumes, and
wherein the re-slicing operation comprises identifying one or more features of the heart in at least one predefined plane in a coordinate system associated with the cardiac CT operation.
2 . The method of claim 1 , wherein the CNN is trained based on manual segmentation and manual re-slicing of a second plurality of input image frames, each of the second plurality of input image frames comprising the representation of the two or more chambers.
3 . The method of claim 1 , comprising:
performing a comparison between an output image frame and a manual segmentation of a corresponding input image frame.
4 . The method of claim 3 , wherein an efficacy of the comparison is quantified based on a Dice similarity and an ejection fraction (EF).
5 . The method of claim 1 , wherein an efficacy of a comparison between an output image frame and a manual re-slicing of a corresponding image frame is quantified based on errors in a plane location and a plane angle.
6 . The method of claim 1 , wherein each of the plurality of down-sampling steps and each of the plurality of up-sampling steps comprises two fully-connected convolutional layers and a filter.
7 . The method of claim 1 , wherein the modified U-Net architecture comprises three-dimensional convolution layers, max-pooling layers, and up-sampling layers that are configured to accept three-dimensional CT image volumes as inputs.
8 . The method of claim 1 , wherein:
each of the plurality of down-sampling steps comprising multiple convolutions, a rectified linear unit, and a max-pooling operation;
each of the plurality of up-sampling steps comprising an up-sampling operation and multiple convolutions; and
the modified U-Net architecture further comprises a softmax operation connected to an output of a last convolution in the up-sampling path,
wherein an output of the re-slicing operation is generated at an output of the fully-connected layer, and wherein an output of the segmentation operation is generated at an output of the softmax operation.
9 . The method of claim 8 , wherein each of the multiple convolutions comprises a three-dimensional convolution operation.
10 . The method of claim 8 , wherein the output of the re-slicing operation comprises a translation vector and at least one of (a) two directional vectors, (b) a normal vector, or (c) three three-dimensional points.
11 . The method of claim 1 , wherein the two or more chambers of a heart comprise a left ventricle and a left atrium, and wherein the at least one predefined plane comprises a 2CH plane, a 3CH plane, or a 4CH plane.
12 . A medical imaging system for automated segmentation and slicing of cardiac computed tomography (CT) images, comprising:
one or more processors configured to:
receive a first plurality of input image frames associated with a cardiac CT operation, each of the first plurality of input image frames comprising a representation of two or more chambers of a heart; and
perform, using a convolutional neural network (CNN), a segmentation operation and a re-slicing operation on each of the first plurality of input image frames to generate each of a plurality of output image frames comprising results of the segmentation operation and the re-slicing operation,
wherein the CNN comprises a modified U-Net architecture that includes:
a down-sampling path comprising a plurality of down-sampling steps,
a fully-connected layer connected to an output of a last max-pooling operation in the down-sampling path, and
an up-sampling path comprising a plurality of up-sampling steps,
wherein the segmentation operation comprises identifying volumes of each of the two or more chambers of the heart based on blood volumes, and
wherein the re-slicing operation comprises identifying one or more features of the heart in at least one predefined plane in a coordinate system associated with the cardiac CT operation.
13 . The medical imaging system of claim 12 , wherein the CNN is trained based on manual segmentation and manual re-slicing of a second plurality of input image frames, each of the second plurality of input image frames comprising the representation of the two or more chambers.
14 . The medical imaging system of claim 12 , wherein the one or more processors are configured to:
perform a comparison between an output image frame and a manual segmentation of a corresponding input image frame.
15 . The medical imaging system of claim 14 , wherein an efficacy of the comparison is quantified based on a Dice similarity and an ejection fraction (EF).
16 . The medical imaging system of claim 12 , wherein an efficacy of a comparison between an output image frame and a manual re-slicing of a corresponding image frame is quantified based on errors in a plane location and a plane angle.
17 . A non-transitory computer-readable storage medium having instructions stored thereupon for automated segmentation and slicing of cardiac computed tomography (CT) images, comprising:
instructions for receiving a first plurality of input image frames associated with a cardiac CT operation, each of the first plurality of input image frames comprising a representation of two or more chambers of a heart; and
instructions for performing, using a convolutional neural network (CNN), a segmentation operation and a re-slicing operation on each of the first plurality of input image frames to generate each of a plurality of output image frames comprising results of the segmentation operation and the re-slicing operation,
wherein the CNN comprises a modified U-Net architecture that includes:
a down-sampling path comprising a plurality of down-sampling steps,
a fully-connected layer connected to an output of a last max-pooling operation in the down-sampling path, and
an up-sampling path comprising a plurality of up-sampling steps,
wherein the segmentation operation comprises identifying volumes of each of the two or more chambers of the heart based on blood volumes, and
wherein the re-slicing operation comprises identifying one or more features of the heart in at least one predefined plane in a coordinate system associated with the cardiac CT operation.