SYSTEMS AND METHODS FOR IMAGE SEGMENTATION OF PET/CT USING CASCADED AND ENSEMBLED CONVOLUTIONAL NEURAL NETWORKS
A computer-implemented method is provided for segmentation of Positron emission tomography (PET)/computed tomography (CT). The method comprises: acquiring an original medical image including a PET image and CT image of a subject; transforming the original medical image into an input image with a predetermined resolution and a plurality of channels; processing the input image using an ensembled CNNs to output an intermediate segmentation mask; and taking the intermediate segmentation mask as input to a refiner model to output a final segmentation mask, where the final segmentation mask has a resolution same as the resolution of the original medical image.
1 . A computer-implemented method for segmentation of Positron emission tomography (PET)/computed tomography (CT), the method comprising:
(a) acquiring an original medical image including a PET image and CT image of a subject;
(b) transforming the original medical image into an input image with a predetermined resolution and a plurality of channels, wherein the plurality of channels correspond to a plurality of intensity ranges;
(c) processing the input image using an ensembled convolutional neural networks (CNNs) to output an intermediate segmentation mask; and
(d) taking the intermediate segmentation mask as input to a refiner model to output a final segmentation mask, wherein the final segmentation mask has a resolution same as the resolution of the original medical image.
2 . The computer-implemented method of claim 1 , wherein the predetermined resolution is lower than the resolution of the original medical image.
3 . The computer-implemented method of claim 1 , wherein the intermediate segmentation mask has a resolution same as the predetermine resolution.
4 . The computer-implemented method of claim 1 , wherein the plurality of channels are determined automatically by processing the original medical image.
5 . The computer-implemented method of claim 1 , wherein the plurality of channels are determined manually by a user.
6 . The computer-implemented method of claim 1 , wherein the ensembled CNNs comprise a plurality of 3D U-net like CNNs.
7 . The computer-implemented method of claim 6 , wherein a plurality of outputs of the 3D U-net like CNNs are linearly weighted to generate the intermediate segmentation mask.
8 . The computer-implemented method of claim 1 , wherein the input to the refiner model further comprises at least a portion of the original medial image.
9 . The computer-implemented method of claim 1 , wherein the ensembled CNNs and the refiner model are trained separately using a loss function.
10 . The computer-implemented method of claim 9 , wherein the loss function comprises a combination of dice loss and a cross-entropy loss to stabilize the training.
11 . The computer-implemented method of claim 10 , wherein the loss function further comprises a sensitivity loss.
12 . A non-transitory computer-readable storage medium including instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
(a) acquiring an original medical image including a PET image and CT image of a subject;
(b) transforming the original medical image into an input image with a predetermined resolution and a plurality of channels, wherein the plurality of channels correspond to a plurality of intensity ranges;
(c) processing the input image using an ensembled convolutional neural networks (CNNs) to output an intermediate segmentation mask; and
(d) taking the intermediate segmentation mask as input to a refiner model to output a final segmentation mask, wherein the final segmentation mask has a resolution same as the resolution of the original medical image.
13 . The non-transitory computer-readable storage medium of claim 12 , wherein the predetermined resolution is lower than the resolution of the original medical image.
14 . The non-transitory computer-readable storage medium of claim 12 , wherein the intermediate segmentation mask has a resolution same as the predetermine resolution.
15 . The non-transitory computer-readable storage medium of claim 12 , wherein the plurality of channels are determined automatically by processing the original medical image.
16 . The non-transitory computer-readable storage medium of claim 12 , wherein the plurality of channels are determined manually by a user.
17 . The non-transitory computer-readable storage medium of claim 12 , wherein the ensembled CNNs comprise a plurality of 3D U-net like CNNs.
18 . The non-transitory computer-readable storage medium of claim 17 , wherein a plurality of outputs of the 3D U-net like CNNs are linearly weighted to generate the intermediate segmentation mask.
19 . The non-transitory computer-readable storage medium of claim 12 , wherein the input to the refiner model further comprises at least a portion of the original medial image.
20 . The non-transitory computer-readable storage medium of claim 12 , wherein the ensembled CNNs and the refiner model are trained separately using a loss function.