System for segmentation of anatomical structures in cardiac CTA using fully convolutional neural networks

1. A method, in a data processing system comprising a processor and a memory, the memory comprising instructions that are executed by the processor to specifically configure the processor to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, the method comprising:

(a) collecting (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures, wherein the multiple anatomical structures include ascending/descending/arch/root aorta, left/right/trunk pulmonary artery, vertebrae, left/right atrium, left/right ventricle, left ventricular myocardium, and superior/inferior vena cava;

(b) pre-processing the images such that they share the same field of view (FOV);

(c) using both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures;

(d) testing a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures; and

(e) performing post-processing on the segmented output of the given image.

2. The method of claim 1 , wherein pre-processing the images comprises down-sampling the segmented output of the given image.

3. The method of claim 2 , wherein performing post-processing comprises performing up-sampling on the segmented output of the given image.

4. The method of claim 2 , wherein performing post-processing comprises performing smoothing on the segmented output of the given image.

5. The method of claim 1 , wherein pre-processing the images comprises intensity equalizing the images to eliminate intensity variation between patients.

6. A method in a data processing system comprising a processor and a memory, the memory comprising instructions that are executed by the processor to specifically configure the processor to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, the method comprising:

(a) collecting (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures;

(b) pre-processing the images such that they share the same field of view (FOV);

(c) using both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures, wherein the supervised deep learning segmentation network is extended to a multi-label network, where the goal is to find find segmentation mask image F i , for anatomical structure i, by establishing that f j i ∈ F i , where f j i is the j th voxel of the i th segmented foreground, given the ground truth that g i j is in the i th ground truth structure, and wherein the overall dice score between the segmentation and the ground truth can be differentiated w.r.t the i th segmentation's j th voxel, yielding the gradient:

δ

⁢

⁢

D

δ

⁢

⁢

f

j

i

=

2

⁢

(

g

j

i

⁡

(

∑

n

N

⁢

f

n

i

2

+

∑

n

N

⁢

g

n

i

2

)

-

2

⁢

f

j

i

⁢

∑

n

N

⁢

f

n

i

⁢

g

n

i

(

∑

n

N

⁢

f

n

i

2

+

∑

n

N

⁢

g

n

i

2

)

2

)

where N is the total number of voxels in the image;

(d) testing a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures; and

(e) performing post-processing on the segmented output of the given image.

7. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, wherein the computer readable program causes the computing device to:

(a) collect (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures, wherein the multiple anatomical structures include ascending/descending/arch/root aorta, left/right/trunk pulmonary artery, vertebrae, left/right atrium, left/right ventricle, left ventricular myocardium, and superior/inferior vena cava;

(b) pre-process the images such that they share the same field of view (FOV);

(c) use both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures;

(d) test a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures; and

(e) perform post-processing on the segmented output of the given image.

8. The computer program product of claim 7 , wherein pre-processing the images comprises down-sampling the segmented output of the given image.

9. The computer program product of claim 8 , wherein performing post-processing comprises performing up-sampling on the segmented output of the given image.

10. The computer program product of claim 8 , wherein performing post-processing comprises performing smoothing on the segmented output of the given image.

11. The computer program product of claim 7 , wherein pre-processing the images comprises intensity equalizing the images to eliminate intensity variation between patients.

12. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, wherein the computer readable program causes the computing device to:

(a) collect (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures;

(b) pre-process the images such that they share the same field of view (FOV);

(c) use both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures, wherein the supervised deep learning segmentation network is extended to a multi-label network, where the goal is to find segmentation mask image F i , for anatomical structure i, by establishing that f j i ∈ F i , where f j i is the j th voxel of the i th segmented foreground, given the ground truth that g i j is in the i th ground truth structure, and wherein the overall dice score between the segmentation and the ground truth can be differentiated w.r.t the i th segmentation's j th voxel, yielding the gradient:

δ

⁢

⁢

D

δ

⁢

⁢

f

j

i

=

2

⁢

(

g

j

i

⁡

(

∑

n

N

⁢

f

n

i

2

+

∑

n

N

⁢

g

n

i

2

)

-

2

⁢

f

j

i

⁢

∑

n

N

⁢

f

n

i

⁢

g

n

i

(

∑

n

N

⁢

f

n

i

2

+

∑

n

N

⁢

g

n

i

2

)

2

)

where N is the total number of voxels in the image;

(d) test a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures; and

(e) perform post-processing on the segmented output of the given image.

13. An apparatus comprising:

at least one processor; and

a memory coupled to the at least one processor, wherein the memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, wherein the instructions cause the at least one processor to:

(a) collect (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures, wherein the multiple anatomical structures include ascending/descending/arch/root aorta, left/right/trunk pulmonary artery, vertebrae, left/right atrium, left/right ventricle, left ventricular myocardium, and superior/inferior vena cava;

(b) pre-process the images such that they share the same field of view (FOV);

(c) use both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures;

(d) test a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures; and

(e) perform post-processing on the segmented output of the given image.

14. The apparatus of claim 13 , wherein pre-processing the images comprises down-sampling the segmented output of the given image and wherein performing post-processing comprises performing up-sampling on the segmented output of the given image.

15. The apparatus of claim 14 , wherein performing post-processing comprises performing smoothing on the segmented output of the given image.

16. The apparatus of claim 13 , wherein pre-processing the images comprises intensity equalizing the images to eliminate intensity variation between patients.

17. An apparatus comprising:

at least one processor; and

a memory coupled to the at least one processor, wherein the memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement a system for segmentation of anatomical structures in cardiac computed tomography angiography (CTA) using fully convolutional neural networks, wherein the instructions cause the at least one processor to:

(a) collect (i) a set of chest CTA images scanned in the axial view and (ii) a manual segmentation of the images, for each one of multiple anatomical structures;

(b) pre-process the images such that they share the same field of view (FOV);

(c) use both the images and their manual segmentation to train a supervised deep learning segmentation network, wherein loss is determined from a multi-dice score that is a summation of dice scores for all the multiple anatomical structures, each dice score being computed as the similarity between the manual segmentation and the output of the network for one of the anatomical structures, wherein the supervised deep learning segmentation network is extended to a multi-label network, where the goal is to find find segmentation mask image F i , for anatomical structure i, by establishing that f j i ∈ F i , where f j i is the j th voxel of the i th segmented foreground, given the ground truth that g i j is in the i th ground truth structure, and wherein the overall dice score between the segmentation and the ground truth can be differentiated w.r.t the i th segmentation's j th voxel, yielding the gradient:

δ

⁢

⁢

D

δ

⁢

⁢

f

j

i

=

2

⁢

(

g

j

i

⁡

(

∑

n

N

⁢

f

n

i

2

+

∑

n

N

⁢

g

n

i

2

)

-

2

⁢

f

j

i

⁢

∑

n

N

⁢

f

n

i

⁢

g

n

i

(

∑

n

N

⁢

f

n

i

2

+

∑

n

N

⁢

g

n

i

2

)

2

)

where N is the total number of voxels in the image;

(d) test a given pre-processed image on the trained network, thereby obtaining segmented output of the given image, wherein the segmented output of the given image is segmented for the multiple anatomical structures; and

(e) perform post-processing on the segmented output of the given image.