Fissurenet: a deep learning approach for pulmonary fissure detection in CT images

1. A method comprising:

imaging, using a computed tomography system, at least one lung, to generate, at a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, at least one computed tomography image of the at least one lung, the at least one computed tomography image comprising a plurality of voxels;

determining, at the computer system, using a first convolutional neural network, at least one fissure region of interest in the at least one lung image, wherein determining at least one fissure region of interest comprises:

classifying, at the computer system, using the first convolutional neural network trained for a right lung, each of the plurality of voxels in a masked image of a right lung as being included in a right oblique fissure region of interest, being included in a right horizontal fissure region of interest, or being included in a non-fissure region of interest, the classification forming a coarse right lung probability map;

classifying, at the computer system, using the first convolutional neural network trained for a left lung, each of the plurality of voxels in a masked image of a left lung as being included in a left oblique fissure region of interest or being included in a non-fissure region of interest, the classification forming a coarse left lung probability map, wherein a selected voxel, of the plurality of voxels, for training data for the first convolutional neural network is assigned to a fissure region of interest if the selected voxel is within 5 mm of a fissure;

determining, at the computer system, using a second convolutional neural network, a more precise fissure location within the at least one fissure region of interest, wherein determining a more precise fissure location comprises:

classifying, at the computer system, using the second convolutional neural network trained for a right lung, each of the plurality of voxels classified in the right oblique fissure region of interest as being included in the right oblique fissure or not being included in a fissure, and classifying, at the computer system, each of the plurality of voxels classified in the right horizontal fissure region of interest as being included in the right horizontal fissure or not being included in a fissure, the classification forming a precise right lung probability map; and

classifying, at the computer system, using the second convolutional neural network trained for a left lung, each of the plurality of voxels classified in the left oblique fissure region of interest as being included in the left oblique fissure or not being included in a fissure, the classification forming a precise left lung probability map; and

generating an image of the lung including indication of the determined fissure location.

2. The method of claim 1 , further comprising:

masking, at the computer system, the image of the at least one lung based on whether the lung is a right lung or a left lung to generate a masked image of the lung.

3. The method of claim 1 , wherein determining a more precise fissure location is performed using the second convolutional neural network trained to detect a more precise location within the fissure region of interest for a right lung and using the second convolutional neural network trained to detect a more precise location within the fissure region of interest for a left lung and wherein a total loss function for the second convolutional neural network comprises a weighted average using a probability that the selected voxel, of the plurality of voxels, is in a fissure region of interest.

4. The method of claim 1 , further comprising determining fissure integrity based on at least one of the coarse right lung probability map, the coarse left lung probability map, the precise right lung probability map, and the precise left lung probability map.

5. A system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:

imaging, using a computed tomography system, at least one lung, to generate at least one computed tomography image of the at least one lung, the at least one computed tomography image comprising a plurality of voxels;

determining, using a first convolutional neural network, at least one fissure region of interest in the at least one lung image, wherein determining at least one fissure region of interest comprises:

classifying using the first convolutional neural network trained for a right lung, each of the plurality of voxels in a masked image of a right lung as being included in a right oblique fissure region of interest, being included in a right horizontal fissure region of interest, or being included in a non-fissure region of interest, the classification forming a coarse right lung probability map;

classifying using the first convolutional neural network trained for a left lung, each of the plurality of voxels in a masked image of a left lung as being included in a left oblique fissure region of interest, or being included in a non-fissure region of interest, the classification forming a coarse left lung probability map, wherein a selected voxel, of the plurality of voxels, for training data for the first convolutional neural network is assigned to a fissure region of interest if the selected voxel is within 5 mm of a fissure;

determining using a second convolutional neural network, a more precise fissure location within the at least one fissure region of interest, wherein determining a more precise fissure location comprises:

classifying using the second convolutional neural network trained for a right lung, each of the plurality of voxels classified in the right oblique fissure region of interest as being included in the right oblique fissure or not being included in a fissure, and classifying, at the computer system, each of the plurality of voxels classified in the right horizontal fissure region of interest as being included in the right horizontal fissure or not being included in a fissure, the classification forming a precise right lung probability map; and

classifying using the second convolutional neural network trained for a left lung, each of the plurality of voxels classified in the left oblique fissure region of interest as being included in the left oblique fissure or not being included in a fissure, the classification forming a precise left lung probability map; and

generating an image of the lung including indication of the determined fissure location.

6. The system of claim 5 , further comprising:

masking, the image of the at least one lung based on whether the lung is a right lung or a left lung to generate a masked image of the lung.

7. The system of claim 5 , wherein determining a more precise fissure location is performed using the second convolutional neural network trained to detect a more precise location within the fissure region of interest for a right lung and using the second convolutional neural network trained to detect a more precise location within the fissure region of interest for a left lung and wherein a total loss function for the second convolutional neural network comprises a weighted average using a probability that the selected voxel, of the plurality of voxels, is in a fissure region of interest.

8. The system of claim 5 , further comprising determining fissure integrity based on at least one of the coarse right lung probability map, the coarse left lung probability map, the precise right lung probability map, and the precise left lung probability map.

9. A computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer system comprising a processor, memory accessible by the processor, and the program instructions stored in the memory and executable by the processor, to cause the computer to perform a method comprising:

imaging, using a computed tomography system, at least one lung, to generate, at a computer system, at least one computed tomography image of the at least one lung, the at least one computed tomography image comprising a plurality of voxels;

determining, at the computer system, using a first convolutional neural network, at least one fissure region of interest in the at least one lung image, wherein determining at least one fissure region of interest comprises:

classifying, at the computer system, using the first convolutional neural network trained for a right lung, each of the plurality of voxels in a masked image of a right lung as being included in a right oblique fissure region of interest, being included in a right horizontal fissure region of interest, or being included in a non-fissure region of interest, the classification forming a coarse right lung probability map;

classifying, at the computer system, using the first convolutional neural network trained for a left lung, each of the plurality of voxels in a masked image of a left lung as being included in a left oblique fissure region of interest, or being included in a non-fissure region of interest, the classification forming a coarse left lung probability map, wherein a selected voxel, of the plurality of voxels, for training data for the first convolutional neural network is assigned to a fissure region of interest if the selected voxel is within 5 mm of a fissure;

determining, at the computer system, using a second convolutional neural network, a more precise fissure location within the at least one fissure region of interest, wherein determining a more precise fissure location comprises:

classifying, at the computer system, using the second convolutional neural network trained for a right lung, each of the plurality of voxels classified in the right oblique fissure region of interest as being included in the right oblique fissure or not being included in a fissure, and classifying, at the computer system, each of the plurality of voxels classified in the right horizontal fissure region of interest as being included in the right horizontal fissure or not being included in a fissure, the classification forming a precise right lung probability map; and

classifying, at the computer system, using the second convolutional neural network trained for a left lung, each of the plurality of voxels classified in the left oblique fissure region of interest as being included in the left oblique fissure or not being included in a fissure, the classification forming a precise left lung probability map; and

generating an image of the lung including indication of the determined fissure location.

10. The computer program product of claim 9 , further comprising:

masking, at the computer system, the image of the at least one lung based on whether the lung is a right lung or a left lung to generate a masked image of the lung.

11. The computer program product of claim 9 , wherein determining a more precise fissure location is performed using the second convolutional neural network trained to detect a more precise location within the fissure region of interest for a right lung and using the second convolutional neural network trained to detect a more precise location within the fissure region of interest for a left lung and wherein a total loss function for the second convolutional neural network comprises a weighted average using a probability that the selected voxel, of the plurality of voxels, is in a fissure region of interest.

12. The computer program product of claim 9 , further comprising determining fissure integrity based on at least one of the coarse right lung probability map, the coarse left lung probability map, the precise right lung probability map, and the precise left lung probability map.