Method for detection and diagnosis of lung and pancreatic cancers from imaging scans
A method of detecting and diagnosing cancers characterized by the presence of at least one nodule/neoplasm from an imaging scan is presented. To detect nodules in an imaging scan, a 3D CNN using a single feed forward pass of a single network is used. After detection, risk stratification is performed using a supervised or an unsupervised deep learning method to assist in characterizing the detected nodule/neoplasm as benign or malignant. The supervised learning method relies on a 3D CNN used with transfer learning and a graph regularized sparse MTL to determine malignancy. The unsupervised learning method uses clustering to generate labels after which label proportions are used with a novel algorithm to classify malignancy. The method assists radiologists in improving detection rates of lung nodules to facilitate early detection and minimizing errors in diagnosis.
1. A method of detecting and diagnosing cancer characterized by presence of at least one nodule in a subject comprising:
providing an imaging scan of the subject;
automatically detecting the presence of the at least one nodule in the imaging scan using a volumetric method of the whole imaging scan, the volumetric method using a 3D convolutional neural network (CNN) having convolution blocks with dense connections wherein a cell-wise classification of input is done in a single feed forward path of the CNN in one shot to detect all of the at least one nodule in a given volume simultaneously and in the absence of postprocessing or other steps to remove false positives; and
automatically determining a classification of malignancy of the at least one detected nodule in the imaging scan using a supervised or an unsupervised deep learning method;
wherein the supervised learning method comprising
automatically determining imaging attributes of the at least one nodule using transfer learning of a pre-trained 3D convolutional neural network (C3D);
fine-tuning the C3D network with binary labels for malignancy and the imaging attributes; and
incorporating the binary labels for malignancy and the binary labels for the imaging attributes of the at least one nodule into a graph regularized sparse multi-task learning (MTL) framework to obtain the classification of malignancy of the at least one nodule;
wherein the unsupervised learning method comprising
performing clustering on the imaging attributes of the at least one nodule to estimate an initial set of labels;
computing label proportions corresponding to each cluster; and
training a classifier using the label proportions and clusters to obtain the classification of malignancy of the at least one nodule.
2. The method of claim 1 , wherein the input to the 3D CNN of the detection step is a 512×512×8 volume.
3. The method of claim 2 , wherein output of the 3D CNN of the detection step is a 16×16×8 probability map representing likelihood of nodule presence.
4. The method of claim 1 , wherein the 3D CNN of the detection step comprising:
a total of 36 3D convolution layers wherein 6 convolution layers form each of 5 dense blocks and remaining convolution layers form transition layers;
4 max-pooling layers;
4 transition layers; and
a sigmoid activation function.
5. The method of claim 1 , wherein the cancer is lung cancer or pancreatic cancer.
6. The method of claim 1 , wherein the imaging scan is created using computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), positron emission tomography/computed tomography (PET/CT), positron emission tomography/magnetic resonance imaging (PET/MRI) or contrast-enhanced ultrasound (CEUS).
7. The method of claim 1 , wherein the imaging attributes are selected from the group consisting of calcification, speculation, sphericity, lobulation, margin, texture and combinations thereof.
8. The method of claim 1 , wherein the classifier of the unsupervised learning method of the diagnosing step is proportion-support vector machine (aSVM).
9. A method of detecting and diagnosing lung cancer in a subject comprising:
providing a computed tomography (CT) scan of the subject; and
automatically detecting presence of at least one nodule in the CT scan using a volumetric method of the whole imaging scan, the volumetric method using a 3D convolutional neural network (CNN) having convolution blocks with dense connections wherein a cell-wise classification of input is done in a single feed forward path of the CNN in one shot to detect all of the at least one nodule in a given volume simultaneously and in the absence of postprocessing or other steps to remove false positives.
10. The method of claim 9 , wherein the 3D CNN comprising:
a total of 36 3D convolution layers wherein 6 convolution layers form each of 5 dense blocks and remaining convolution layers form transition layers;
4 max-pooling layers;
4 transition layers; and
a sigmoid activation function.
11. The method of claim 9 , further comprising:
automatically determining a classification of malignancy of the at least one detected nodule in the imaging scan using a supervised deep learning method wherein the supervised deep learning method comprising
automatically determining imaging attributes of the at least one nodule using transfer learning of a pre-trained 3D convolutional neural network (C3D);
fine-tuning the C3D network with binary labels for malignancy and the imaging attributes; and
incorporating the binary label for malignancy and the binary labels for the imaging attributes of the at least one nodule into a graph regularized sparse multi-task learning (MTL) framework to obtain the classification of malignancy of the at least one nodule.
12. The method of claim 11 , wherein the imaging attributes are selected from the group consisting of calcification, speculation, sphericity, lobulation, margin, texture and combinations thereof.
13. The method of claim 9 , further comprising:
automatically determining a classification of malignancy of the at least one detected nodule in the imaging scan using an unsupervised deep learning method wherein the unsupervised deep learning method comprising
performing clustering on the imaging attributes of the at least one nodule to estimate an initial set of labels;
computing label proportions corresponding to each cluster; and
training a classifier using the label proportions and clusters to obtain the classification of malignancy of the at least one nodule.
14. The method of claim 13 , wherein the imaging attributes are selected from the group consisting of calcification, speculation, sphericity, lobulation, margin, texture and combinations thereof.
15. The method of claim 13 , wherein the classifier of the unsupervised learning method is proportion-support vector machine (αSVM).
16. A method of detecting and diagnosing pancreatic cancer in a subject comprising:
providing an imaging scan of the subject;
automatically detecting presence of the at least one neoplasm in the imaging scan using a volumetric method of the whole imaging scan, the volumetric method using a 3D convolutional neural network (CNN) having convolution blocks with dense connections wherein a cell-wise classification of input is done in a single feed forward path of the CNN in one shot to detect all of the at least one neoplasm in a given volume simultaneously and in the absence of postprocessing or other steps to remove false positives; and
automatically determining a classification of malignancy of the at least one detected neoplasm in the imaging scan using a supervised or an unsupervised deep learning method;
wherein the supervised deep learning method comprising
automatically determining imaging attributes of the at least one neoplasm using transfer learning of a pre-trained 3D convolutional neural network (C3D);
fine-tuning the C3D network with binary labels for malignancy and the imaging attributes; and
incorporating the binary label for malignancy and the binary labels for the imaging attributes of the at least one nodule into a graph regularized sparse multi-task learning (MTL) framework to obtain the classification of malignancy of the at least one neoplasm;
wherein the unsupervised deep learning method comprising
performing clustering on the imaging attributes of the at least one neoplasm to estimate an initial set of labels;
computing label proportions corresponding to each cluster; and
training a classifier using the label proportions and clusters to obtain the classification of malignancy of the at least one neoplasm.
17. The method of claim 16 , wherein the 3D CNN of the diagnosing step comprising:
a total of 36 3D convolution layers wherein 6 convolution layers form each of 5 dense blocks and remaining convolution layers form transition layers;
4 max-pooling layers;
4 transition layers; and
a sigmoid activation function.
18. The method of claim 16 , wherein the imaging scan is a magnetic resonance image (MRI) scan.
19. The method of claim 16 , wherein the imaging attributes are selected from the group consisting of calcification, speculation, sphericity, lobulation, margin, texture and combinations thereof.
20. The method of claim 16 , wherein the classifier of the unsupervised learning method is proportion-support vector machine (αSVM).