Deep-learning-based cancer classification using a hierarchical classification framework
An automatic classification method for distinguishing between indolent and clinically significant carcinoma using multiparametric MRI (mp-MRI) imaging is provided. By utilizing a convolutional neural network (CNN), which automatically extracts deep features, the hierarchical classification framework avoids deficiencies in current schemes in the art such as the need to provide handcrafted features predefined by a domain expert and the precise delineation of lesion boundaries by a human or computerized algorithm. This hierarchical classification framework is trained using previously acquired mp-MRI data with known cancer classification characteristics and the framework is applied to mp-MRI images of new patients to provide identification and computerized cancer classification results of a suspicious lesion.
1. An apparatus for detecting and grading carcinoma, comprising:
(a) a computer processor; and
(b) a non-transitory computer-readable memory storing instructions executable by the computer processor;
(c) wherein said instructions, when executed by the computer processor, perform steps comprising:
(i) acquiring a plurality of multi-parametric MRI (mp-MRI) images of a subject with an mp-MRI imager;
(ii) pre-processing the acquired mp-MRI images to produce standardized small image patches;
(iii) extracting deep learning features from T2-weighted (T2w), apparent diffusion coefficient (ADC) and K trans data of the standardized small image patches with a convolution neural network (CNN) method;
(iv) obtaining a prediction score for each set of deep learning features by applying a first order classification of support vector machine (SVM) classifiers; and
(v) applying as second order classification of a Gaussian radial basis function kernel SVM classification of combined first order classification data to produce a final classification.
2. The apparatus of claim 1 , wherein said pre-processing of mp-MRI images instructions further comprise pre-processing the mp-MRI images with pixel intensity normalization, pixel spacing normalization and rescaling to produce said standardized small image patches.
3. The apparatus of claim 1 , wherein said convolution neural network (CNN) method is pre-trained.
4. The apparatus of claim 3 , wherein said pre-trained convolution neural network (CNN) method comprises OverFeat.
5. The apparatus of claim 1 , wherein said second order classification comprises a Gaussian radial basis function kernel SVM classification of combined first order classification data and one or more standard imaging features selected from the group of features consisting of: skewness of intensity histograms in T2w images; an average ADC value; lowest 10 th percentile; an average K trans , highest 10 th percentile K trans value; and region of interest size in T2w images.
6. A computer implemented method for detecting and grading carcinoma, the method comprising:
(a) acquiring a plurality of magnetic resonance images of a subject;
(b) pre-processing the acquired images;
(c) applying a convolution neural network (CNN) method to extract deep learning features from said pre-processed images;
(d) applying support vector machine (SVM) classifiers to the extracted deep learning features to produce SVM decision values; and
(e) obtaining a Gaussian radial basis function (RBF) kernel SVM classification of combined support vector machine (SVM) decision values and statistical features to produce a final decision; and
(f) wherein said method is performed by a computer processor executing instructions stored on a non-transitory computer-readable medium.
7. The method of claim 6 , wherein said magnetic resonance images comprise multi-parametric MRI (mp-MRI) images.
8. The method of claim 6 , wherein said pre-processing comprises:
(a) pixel intensity normalization;
(b) pixel spacing normalization; and
(c) rescaling.
9. The method of claim 6 , wherein said convolution neural network (CNN) method is pre-trained.
10. The method of claim 9 , wherein said pre-trained convolution neural network (CNN) method comprises OverFeat.
11. The method of claim 7 , wherein said applying a convolution neural network (CNN) method to extract deep learning features from said pre-processed images comprises extracting deep learning features from T2-weighted (T2w), apparent diffusion coefficient (ADC) and K trans data of standardized small image patches.
12. The method of claim 7 , wherein said applying support vector machine (SVM) classifiers to the extracted deep learning features to produce SVM decision values comprises obtaining a prediction score for each set of deep learning features by applying a first order classification of support vector machine (SVM) classifiers.
13. The method of claim 11 , wherein said support vector machine (SVM) decision values are combined with one or more statistical features (fs) from the group of statistical features consisting of:
(a) skewness of intensity histograms in T2w images;
(b) average ADC value;
(c) lowest 10 th percentile;
(d) ADC value;
(e) average K trans ;
(f) highest 10 th percentile K trans value; and
(g) ROI size in T2w images.
14. A non-transitory computer readable medium storing instructions executable by a computer processor, said instructions when executed by the computer processor performing steps comprising:
(a) acquiring a plurality of multi-parametric MRI (mp-MRI) images of a subject;
(b) preprocessing the images to produce standardized small image patches;
(c) extracting deep learning features from T2-weighted (T2w), apparent diffusion coefficient (ADC) and K trans data of the standardized small image patches with a convolution neural network (CNN);
(d) obtaining a prediction score for each set of deep learning features by applying a first order classification of support vector machine (SVM) classifiers; and
(e) applying as second order classification of a Gaussian radial basis function kernel SVM classification of combined first order classification data to produce a final classification.
15. The medium of claim 14 , wherein said pre-processing of mp-MRI images instructions further comprise pre-processing the mp-MRI images with pixel intensity normalization, pixel spacing normalization and rescaling to produce said standardized small image patches.
16. The medium of claim 14 , wherein said convolution neural network (CNN) method is pre-trained.
17. The medium of claim 16 , wherein said pre-trained convolution neural network (CNN) method comprises OverFeat.
18. The medium of claim 14 , wherein said second order classification comprises a Gaussian radial basis function kernel SVM classification of combined first order classification data and one or more standard imaging features selected from the group of features consisting of: skewness of intensity histograms in T2w images; an average ADC value; lowest 10 th percentile; an average K trans , highest 10 th percentile K trans value; and region of interest size in T2w images.
19. The method of claim 7 , wherein said pre-processing comprises:
(a) pixel intensity normalization;
(b) pixel spacing normalization; and
(c) rescaling.
20. The method of claim 7 , wherein said convolution neural network (CNN) method is pre-trained.
21. The method of claim 20 , wherein said pre-trained convolution neural network (CNN) method comprises OverFeat.