SYSTEMS AND METHODS OF DEEP LEARNING FOR COLORECTAL POLYP SCREENING

1 . A method of providing decision support in screening of colorectal polyps in a subject, the method comprising:

receiving, by a hardware processor, a dataset comprising a plurality of images of polyps, wherein image-based features of the images include a numerical feature;

converting, by the hardware processor, clinical information into multiple categories and from a categorical format into a numerical format of the image-based features;

combining, by the hardware processor, the converted clinical information with the image-based features, wherein the combined converted clinical information and the image-based features form a feature pool;

selecting, by the hardware processor, an optimal feature subset from the feature pool, the selecting comprising selecting image-based features and converted clinical information based at least in part on a training dataset;

creating, by the hardware processor, a classifier or a group of classifiers using the selected optimal feature subset and the training dataset; and

employing, by the hardware processor, the classifier or the group of classifiers to provide a prediction of neoplasticity of a polyp.

2 . The method according to claim 1 , wherein the numerical feature represents at least one of: a coefficient, a moment, a gray scale value, a derivative of an image intensity, or an image intensity.

3 . The method according to claim 1 , wherein selecting the optimal feature subset further comprises using at least one of a correlation filter, a recursive feature elimination, and a random feature selection to select the optimal feature subset.

4 . The method according to claim 1 , wherein selecting the optimal feature subset comprises executing a genetic algorithm-based feature selection for a plurality of iterations with a different randomly selected set of training data and testing data to obtain resulting plurality of feature subsets.

5 . The method according to claim 1 , wherein the image-based features comprise a fractal dimension of a shape of the polyp, an intensity and a size of dark internal pixel clusters in the polyp, an intensity and a size of bright internal pixel clusters in the polyp, or a chain code measurement of the polyp.

6 . The method according to claim 1 , wherein the classifier or one of the group of classifiers is a linear discriminant analysis.

7 . The method according to claim 1 , wherein the classifier or the group of classifiers determines whether the polyp is malignant or benign.

8 . The method according to claim 1 , wherein the classifier or the group of classifiers determines a likelihood of malignancy of the polyp.

9 . The method according to claim 1 , wherein the classifier or the group of classifiers is determined based at least in part by at least one calculation comprising: a simple mean value, a simple score, a weighted mean value, or a weighted score.

10 . A method for colorectal polyp screening, comprising:

capturing, by a computing device, a video of a colonoscopy procedure;

extracting, by the computing device, frames from the video or images associated with the colonoscopy procedure;

obtaining, by the computing device, a model for classifying objects that appear in the frames or images; and

determining, by the computing device, a classification of at least one polyp that appears in at least one of the frames or images based at least in part on applying the frames or images to an input layer of the model.

11 . The method of claim 10 , wherein the classification of the at least one polyp comprises an adenoma polyp, a hyperplastic polyp, or a serrated polyp.

12 . The method of claim 10 , wherein obtaining the model is based at least in part on determining weights and biases for the model.

13 . The method of claim 10 , further comprising obtaining the model based at least in part on:

generating pre-processed data based at least in part on at least one of normalizing, unstacking, or reducing noise associated with images from colonoscopy videos; and

generating, based at least in part on applying a nonlinear dimensionality reduction algorithm to the pre-processed data, embeddings of a plurality of image representations of polyps that appear in the images.

14 . The method of claim 13 , wherein the nonlinear dimensionality reduction algorithm comprises T-distributed Stochastic Neighbor Embedding (t-SNE).

15 . The method of claim 13 , wherein generating pre-processed data comprises identifying each respective image as associated with a White Light (WL) diagnostic method or a Narrow Band Imaging (NBI) diagnostic method.

16 . The method of claim 10 , wherein the model is a neural network that is a multilayer perceptron or deep neural network that utilizes forward and backward propagation for training.

17 . The method of claim 16 , wherein the multilayer perceptron comprises at least: the input layer, a first hidden layer, a second hidden layer, and an output layer.

18 . A system for colorectal polyp screening, comprising:

at least one computing device; and

program instructions executable in the at least one computing device that, when executed by the at least one computing device, cause the at least one computing device to:

capture a video of a colonoscopy procedure;

extract frames from the video or images associated with the colonoscopy procedure;

obtain a model for classifying objects that appear in the frames or images; and

determine a classification of at least one polyp that appears in at least one of the frames based at least in part on applying the frames or images to an input layer of the model.

19 . The system of claim 18 , wherein the classification comprises an adenoma polyp, a hyperplastic polyp, or a serrated polyp.

20 . The system of claim 18 , wherein the program instructions, when executed by the at least one computing device, cause the at least one computing device to:

render, on a display of the at least one computing device, a user interface comprising an indication associated with the classification of the at least one polyp.