Automatic detection and differentiation of small bowel lesions in capsule endoscopy

1 . A computer-implemented method capable of automatically identifying and characterizing small bowel lesions in capsules colon colonoscopy medical images by classifying pixels as lesion or non-lesion, wherein such method comprises:

selecting a number of subsets of all colon capsule colonoscopy images, each of said subsets considering only images from a same patient;

selecting another subset as validation set, wherein the subset does not overlap chosen images on the previously selected subsets;

pre-training ( 8000 ) of each of chosen subsets with one of a plurality of combinations of convolutional neural network image feature extraction component, followed by a subsequent classification neural network component for pixel classification as small bowel lesion wherein said pre-training:

early stops when scores do not improved over a given number of epochs, namely three;

evaluates a performance of each of the combinations;

is repeated on new, different subsets, with another networks combination and training hyperparameters, wherein such new combination considers a higher number of dense layers if an f1 metric is low and fewer dense layers if f1 metric suggests overfitting;

selecting ( 400 ) an architecture combination that performs best during pre-training;

fully training and validating during training ( 9000 ) the selected architecture combination using an entire set of colon capsule colonoscopy images to obtain an optimized architecture combination;

prediction ( 6000 ) of small bowel lesions using said optimized architecture combination for classification;

receiving a classification output ( 270 ) of the prediction ( 6000 ) by an output collect module with means of communication to a third-party capable of performing validation by interpreting accuracy of the classification output and of correcting a wrong prediction, wherein the third-party comprises at least one of: another neural network, any other computational system adapted to perform the validation task or, optionally, a physician expert in gastroenterological imagery;

storing the corrected prediction into a storage component.

2 . The method of claim 1 , wherein the classification network architecture comprises at least two blocks, each having a Dense layer followed by a Dropout layer.

3 . The method of claim 1 or 2 , wherein the last block of the classification component includes a BatchNormalization layer followed by a Dense layer where the depth size is equal to the number of lesions type one desires to classify.

4 . The method of claim 1 , wherein the set of pre-trained neural networks is taken from a group including: VGG16, InceptionV3, Xception, EfficientNetB5, EfficientNetB7, Resnet50 and Resnet125.

5 . The method of claim 1 or 4 , wherein the best performing combination is chosen based on the overall accuracy and on the f1-metrics.

6 . The method of claim 1 or 4 , wherein the training of the best performing combination comprises two to four dense layers in sequence, starting with 40096 and decreasing in half up to 512.

7 . The method of claim 1 or 4 , wherein between the final two layers of the best performing combination there is a dropout layer of 0.1 drop rate.

8 . The method of claim 1 or 4 , wherein the training of the samples includes a ratio of training-to-validation of 10%-90%.

9 . The method of claim 1 or 4 , wherein the third-party validation is done by user-input.

10 . The method of claim 1 , wherein the training dataset includes images in the storage component that were predicted sequentially performing the steps of such method.

11 . A portable endoscopic device comprising instructions which, when executed by a processor, cause the computer to carry out the steps of the method of claim 1, 2 or 4 .