Automatic detection of erosions and ulcers in Crohn's capsule endoscopy
The present invention relates to a computer-implemented method capable of automatically detecting small bowel and colonic ulcers and erosions in Crohn's capsule endoscopy image data, by classifying pixels as lesion or non-lesion, using a convolutional image feature extraction step followed by a classification step and indexing such lesions in the set of one or more classes.
1 . A computer-implemented method capable of automatically identifying erosions and ulcers in colon capsule colonoscopy medical images by classifying the pixels as erosions or ulcers comprising:
selecting a number of subsets of all colon capsule colonoscopy images, each of said subsets comprising only images from the same patient;
selecting another subset as validation set, wherein the another subset does not overlap chosen images in previously selected subsets;
pre-training of each chosen subset with one of a plurality of combinations of convolutional neural network image feature extraction component, followed by a subsequent classification neural network component for pixel classification as erosions and ulcers, wherein said pre-training;
stops early when the scores do not improve over a given number of epochs, namely three;
evaluates the performance of each of the combinations;
is repeated on new, different subsets, with another networks combination and training hyperparameters, wherein such new combination considers a higher number of dense layers if a f1-metric is low and fewer dense layers if the f1-metric suggests overfitting;
selecting the architecture combination that performs best during pre-training;
fully training and validating during training the selected architecture combination using the entire set of colon capsule colonoscopy images to obtain an optimized architecture combination;
prediction of erosions and ulcers using said optimized architecture combination for classification;
receiving the classification output of the prediction by an output collect module with means of communication to a third-party capable of performing validation by interpreting the accuracy of the classification output and of correcting a wrong prediction, wherein the third-party comprises at least one of: another neural network, any other computational system adapted to perform the validation task or, optionally, a physician expert in gastroenterological imagery; and
storing the corrected prediction into the storage component.
2 . The method of claim 1 , wherein classification network architecture comprises at least two blocks, each having a Dense layer followed by a Dropout layer.
3 . The method of claim 1 , wherein a last block of the classification component includes a BatchNormalization layer followed by a Dense layer where a depth size is equal to a number of lesions of a type one desires to classify.
4 . The method of claim 1 , wherein a set of pre-trained neural networks is best performing among the following: VGG16, Inception V3, Xception, EfficientNetB5, EfficientNetB7, Resnet50 and Resnet125.
5 . The method of claim 1 , wherein a best performing combination is chosen based on overall accuracy and on f1-metrics.
6 . The method of claim 1 , wherein training of a best performing combination comprises two to four dense layers in sequence, starting with 4096 and decreasing in half up to 512.
7 . The method of claim 1 , wherein, between a final two layers of a best performing combination there is a dropout layer of 0.1 drop rate.
8 . The method of claim 1 , wherein training of the samples includes a ratio of training to-validation of 10%-90%.
9 . The method of claim 1 , wherein third-party validation is done by user-input.
10 . The method of claim 1 , wherein a training dataset includes images in a storage component that were predicted sequentially performing the method of claim 1 .
11 . A portable endoscopic device comprising instructions which, when executed by a processor, cause the computer to carry out the steps of the method of claim 1 .