Methods for polyp detection
Disclosed herein are methods for identifying polyps or lesions in a colon. In some variations, computer-implemented methods for polyp detection may be used in conjunction with an endoscope system to analyze the images captured by the endoscopic system, identify any polyps and/or lesions in a visual scene captured by the endoscopic system, and provide an indication to the practitioner that a polyp and/or lesion has been detected.
1. A method for polyp detection, the method comprising:
acquiring an image from an imaging module located at a distal portion of an endoscope;
identifying surface peaks in the image;
identifying clusters of surface peaks based on a predetermined threshold separation distance;
selecting a surface peak from each identified cluster;
defining a pixel region around each of the selected surface peaks;
comparing an image feature in each of said defined pixel regions with a corresponding image feature of a plurality of images containing polyps and a corresponding image feature of a plurality of images that do not contain polyps; and
if the image feature in a defined pixel region matches the corresponding image feature of a plurality of images containing polyps, generating a notification that a polyp has been detected.
2. The method of claim 1 , wherein comparing the image feature comprises:
computing a histogram of oriented gradients to extract surface peaks from the plurality of images containing polyps (HOG-PI);
computing a histogram of oriented gradients to extract surface peaks from the plurality of images that do not contain polyps (HOG-NPI);
computing a histogram of oriented gradients of the image enclosed by a defined rectangle (HOG-ROI);
comparing HOG-ROI with HOG-PI and HOG-NPI; and
if the similarity between HOG-ROI to HOG-PI exceeds a preselected similarity threshold, determining that a polyp is detected.
3. The method of claim 2 , wherein the preselected similarity threshold is at least 50% similarity.
4. The method of claim 1 , wherein the image feature comprises a curvature of a high-contrast edge.
5. The method of claim 1 , wherein the image feature comprises spatial frequency.
6. The method of claim 1 , wherein comparing image features in each of said defined pixel regions comprises:
applying a convolutional neural network (CNN) to said pixel regions; and
calculating a numerical output based on the CNN for each pixel region that indicates whether the pixel region contains a polyp.
7. The method of claim 1 , wherein comparing an image feature comprises applying a convolutional neural network (CNN) to each pixel region, wherein applying the CNN comprises:
generating a first filtered pixel region by filtering the pixel region with a first filter to identify one or more polyp-like features,
generating a second filtered pixel region by filtering the first filtered pixel region with a second filter to identify one or more non-polyp features, and
wherein generating the notification that a polyp has been detected comprises generating the notification if a second filtered pixel region of said defined pixel regions has been identified that has a higher incidence of polyp-like features than non-polyp features.
8. The method of claim 1 , wherein generating a notification comprises transmitting an image of the detected polyp to a display.
9. The method of claim 8 , wherein generating a notification further comprises providing an arrow configured to indicate the location of the polyp with respect to a distal end of the endoscope.
10. The method of claim 1 , wherein the imaging module comprises a first side-facing imaging device and a second side-facing imaging device, and wherein acquiring an image comprises acquiring a first image from the first side-facing imaging device and a second image from the second side-facing imaging device, and wherein
comparing image features in each of said defined pixel regions comprises applying a first CNN to pixel regions of the first image and applying a second CNN to pixel regions of the second image.
11. The method of claim 10 , wherein the endoscope comprises a front-facing imaging device, and wherein acquiring an image comprises acquiring a third image from the front-facing imaging device and wherein comparing image features comprises applying a third CNN to pixel regions of the third image.
12. A method for polyp detection comprising:
applying a convolutional neural network (CNN) to an image of the colon, wherein applying the CNN comprises:
selecting a first set of sub-regions of the image by applying a first convolution stage of the CNN to the image, the first convolution stage comprising a first polyp-positive filter that identifies sub-regions of the image containing a polyp-like feature;
selecting a second set of sub-regions from the first set of sub-regions by applying a second convolution stage of the CNN to the first set of sub-regions, the second convolution stage comprising a second polyp-positive filter that identifies the incidence of a polyp-like feature in a sub-region and a polyp-negative filter that identifies the incidence of a non-polyp feature in a sub-region;
selecting a third set of sub-regions by identifying sub-regions in the second set of sub-regions where a ratio of the incidence of the polyp-like feature to the incidence of the non-polyp feature exceeds a pre-determined threshold; and
generating an output that indicates the presence of a polyp within the image if the number of sub-regions in the third set of sub-regions meets or exceeds a pre-determined count threshold.
13. The method of claim 12 , wherein generating an output comprises generating an output if the ratio of the number of sub-regions in the third set to the number of sub-regions in the second set meets or exceeds a pre-determined ratio threshold.
14. The method of claim 12 , wherein the polyp-like feature comprises a high-contrast edge having a curve with a radius-of-curvature from about 2 mm to about 7 mm.
15. The method of claim 14 wherein the polyp-like feature further comprises a pixel having a local maximum intensity that is located within an inner curve of the high-contrast edge.
16. The method of claim 12 , wherein the polyp-like feature comprises surface peaks identified by calculating a histogram of oriented gradients of a plurality of polyp-positive colon images (HOG-PI).
17. The method of claim 12 , wherein the non-polyp feature comprises low-contrast edges with a spatial frequency that exceeds a pre-determined spatial frequency threshold.
18. The method of claim 12 , wherein the non-polyp feature comprises surface peaks identified by calculating a histogram of oriented gradients of a plurality of polyp-negative colon images (HOG-NPI).
19. The method of claim 12 , wherein the first polyp-positive filter is the same as the second polyp-positive filter.
20. The method of claim 12 , wherein the first polyp-positive filter is the different from the second polyp-positive filter.
21. The method of claim 12 , wherein the first convolution stage comprises a low-pass filter.
22. The method of claim 12 , wherein the second convolution stage comprises a low-pass filter.
23. The method of claim 12 , wherein the CNN is a first CNN, and the method further comprises applying a second CNN to the image of the colon, wherein the second CNN may comprise a first convolution stage having a third polyp-positive filter and a second convolution stage having a fourth polyp-positive filter and a second polyp-negative filter.
24. The method of claim 23 , wherein the polyp-like feature is a first polyp-like feature and the third polyp-positive filter identifies sub-regions of the image containing a second polyp-like feature different from the first polyp-like feature.
25. The method of claim 12 , wherein the image is a first image acquired by a first imaging device, and the CNN is a first CNN, and the method further comprises applying a second CNN to a second image of the colon acquired by a second imaging device.
26. The method of claim 25 , wherein the first imaging device is a first side-viewing device and the second imaging device is a second side-viewing device.
27. The method of claim 1 , further comprising:
applying a convolutional neural network (CNN) to an image of the colon, wherein applying the CNN comprises:
within the selected pixel regions around the surface peaks, applying a first convolution stage of the CNN to the image, the first convolution stage comprising a first polyp-positive filter that identifies sub-regions of the image containing a polyp-like feature;
selecting a second set of sub-regions from the first set of sub-regions by applying a second convolution stage of the CNN to the first set of sub-regions, the second convolution stage comprising a second polyp-positive filter that identifies the incidence of a polyp-like feature in a sub-region and a polyp-negative filter that identifies the incidence of a non-polyp feature in a sub-region;
selecting a third set of sub-regions by identifying sub-regions in the second set of sub-regions where a ratio of the incidence of the polyp-like feature to the incidence of the non-polyp feature exceeds a pre-determined threshold; and
generating an output that indicates the presence of a polyp within the image if the number of sub-regions in the third set of sub-regions meets or exceeds a pre-determined count threshold.
28. The method of claim 12 , further comprising the steps of:
identifying surface peaks in the image;
identifying clusters of surface peaks based on a predetermined threshold separation distance;
selecting a surface peak from each identified cluster;
defining a pixel region around each of the selected surface peaks;
comparing an image feature in each of said defined pixel regions with a corresponding image feature of a plurality of images containing polyps and a corresponding image feature of a plurality of images that do not contain polyps; and
if the image feature in a defined pixel region matches the corresponding image feature of a plurality of images containing polyps, generating a notification that a polyp has been detected.
29. A method for polyp detection, the method comprising:
acquiring an image from an imaging module located at a distal portion of an endoscope;
identifying surface peaks in the image;
identifying clusters of surface peaks based on a predetermined threshold separation distance;
selecting a surface peak from each identified cluster;
defining a pixel region around each of the selected surface peaks;
comparing an image feature in each of said defined pixel regions with a corresponding image feature of a plurality of images containing polyps and a corresponding image feature of a plurality of images that do not contain polyps;
the comparing of the pixel region with a corresponding image containing polyps further comprising applying a first convolution stage of the CNN to the pixel region, the first convolution stage comprising a polyp-positive filter that identifies sub-regions of the image containing a polyp-like feature;
the comparing of the pixel region with a corresponding image containing polyps further comprising applying a second convolution stage of the CNN to the pixel region, the second convolution stage comprising a polyp-negative filter that identifies the incidence of a non-polyp feature in a sub-region; and
if the image feature in a defined pixel region matches the corresponding image feature of a plurality of images containing polyps, generating a notification that a polyp has been detected.