Artificial intelligence based colonoscopy system and method
Described are colonoscopy systems and methods of using such systems. The colonoscopy systems may include an optical scanning system having at least one illuminator configured to produce spatially patterned light and solid light in at least one frame to illuminate tissue within the colon, and at least one camera configured to capture the at least one image of the illuminated tissue within the colon. Additionally, the optical scanning system may include at least one control system configured to construct at least one three dimensional point cloud representations of the tissue within the colon and detect at least one feature of interest using the at least one three dimensional point cloud and a pre-trained artificial intelligence engine.
1. A system, comprising:
at least one optical scanning system configured to scan and capture at least one image of a three-dimensional environment within a colon, the optical scanning system comprising:
a housing configured to be positioned within the colon and having a proximal end and a distal end, the housing comprising:
at least one illuminator positioned at the distal end of the housing and configured to produce spatially patterned light and solid light in at least one frame to illuminate tissue within the colon;
at least one camera positioned at the distal end of the housing and configured to capture the at least one image of the illuminated tissue within the colon;
at least one control system configured to:
construct at least one three dimensional point cloud representations of the tissue within the colon using the at least one image of the illuminated tissue within the colon; and,
detect presence of at least one polyp within the colon using the at least one three dimensional point cloud and a pre-trained artificial intelligence engine.
2. The system of claim 1 , wherein the pre-trained artificial intelligence engine includes a key-points detection engine.
3. The system of claim 2 , wherein the key-point detection engine is configured to:
select a plurality of key points on a depth map, the key point being repeatable with respect to noise and local variations; and,
classify key points as a polyp or surface of the colon using a pre-determined set of weights.
4. The system of claim 3 , wherein the set of weights are obtained by applying and training a two-class classifier, and training occurs on a pre-constructed sample set of data.
5. The system of claim 4 , wherein the set of weights are fixed during initial operation of the control system.
6. The system of claim 1 , wherein the pre-trained artificial intelligence engine includes a meta heuristic artificial intelligence engine, and the meta heuristic artificial intelligence engine provides a single module performing detection of polyps.
7. The system of claim 1 , wherein the pre-trained artificial intelligence engine includes an unsupervised artificial intelligence engine configured to:
perform a segmentation process to separate at least one three-dimensional point cloud into a plurality of segments based on geometrical distance; and,
classify each of the segments into a polyp category or non-polyp category based on geometrical distance.
8. The system of claim 7 , wherein the segmentation process occurs unsupervised and non-object aware.
9. The system of claim 7 , wherein the unsupervised artificial intelligence engine is further configured to:
combine, using at last one post processing algorithm, at least two segments classified in the polyp category to provide a single polyp when the at least two segments are directly adjacent to each other.
10. The system of claim 1 , wherein the control system is further configured to:
receive visible light data obtained by an RGB camera; and,
combine at least one three-dimensional point cloud with the visible light data to provide a plurality of enhanced images, the plurality of enhanced images providing additional reference points for location of the polyps.
11. The system of claim 1 , wherein the at least one control system is configured to further measure the at least one polyp within the colon in a lateral and depth dimension.
12. The system of claim 1 , wherein the control system is further configured to:
provide data to an endoscopic system to autonomously guide positioning of at least one instrument for removal of at least one polyp wherein the control system uses the three dimensional point cloud and measurement data as inputs to control and feedback systems utilizing artificial intelligence techniques to direct movements of the endoscopic system autonomously.
13. A system, comprising:
a housing configured to be positioned within a colon and having a proximal end and a distal end, the housing comprising:
at least one illuminator positioned at the distal end of the housing and configured to produce spatially patterned light and solid light in at least one frame to illuminate tissue within the colon;
at least one camera positioned at the distal end of the housing and configured to capture at least one image of the illuminated tissue within the colon;
at least one control system configured to:
construct at least one three dimensional point cloud representations of the tissue within the colon using at least one image of the illuminated tissue within the colon; and,
detect presence of at least one feature of interest within the colon using the at least one three dimensional point cloud and a pre-trained artificial intelligence engine.
14. The system of claim 13 , wherein the feature of interest is a polyp.
15. The system of claim 13 , wherein the feature of interest is an adenoma.
16. The system of claim 13 , wherein the pre-trained artificial intelligence engine is a key-points detection engine configured to:
select a plurality of key points on a depth map, the key point being repeatable with respect to noise and local variations; and,
classify key points as at least one feature of interest or surface of the colon using a pre-determined set of weights.
17. The system of claim 13 , wherein the control system is further configured to:
receive visible light data obtained by an RGB camera; and,
combine at least one three-dimensional point cloud with the visible light data to provide a plurality of enhanced images, the plurality of enhanced images providing additional reference points for location of at least one feature of interest.
18. A system, comprising:
a housing configured to be positioned within a colon and having a proximal end and a distal end, the housing comprising:
at least one illuminator positioned at the distal end of the housing and configured to produce spatially patterned light and solid light in at least one frame to illuminate tissue within the colon;
at least one camera positioned at the distal end of the housing and configured to capture at least one image of the illuminated tissue within the colon;
at least one control system configured to:
process data from the at least one image of the illuminated tissue within the colon and data from the illuminator using at least one triangulation algorithm to construct at least one three dimensional point cloud representations of the tissue within the colon; and,
detect presence of at least one feature of interest within the colon using the at least one three dimensional point cloud and a pre-trained artificial intelligence engine.
19. The system of claim 18 , wherein the control system is further configured to measure the at least one feature within the colon in a lateral and depth dimension.
20. The system of claim 18 , wherein the control system is further configured to provide data to an endoscopic system to guide positioning of at least one instrument for removal of the at least one feature.