Endolumenal object sizing
An object sizing system sizes an object positioned within a patient. The object sizing system identifies a presence of the object. The object sizing system navigates an elongate body of an instrument to a position proximal to the object within the patient. An imaging sensor coupled to the elongate body captures one or more sequential images of the object. The instrument may be further moved around within the patient to capture additional images at different positions/orientations relative to the object. The object sizing system also acquires robot data and/or EM data associated with the positions and orientations of the elongate body. The object sizing system analyzes the captured images based on the acquired robot data to estimate a size of the object.
1. A method for sizing an object positioned within a patient, comprising:
navigating an elongate body to a location within the patient and proximate to the object, the elongate body comprising an imaging sensor for capturing images of the object;
capturing sequential images of the object with the imaging sensor;
navigating the elongate body into at least one of a different orientation and a different position during the capture of the sequential images;
determining orientations and positions of the elongate body during the capture of the sequential images based on a set of robot data;
comparing the orientation and position of the elongate body determined based on the set of robot data for at least one captured sequential image to the orientation and position determined based on the set of robot data for at least another captured sequential image; and
analyzing, with a processor, the sequential images of the object and the comparison of the set of robot data to estimate a size of the object.
2. The method of claim 1 , wherein the set of robot data comprises data from which a depth of the imaging sensor into the patient can be determined.
3. The method of claim 1 , wherein the set of robot data comprises data from which position measurements about a position of the imaging sensor relative to the object can be determined.
4. The method of claim 1 , wherein the set of robot data comprises information from which an articulation of the elongate body relative to the object can be determined.
5. The method of claim 1 , wherein the set of robot data comprises at least one of data collected from a robotic manipulator and data collected from an electromagnetic sensor coupled to the elongate body.
6. The method of claim 1 , further comprising:
inserting a guide wire into the patient to measure distance from the imaging sensor to the object.
7. The method of claim 1 , wherein analyzing the sequential images of the object comprises:
combining the sequential images via image stitching techniques;
identifying a structure of the object; and
determining an estimated size of the identified object.
8. The method of claim 7 , wherein the structure of the object comprises at least one of an outer axial diameter and a radius of the object.
9. The method of claim 1 , further comprising:
capturing the object with a basket;
analyzing the sequential images depicting the object captured within the basket; and
determining an estimated size of the object based on the analyzed images.
10. The method of claim 9 , wherein analyzing the sequential images of the captured object comprises:
detecting a basket marker in the sequential images; and
determining a distance to the object based on the detected basket marker; and
wherein determining the estimated size of the object is based on the distance to the object.
11. The method of claim 1 , wherein:
the accessed set of robot data comprises data from which a position and size of a basket can be determined; and
analyzing the sequential images of the object and the set of robot data comprises:
identifying those images depicting at least a portion of the object and at least a portion of a basket; and
determining an estimated size of the object based on the position and the size of the basket relative to the object in the identified images.
12. The method of claim 11 , wherein analyzing the sequential images of the object and the set of robot data comprises segmenting the sequential images to remove the basket from the analyzed images.
13. The method of claim 12 , further wherein analyzing the sequential images of the object and the set of robot data comprises determining a structure of the object based on the segmented images.
14. The method of claim 1 , further comprising:
illuminating the object with a structured light source;
wherein the sequential images are structured light images of the object;
wherein the analysis of the images of the object comprises
analyzing an illumination pattern projected on the object by the structured light source; and
determining an estimated size of the object based on the analysis of the illumination pattern.
15. The method of claim 14 , wherein analyzing the sequential images comprises:
applying optical flow techniques on a plurality of pixels of the images of the object.
16. A system for sizing an object positioned within a patient comprising:
an elongate body, the elongate body comprising an imaging sensor for capturing images of the object, and
a computer comprising instructions that, when executed by a processor of the computer, cause the computer to perform steps of:
navigating the elongate body into the patient and to a location proximate to the object, the navigating enabled by a robotic manipulator;
capturing sequential images of the object with the imaging sensor;
navigating the elongate body into at least one of a different orientation and a different position during the capture of the sequential images;
determining orientations and positions of the elongate body during the capture of the sequential images based on a set of robot data;
comparing the orientation and position of the elongate body determined based on the set of robot data for at least one captured sequential image to the orientation and position determined based on the set of robot data for at least another captured sequential image; and
analyzing the sequential images and the comparison of the set of robot data to estimate a size of the object.
17. The system of claim 16 , wherein the set of robot data comprises data from which a depth of the imaging sensor into the lumen can be determined.
18. The system of claim 16 , wherein the set of robot data comprises data from which position measurements about a position of the imaging sensor relative to the object can be determined.
19. The system of claim 16 , wherein the set of robot data comprises data from which an articulation of the elongate body can be determined.
20. The system of claim 16 , wherein the set of robot data comprises at least one of data collected from a robotic manipulator and data collected from an electromagnetic sensor coupled to the elongate body.
21. The system of claim 16 , further comprising:
a guide wire;
wherein the computer further comprises instructions that, when executed by the processor of the computer, cause the computer to perform steps of:
inserting a guide wire into the patient to measure distance from the imaging sensor to the object.
22. The system of claim 16 , wherein analyzing the sequential images to estimate a size of the object further comprises:
combining the sequential images via image stitching techniques;
identifying a structure of the object; and
determining an estimated size of the object based on the identified structure.
23. The system of claim 22 , wherein the identified structure of the object comprises at least one of an outer axial diameter and a radius of the object.
24. The system of claim 16 , wherein the computer further comprises instructions that, when executed by the processor of the computer, cause the computer to perform steps of:
instructing capture the object with a basket;
analyzing the sequential images depicting the object captured within the basket; and
determining an estimated size of the object based on the analyzed images.
25. The system of claim 24 , wherein analyzing the sequential images of the captured object comprises:
detecting a basket marker in the sequential images; and
determining a distance to the object based on the detected basket marker; and
wherein determining the estimated size of the object is based on the distance to the object.
26. The system of claim 16 ,
wherein the accessed set of robot data further data from which a position and size of a basket can be determined; and
wherein analyzing the sequential images of the object and the set of robot data comprises:
identifying those images depicting at least a portion of the object and at least a portion of a basket; and
determining an estimated size of the object based on the position and the size of the basket relative to the object in the identified images.
27. The system of claim 26 ,
wherein analyzing the sequential images of the object and the comparison of the set of robot data comprises segmenting the sequential images to remove the basket from the analyzed images.
28. The system of claim 16 , further comprising:
a structured light source positionable proximate to the object;
wherein the computer further comprises instructions that, when executed by the processor of the computer, cause the computer to perform steps of:
illuminating the object with a structured light source;
wherein the sequential images are structured light images of the object; and
wherein analyzing the sequential images of the object comprises
analyzing an illumination pattern projected on the object by the structured light source; and
determining an estimated size of the object based on the analysis of the illumination pattern.
29. The system of claim 16 , wherein analyzing the sequential images comprises:
performing image analysis on the sequential images via applying optical flow techniques on a plurality of pixels of the object in the sequential images.