Navigation of tubular networks
Methods and apparatuses provide improved navigation through tubular networks such as lung airways by providing improved estimation of location and orientation information of a medical instrument (e.g., an endoscope) within the tubular network. Various input data such as image data, EM data, and robot data are used by different algorithms to estimate the state of the medical instrument, and the state information is used to locate a specific site within a tubular network and/or to determine navigation information for what positions/orientations the medical instrument should travel through to arrive at the specific site. Probability distributions together with confidence values are generated corresponding to different algorithms are used to determine the medical instrument's estimated state.
1 . A method comprising:
accessing a model of a tubular network;
accessing robotic data including insertion data associated with robotic insertion of an elongated medical instrument into the tubular network;
accessing sensor data captured using a sensor associated with a distal portion of the elongated medical instrument;
determining a first estimated state for the elongated medical instrument based at least in part on the insertion data;
determining a second estimated state for the elongated medical instrument based at least in part on the sensor data by estimating a position and orientation of the elongated medical instrument relative to one or more branches of the tubular network, wherein the second estimated state comprises a probability distribution indicating a likelihood of the elongated medical instrument being inside each of the one or more branches;
determining a position of the elongated medical instrument relative to the model of the tubular network based at least in part on the second estimated state; and
determining a third estimated state for the elongated medical instrument based at least in part on the first estimated state and the position of the elongated medical instrument relative to the model of the tubular network.
2 . The method of claim 1 , wherein the elongated medical instrument is a flexible endoscope comprising an endoluminal structure and a plurality of cables for controlling an orientation of the elongated medical instrument.
3 . The method of claim 2 , wherein the robotic data indicates at least one of:
a pitch, a roll, or a yaw of the elongated medical instrument based at least in part on a retraction of at least one of the plurality of cables.
4 . The method of claim 1 , wherein the second estimated state includes at least one of: a position of the elongated medical instrument in three-dimensional (3D) space, an orientation of the elongated medical instrument in 3D space, an absolute depth of the elongated medical instrument within the tubular network, a relative depth of the elongated medical instrument within a branch of the tubular network, or a branch position with respect to the model of the tubular network.
5 . The method of claim 1 , further comprising:
accessing a prior estimated state for the elongated medical instrument based at least in part on data captured at a prior instant in time, wherein the first estimated state is determined based at least in part on the prior estimated state.
6 . The method of claim 1 , further comprising:
registering a sensor system to a 3D model based at least in part on coordinates associated with the sensor data and coordinates associated with the model of the tubular network, the second estimated state being determined based at least in part on the registration.
7 . The method of claim 1 , wherein each probability of the probability distribution is proportional to a size of a respective branch of the one or more branches.
8 . The method of claim 1 , further comprising:
accessing image data captured by an imaging device proximal to an instrument tip of the elongated medical instrument;
measuring movement of the elongated medical instrument within the tubular network based at least in part on the image data; and
determining a fourth estimated state for the elongated medical instrument based at least in part on the measured movement.
9 . The method of claim 8 , wherein the fourth estimated state comprises a continuous probability distribution associated with at least one of:
an absolute depth of the elongated medical instrument within the tubular network,
a depth of the elongated medical instrument relative to a branch of the tubular network, or
a roll of the elongated medical instrument relative to the tubular network.
10 . The method of claim 8 , further comprising:
mapping coordinates for an object in an image depicted by the image data to respective coordinates for the object in the model of the tubular network, the fourth estimated state being determined based at least in part on the mapping.
11 . The method of claim 1 , further comprising:
determining a topology of an area of the tubular network in which the elongated medical instrument is located; and
determining a fourth estimated state for the elongated medical instrument based at least in part on the determined topology.
12 . The method of claim 11 , wherein the determined topology includes a plurality of branches of the tubular network.
13 . The method of claim 1 , further comprising:
commanding movement of the elongated medical instrument deeper into the tubular network; and
updating the first estimated state based at least in part on the commanded movement.
14 . A system comprising:
a robotic system;
a robotically-controllable elongated medical instrument removably coupled to the robotic system and configured for insertion into a tubular network, wherein a distal end of the elongated medical instrument is articulable and the elongated medical instrument includes a sensor;
at least one non-transitory computer-readable medium having stored thereon instructions that, when executed, cause one or more processors to:
access a model of the tubular network;
access robotic data including insertion data associated with robotic insertion of the elongated medical instrument, by the robotic system, into the tubular network;
access sensor data derived from the sensor of the elongated medical instrument;
determine a first estimated state for the elongated medical instrument based at least in part on the insertion data;
determine a second estimated state for the elongated medical instrument based at least in part on the sensor data by estimating a position and orientation of the elongated medical instrument relative to one or more branches of the tubular network, wherein the second estimated state comprises a probability distribution indicating a likelihood of the elongated medical instrument being inside each of the one or more branches;
determine a position of the elongated medical instrument relative to the model of the tubular network based at least in part on the second estimated state; and
determine a third estimated state for the elongated medical instrument based at least in part on the first estimated state and the position of the elongated medical instrument relative to the model of the tubular network.
15 . The system of claim 14 , wherein each probability of the probability distribution is proportional to a size of a respective branch of the one or more branches.
16 . A system for bronchoscopy, comprising:
an instrument device manipulator configured to be coupled with an elongated medical instrument that is configured to be inserted into a tubular network, wherein a distal end of the elongated medical instrument is articulable and the elongated medical instrument includes a sensor;
at least one non-transitory computer-readable medium having stored thereon instructions that, when executed, cause one or more processors to:
access a model of the tubular network;
access robotic data including insertion data associated with robotic insertion of the elongated medical instrument into the tubular network;
access sensor data derived from the sensor of the elongated medical instrument;
determine a first estimated state for the elongated medical instrument based at least in part on the insertion data;
determine a second estimated state for the elongated medical instrument based at least in part on the sensor data by estimating a position and orientation of the elongated medical instrument relative to one or more branches of the tubular network, wherein the second estimated state comprises a probability distribution indicating a likelihood of the elongated medical instrument being inside each of the one or more branches;
determine a position of the elongated medical instrument relative to the model of the tubular network based at least in part on the second estimated state; and
determine a third estimated state for the elongated medical instrument based at least in part on the first estimated state and the position of the elongated medical instrument relative to the model of the tubular network.