Robotic system configured for navigation path tracing
Certain aspects relate to systems and techniques for navigation path tracing. In one aspect, a system displays a preoperative model of a luminal network is displayed. The system determines a position of an instrument within the luminal network relative to the preoperative model. Based on the position of the instrument relative to the preoperative model, the system determines whether to enter a path tracing mode. In path tracing mode the system displays visual indicia indicative of a path of the instrument with respect to the displayed preoperative model. The visual indicia may be used to visual the navigation path of the instrument and/or to extend the preoperative model.
1. A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause a processor of a device to at least:
display, on a user display, a preoperative model corresponding to a mapped portion of a luminal network of a patient;
determine a position of a distal end of an instrument that is configured to be positioned within the luminal network relative to the mapped portion of the preoperative model;
determine that the position of the distal end of the instrument is within a predetermined distance of a boundary corresponding to a transition from an end of the mapped portion of the preoperative model to a portion of the luminal network that is unmapped in the preoperative model prior to the distal end of the instrument reaching the boundary;
enter a path tracing mode in response to determining that the position of the distal end of the instrument is within the predetermined distance of the boundary prior to the distal end of the instrument reaching the boundary; and
when in the path tracing mode, display, on the user display, visual indicia indicative of a path traversed by the distal end of the instrument with respect to the displayed preoperative model as the instrument is navigated through the luminal network.
2. The non-transitory computer readable storage medium of claim 1 , wherein the instructions are configured to cause the processor of the device to deactivate the path tracing mode when the position of the distal end of the instrument returns to the mapped portion of the preoperative model from the unmapped portion of the preoperative model.
3. The non-transitory computer readable storage medium of claim 1 , wherein the instructions are configured to cause the processor of the device to enter the path tracing mode when the position of the distal end of the instrument is within 25%, 20%, 15%, 10%, or 5% an end of a last segment of the preoperative model.
4. The non-transitory computer readable storage medium of claim 1 , wherein the visual indicia are indicative of historical positions of the distal end of the instrument within the luminal network.
5. The non-transitory computer readable storage medium of claim 1 , wherein, in path tracing mode, the instructions are configured to cause the processor of the device to adjust a frequency of the visual indicia based on a distance traveled by the instrument between the visual indicia.
6. The non-transitory computer readable storage medium of claim 1 , wherein, in path tracing mode, the instructions are configured to cause the processor of the device to adjust a frequency of the visual indicia based on a time elapsed between the visual indicia.
7. The non-transitory computer readable storage medium of claim 1 , wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on a plurality of navigation modalities when the instrument is positioned within the mapped portion of the preoperative model, wherein the plurality of navigation modalities comprise a plurality of preoperative model data, vision data, position sensor data, and robotic command and kinematics data.
8. The non-transitory computer readable storage medium of claim 7 , wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on less than the plurality of navigation modalities when the instrument is positioned outside the mapped portion of the preoperative model.
9. The non-transitory computer readable storage medium of claim 1 , wherein the instructions are configured to cause the processor of the device to associate vision data with the visual indicia, wherein the vision data comprises an image received from an imaging device on the distal end of the instrument.
10. The non-transitory computer readable storage medium of claim 1 , wherein the instructions are configured to cause the processor of the device to associate robotic command and kinematics data with the visual indicia.
11. The non-transitory computer readable storage medium of claim 1 , wherein the instructions are configured to cause the processor of the device to:
receive user input data from a user input; and
associate the user input data with the visual indicia;
wherein the user input data comprises one or more of:
an indication of a lumen traveled;
an indication of a lumen not traveled;
an indication of an end of a lumen;
an indication of an opening of a lumen;
an indication that a current lumen extends beyond a current position of the instrument; and
a lumen diameter.
12. The non-transitory computer readable storage medium of claim 1 , wherein the instructions are configured to cause the processor of the device to:
detect when the position of the distal end of the instrument is moved into the mapped portion of the luminal network; and
stop displaying the visual indicia when the distal end of the instrument is positioned inside the mapped portion of the luminal network.
13. The non-transitory computer readable storage medium of claim 1 , wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on a plurality of navigation modalities when the instrument is positioned within the mapped portion of the preoperative model.
14. The non-transitory computer readable storage medium of claim 13 , wherein the plurality of navigation modalities comprise a plurality of preoperative model data, vision data, position sensor data, and robotic command and kinematics data.
15. The non-transitory computer readable storage medium of claim 13 , wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on less than the plurality of navigation modalities when the instrument is positioned outside the mapped portion of the preoperative model.
16. The non-transitory computer readable storage medium of claim 1 , wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on EM data received from an EM sensor when the instrument is positioned outside the mapped portion of the preoperative model.
17. The non-transitory computer readable storage medium of claim 1 , wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on a combination of vision data and robotic command and kinematics data when the instrument is positioned outside the mapped portion of the preoperative model.
18. The non-transitory computer readable storage medium of claim 1 , wherein the visual indicia comprise:
a first type of visual indicia indicative of a first historical position of the distal end of the instrument within the luminal network at which a first branch is present in the luminal network, the first type of visual indicia further indicating that the distal end of the instrument has traveled down all available child segments of the first branch, and
a second type of visual indicia indicative of a second historical position of the distal end of the instrument within the luminal network at which a second branch is present in the luminal network, the second type of visual indicia further indicating that the distal end of the instrument has not traveled down all available child segments of the second branch.
19. A robotic system for navigating a luminal network, the system comprising:
an instrument having an elongate body and a sensor disposed on a distal end of the elongate body;
at least one computer-readable memory having stored thereon executable instructions; and
one or more processors in communication with the at least one computer-readable memory and configured to execute the instructions to cause the one or more processors to at least:
access a preoperative model of a mapped portion of the luminal network and display the preoperative model on a user display;
determine a position of the distal end of the instrument within the luminal network relative to the preoperative model using the sensor;
determine that the position of the distal end of the instrument is within a predetermined distance of a boundary corresponding to a transition from an end of the mapped portion of the preoperative model to a portion of the luminal network that is unmapped in the preoperative model prior to the distal end of the instrument reaching the boundary;
enter a path tracing mode in response to determining that the position of the distal end of the instrument is within the predetermined distance of the boundary prior to the distal end of the instrument reaching the boundary; and
when in the path tracing mode, display visual indicia of a path traversed by the distal end of the instrument with respect to the displayed preoperative model on the user display as the instrument is navigated through the luminal network.
20. The system of claim 19 , wherein the one or more processors are configured to execute the instructions to cause the one or more processors to register a coordinate frame of the sensor and a coordinate frame of the preoperative model.
21. The system of claim 19 , the system further comprising a field generator configured to generate an EM field, wherein the sensor is an EM sensor, and wherein the one or more processors are configured to execute the instructions to cause the one or more processors to determine a position of the EM sensor within the EM field.
22. The system of claim 19 , wherein the sensor is a shape sensing fiber.
23. The system of claim 19 , wherein the one or more processors are configured to execute the instructions to cause the system to move the instrument within the luminal network.
24. The system of claim 19 , wherein the one or more processors are configured to execute the instructions to cause the one or more processors to at least:
detect when the position of the distal end of the instrument is moved into the mapped portion of the luminal network; and
stop displaying the visual indicia when the distal end of the instrument is positioned inside the mapped portion of the luminal network.
25. A method of determining a navigation path of an instrument within a luminal network, the method comprising:
displaying, on a user interface, a preoperative model corresponding to a mapped portion of a luminal network;
determining a position of a distal end of an instrument within the luminal network relative to the mapped portion of the luminal network;
moving the instrument within the luminal network;
determining that the position of the distal end of the instrument is within a predetermined distance of a boundary corresponding to a transition from an end of the mapped portion of the preoperative model to a portion of the luminal network that is unmapped in the preoperative model prior to the distal end of the instrument reaching the boundary;
entering a path tracing mode in response to determining that the position of the distal end of the instrument is within the predetermined distance of the boundary prior to the distal end of the instrument reaching the boundary; and
when in the path tracing mode, displaying visual indicia of a path traversed by the distal end of the instrument in the portion of the luminal network that is unmapped in the preoperative model relative to the preoperative model of the mapped portion of the luminal network as the instrument is navigated through the luminal network.
26. The method of claim 25 , further comprising:
detecting when the position of the distal end of the instrument is moved into the mapped portion of the luminal network; and
stopping displaying the visual indicia when the distal end of the instrument is positioned inside the mapped portion of the luminal network.
27. The method of claim 25 , further comprising adjusting a frequency of the visual indicia based on a distance traveled by the instrument between the visual indicia.
28. The method of claim 25 , further comprising adjusting a frequency of the visual indicia based on a time elapsed between the visual indicia.
29. The method of claim 25 , further comprising associating vision data with the visual indicia, wherein the vision data comprises an image received from an imaging device on the distal end of the instrument.