Robotic systems for determining a roll of a medical device in luminal networks
Certain aspects relate to systems and techniques for navigation-assisted medical devices. Some aspects relate to correlating features of depth information generated based on captured images of an anatomical luminal network with virtual features of depth information generated based on virtual images of a virtual representation of the anatomical luminal network in order to automatically determine aspects of a roll of a medical device within the luminal network.
1. A method of facilitating navigation of an anatomical luminal network of a patient, the method, executed by a set of one or more computing devices, comprising:
receiving imaging data captured by an imaging device at a distal end of an instrument positioned within the anatomical luminal network, wherein the anatomical luminal network includes a right bronchus and a left bronchus;
identifying one or more features from the imaging data, the one or more features identified from the imaging data representing a first positioning of the right bronchus and the left bronchus in the imaging data;
accessing one or more features associated with a virtual image, the virtual image simulated from a viewpoint of a virtual imaging device positioned at a virtual location within a virtual luminal network representative of the anatomical luminal network, the virtual luminal network including a virtual right bronchus corresponding to the right bronchus and a virtual left bronchus corresponding to the left bronchus, wherein the one or more features associated with the virtual image represent a second positioning of the virtual right bronchus and the virtual left bronchus in the virtual image;
calculating a correspondence between the first positioning and the second positioning; and
determining a roll of the distal end of the instrument within the anatomical luminal network based on the calculated correspondence.
2. The method of claim 1 , wherein the anatomical luminal network includes a trachea leading into the right bronchus and the left bronchus, the method further comprising determining an insertion depth of the distal end of the instrument within the trachea based at least partly on the virtual location of the virtual image.
3. The method of claim 2 , further comprising:
generating an electromagnetic field around the anatomical luminal network; and
determining a registration between a coordinate frame of the virtual luminal network and a coordinate frame of the electromagnetic field based at least partly on the insertion depth and roll of the distal end of the instrument.
4. The method of claim 1 , further comprising:
generating a depth map based on the imaging data; and
identifying, based on the depth map, the first positioning of the right bronchus and the left bronchus in the imaging data;
wherein the one or more features associated with the virtual image comprise the second positioning derived from a virtual depth map associated with a virtual image.
5. The method of claim 4 , further comprising:
generating the depth map by calculating, for each pixel of a plurality of pixels of the imaging data, a depth value representing an estimated distance between the imaging device and a tissue surface within the anatomical luminal network corresponding to the pixel;
identifying a first pixel of the plurality of pixels corresponding to a first depth criterion in the depth map;
determining a first location of the right bronchus in the imaging data based on a position of the first pixel;
identifying a second pixel of the plurality of pixels corresponding to a second depth criterion in the depth map; and
determining a second location of the left bronchus in the imaging data based on a position of the second pixel.
6. The method of claim 5 , further comprising:
determining the first pixel has a greater depth value than the second pixel; and
identifying that the first location corresponds to the right bronchus based on determining the first pixel has the greater depth value.
7. The method of claim 5 , wherein the first depth criterion represents a most distant imaged tissue within the right bronchus, and wherein the second depth criterion represents a most distant imaged tissue within the left bronchus.
8. The method of claim 5 , wherein the first depth criterion corresponds to a first local maximum in a first region of depth values around the first pixel, and wherein the second depth criterion corresponds to a second local maximum in a second region of depth values around the second pixel.
9. The method of claim 4 , further comprising determining the roll of the instrument based on an angular distance between a first position of the right bronchus in the imaging data and a second position of the virtual right bronchus in the virtual image.
10. A system configured to facilitate navigation of an anatomical luminal network of a patient, the system comprising:
an imaging device at a distal end of an instrument;
at least one computer-readable memory having stored thereon executable instructions; and
one or more processors in communication with the at least one computer-readable memory and configured to execute the instructions to cause the system to at least:
receive imaging data captured by the imaging device with the distal end of the instrument positioned within the anatomical luminal network, wherein the anatomical luminal network includes a right bronchus and a left bronchus;
identify one or more features from the imaging data, the one or more features identified from the imaging data representing a first positioning of the right bronchus and the left bronchus in the imaging data;
access one or more features associated with a virtual image, the virtual image simulated from a viewpoint of a virtual imaging device positioned at a virtual location within a virtual luminal network representative of the anatomical luminal network, the virtual luminal network including a virtual right bronchus corresponding to the right bronchus and a virtual left bronchus corresponding to the left bronchus, wherein the one or more features associated with the virtual image represent a second positioning of the virtual right bronchus and the virtual left bronchus in the virtual image;
calculate a correspondence between first positioning and the second positioning; and
determine a roll of the distal end of the instrument within the anatomical luminal network based on the calculated correspondence.
11. The system of claim 10 , wherein the anatomical luminal network includes a trachea leading into the right bronchus and the left bronchus, and wherein the one or more processors are configured to execute the instructions to cause the system to at least determine an insertion depth of the distal end of the instrument within the trachea based at least partly on the virtual location of the virtual image.
12. The system of claim 11 , wherein the one or more processors are configured to execute the instructions to cause the system to at least:
generate an electromagnetic field around the anatomical luminal network; and
determine a registration between a coordinate frame of the virtual luminal network and a coordinate frame of the electromagnetic field based at least partly on the insertion depth and roll of the distal end of the instrument.
13. The system of claim 10 , wherein the one or more processors are configured to execute the instructions to cause the system to at least:
generate a depth map based on the imaging data; and
identify, based on the depth map, the first positioning of the right bronchus and the left bronchus in the imaging data;
wherein the one or more features associated with the virtual image comprise the second positioning derived from a virtual depth map associated with a virtual image.
14. The system of claim 13 , wherein the one or more processors are configured to execute the instructions to cause the system to at least:
generate the depth map by calculating, for each pixel of a plurality of pixels of the imaging data, a depth value representing an estimated distance between the imaging device and a tissue surface within the anatomical luminal network corresponding to the pixel;
identify a first pixel of the plurality of pixels corresponding to a first depth criterion in the depth map;
determine a first location of the right bronchus in the imaging data based on a position of the first pixel;
identify a second pixel of the plurality of pixels corresponding to a second depth criterion in the depth map; and
determine a second location of the left bronchus in the imaging data based on a position of the second pixel.
15. The system of claim 14 , wherein the one or more processors are configured to execute the instructions to cause the system to at least:
determine the first pixel has a greater depth value than the second pixel; and
identify that the first location corresponds to the right bronchus based on determining the first pixel has the greater depth value.
16. The system of claim 14 , wherein the first depth criterion represents a most distant imaged tissue within the right bronchus, and wherein the second depth criterion represents a most distant imaged tissue within the left bronchus.
17. The system of claim 14 , wherein the first depth criterion corresponds to a first local maximum in a first region of depth values around the first pixel, and wherein the second depth criterion corresponds to a second local maximum in a second region of depth values around the second pixel.
18. The system of claim 10 , wherein the one or more processors are configured to execute the instructions to cause the system to at least determine the roll of the instrument based on an angular distance between a first position of the right bronchus in the imaging data and a second position of the virtual right bronchus in the virtual image.
19. A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause at least one computing device to at least:
receive imaging data captured by the imaging device with the distal end of the instrument positioned within the anatomical luminal network, wherein the anatomical luminal network includes a right bronchus and a left bronchus;
identify one or more features of the imaging data, the one or more features identified from the imaging data representing a first positioning of the right bronchus and the left bronchus in the imaging data;
access one or more features associated with a virtual image, the virtual image simulated from a viewpoint of a virtual imaging device positioned at a virtual location within a virtual luminal network representative of the anatomical luminal network, the virtual luminal network including a virtual right bronchus corresponding to the right bronchus and a virtual left bronchus corresponding to the left bronchus, wherein the one or more features associated with the virtual image represent a second positioning of the virtual right bronchus and the virtual left bronchus in the virtual image;
calculate a correspondence between first positioning and the second positioning; and
determine a roll of the distal end of the instrument within the anatomical luminal network based on the calculated correspondence.
20. The non-transitory computer readable storage medium of claim 19 , wherein the anatomical luminal network includes a trachea leading into the right bronchus and the left bronchus, and wherein the instructions, when executed, cause the at least one computing device to at least determine an insertion depth of the distal end of the instrument within the trachea based at least partly on the virtual location of the virtual image.
21. The non-transitory computer readable storage medium of claim 20 , wherein the instructions, when executed, cause the at least one computing device to at least:
generate an electromagnetic field around the anatomical luminal network; and
determine a registration between a coordinate frame of the virtual luminal network and a coordinate frame of the electromagnetic field based at least partly on the insertion depth and roll of the distal end of the instrument.
22. The non-transitory computer readable storage medium of claim 19 , wherein the instructions, when executed, cause the at least one computing device to at least:
generate a depth map based on the imaging data; and
identify, based on the depth map, the first positioning of the right bronchus and the left bronchus in the imaging data;
wherein the one or more features associated with the virtual image comprise the second positioning derived from a virtual depth map associated with a virtual image.
23. The non-transitory computer readable storage medium of claim 22 , wherein the instructions, when executed, cause the at least one computing device to at least:
generate the depth map by calculating, for each pixel of a plurality of pixels of the imaging data, a depth value representing an estimated distance between the imaging device and a tissue surface within the anatomical luminal network corresponding to the pixel;
identify a first pixel of the plurality of pixels corresponding to a first depth criterion in the depth map;
determine a first location of the right bronchus in the imaging data based on a position of the first pixel;
identify a second pixel of the plurality of pixels corresponding to a second depth criterion in the depth map; and
determine a second location of the left bronchus in the imaging data based on a position of the second pixel.
24. The non-transitory computer readable storage medium of claim 23 , wherein the instructions, when executed, cause the at least one computing device to at least:
determine the first pixel has a greater depth value than the second pixel; and
identify that the first location corresponds to the right bronchus based on determining the first pixel has the greater depth value.
25. The non-transitory computer readable storage medium of claim 23 , wherein the first depth criterion represents a most distant imaged tissue within the right bronchus, and wherein the second depth criterion represents a most distant imaged tissue within the left bronchus.
26. The non-transitory computer readable storage medium of claim 23 , wherein the first depth criterion corresponds to a first local maximum in a first region of depth values around the first pixel, and wherein the second depth criterion corresponds to a second local maximum in a second region of depth values around the second pixel.
27. The non-transitory computer readable storage medium of claim 19 , wherein the instructions, when executed, cause the at least one computing device to at least determine the roll of the instrument based on an angular distance between a first position of the right bronchus in the imaging data and a second position of the virtual right bronchus in the virtual image.