Systems for dynamic image-based localization and associated methods

1 . A system for performing a medical procedure within an anatomic region of a patient, the system comprising:

a medical instrument configured to be inserted along a navigational path to a target location within the anatomic region, wherein the medical instrument includes an image capture device;

a processor operably coupled to the image capture device; and

a memory operably coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising:

receiving a three-dimensional (3D) model of the anatomic region;

determining a plurality of image capture locations for the image capture device within the anatomic region, wherein at least one of the plurality of image capture locations is offset from the navigational path to the target location;

obtaining, from the image capture device, image data of an anatomic landmark within the anatomic region from the determined plurality of image capture locations, wherein at least some of the image data of the anatomic landmark is obtained from the at least one of the plurality of image capture locations that is offset from the navigational path to the target location;

identifying, based on the image data, an association between the anatomic landmark and a corresponding model landmark in the 3D model;

determining a localization state of the medical instrument based, at least in part, on the association between the anatomic landmark and the corresponding model landmark, the localization state including an estimated location of the medical instrument and an uncertainty parameter associated with the estimated location; and

dynamically updating the localization state as the medical instrument traverses the anatomic region.

2 . The system of claim 1 wherein obtaining the image data includes:

monitoring location data associated with the image data obtained at the plurality of image capture locations; and

based on the monitored location data, prompting collection of additional image data from at least one additional image capture location.

3 . The system of claim 1 wherein identifying the association comprises:

generating at least one two-dimensional (2D) virtual view of the model landmark; and

comparing the image data to the at least one 2D virtual view.

4 . The system of claim 1 wherein identifying the association comprises:

generating, based on the image data, a 3D representation of the anatomic landmark; and

comparing the 3D representation to a portion of the 3D model including the model landmark.

5 . The system of claim 4 wherein the 3D representation is generated from the image data using one or more of the following: a shape from shading algorithm, a structure from motion algorithm, a single-shot depth estimation algorithm, or an end-to-end depth estimation algorithm.

6 . The system of claim 1 wherein the 3D model represents a first portion of the anatomic region and the operations further comprise:

detecting, based on the image data, that the medical instrument is within a second portion of the anatomic region, the second portion being different from the first portion;

generating, based on the image data, a 3D representation of the second portion of the anatomic region; and

adding the 3D representation to the 3D model.

7 . The system of claim 1 , further comprising a positional sensor configured to generate positional data of the medical instrument, wherein the localization state is determined based, at least in part, on the positional data.

8 . The system of claim 1 wherein determining the localization state comprises:

generating a plurality of state estimates for the medical instrument, each state estimate including a respective estimated location and a respective uncertainty parameter, wherein the state estimates include at least one state estimate generated from the image data; and

combining the state estimates to determine the localization state.

9 . The system of claim 8 wherein the plurality of state estimates include one or more state estimates generated from one or more of the following: positional data from a positional sensor, a registration between the 3D model and the anatomic region, control signals for the medical instrument, user input from an operator of the medical instrument, external image data generated by an external imaging device, or sensor data from at least one additional sensor.

10 . The system of claim 1 wherein the localization state is determined using one or more of the following: a Kalman filter, a factor graph, or a particle filter.

11 . The system of claim 1 wherein the localization state is determined based, at least partly, on one or more previous localization states of the medical instrument.

12 . The system of claim 1 wherein the localization state is continuously updated at a rate of at least 30 times per second.

13 . The system of claim 1 wherein the localization state is determined without performing an initial registration between the 3D model and the anatomic region.

14 . The system of claim 1 wherein the operations further comprise outputting a graphical representation of the localization state to an operator, the graphical representation including a first visual indicator representing the estimated location and a second visual indicator representing the uncertainty parameter.

15 . A non-transitory, computer-readable medium storing instructions thereon that, when executed by one or more processors of a computing system, cause the computing system to perform operations comprising:

receiving a three-dimensional (3D) model of an anatomic region;

determining a plurality of image capture locations for an image capture device carried by a medical instrument within the anatomic region, wherein at least one of the plurality of image capture locations is offset from a navigational path to a target location;

obtaining, from the image capture device, image data of an anatomic landmark within the anatomic region from the determined plurality of image capture locations, wherein at least some of the image data of the anatomic landmark is obtained from the at least one of the plurality of image capture locations that is offset from the navigational path to the target location;

identifying, based on the image data, an association between the anatomic landmark and a corresponding model landmark in the 3D model;

determining a localization state of the medical instrument based, at least in part, on the association between the anatomic landmark and the corresponding model landmark, the localization state including an estimated location of the medical instrument and an uncertainty parameter associated with the estimated location; and

dynamically updating the localization state as the medical instrument traverses the anatomic region.

16 . The non-transitory, computer-readable medium of claim 15 wherein determining the localization state comprises:

generating a plurality of state estimates for the medical instrument, each state estimate including a respective estimated location and a respective uncertainty parameter, wherein the state estimates include at least one state estimate generated from the image data; and

combining the state estimates to determine the localization state.

17 . The non-transitory, computer-readable medium of claim 15 wherein the operations further comprise outputting a graphical representation of the localization state to an operator, the graphical representation including a first visual indicator representing the estimated location and a second visual indicator representing the uncertainty parameter.

18 . The non-transitory, computer-readable medium of claim 17 wherein the first and second visual indicators are displayed together with the 3D model of the anatomic region.

19 . The non-transitory, computer-readable medium of claim 15 wherein the operations further comprise:

detecting that the uncertainty parameter is greater than a specified threshold;

outputting instructions to an operator to obtain additional data of the anatomic region; and

updating the localization state using the additional data.

20 . The system of claim 1 , wherein determining the plurality of image capture locations includes determining a quantity of or an offset between the plurality of image capture locations based on a size of the anatomic region.