Image-based guidance for device path planning based on penalty function values and distances between ROI centerline and backprojected instrument centerline

1. An image-based guidance method, comprising:

segmenting, by a processor, a region of interest in three-dimensional (3D) image data to generate a segmented region of interest;

detecting, by the processor, a region of an endoscopic instrument in two-dimensional (2D) intraoperative image data of the region of interest and determining a centerline of the detected region of the endoscopic instrument;

performing, by the processor, backprojection of the endoscopic instrument and the centerline to generate a 3D backprojected volume including the backprojected endoscopic instrument and the backprojected centerline;

determining, by the processor, a device path of the endoscopic instrument based at least in part on distances from a centerline of the segmented region of interest to the backprojected centerline of the backprojected endoscopic instrument,

wherein determining, by the processor, the device path of the endoscopic instrument based at least in part on the distances from the centerline of the segmented region of interest to the backprojected centerline of the backprojected endoscopic instrument comprises:

determining one or more segment groupings, wherein each of the one or more segment groupings comprises a set of one or more segments that directionally connect a main segment of the segmented region of interest to a most distal segment of the segmented region of interest;

determining one or more penalty function values for the one or more segment groupings, wherein the one or more penalty function values are based on shortest distances between a centerline of a segment of the segmented region of interest and the backprojected centerline; and

selecting at least one of the one or more segment groupings with a smallest penalty function value, wherein the selected segment grouping forms at least a portion of the device path; and

generating, by the processor, a visualization of the device path from a starting point to a target point.

2. The method of claim 1 further comprises acquiring the 3D image data of the region of interest, wherein the region of interest comprises at least a portion of a bronchial tree.

3. The method of claim 1 further comprises acquiring the 2D intraoperative image data of the region of interest during a medical procedure.

4. The method of claim 3 wherein acquiring the 2D intraoperative image data comprises acquiring one or more 2D fluoroscopic images.

5. The method of claim 1 further comprises determining a centerline of the segmented region of interest.

6. The method of claim 1 further comprises identifying, via a user interface, the starting point and the target point of the endoscopic instrument in the segmented region of interest.

7. The method of claim 1 wherein determining the one or more penalty function values comprises determining a normalized sum of the shortest distances.

8. The method of claim 7 wherein determining the one or more penalty function values further comprises calculating a magnitude of a deformation field to be applied to the one or more segment groupings to contain the endoscopic instrument.

9. The method of claim 1 further comprises acquiring electromagnetic (EM) tracked 3D positions of a tip of the endoscopic instrument corresponding to a sequence of the intraoperative 2D image data at different time points.

10. The method of claim 9 wherein determining, by the processor, the device path of the endoscopic instrument based at least in part on the distances from the centerline of the segmented region of interest to the backprojected centerline of the backprojected endoscopic instrument comprises:

selecting an image from the sequence of the intraoperative 2D image data that corresponds to a backprojected endoscopic instrument tip with a smallest distance to a corresponding EM tracked 3D position;

translating the EM tracked 3D positions into a curve representation; and

updating the curve representation to minimize a difference between the curve representation and a backprojected plane of the endoscopic instrument derived from the selected image, wherein the updated curve representation forms at least a portion of the device path.

11. The method of claim 10 wherein translating the EM tracked 3D positions into the curve representation comprises translating the EM tracked 3D positions into a basis spline.

12. The method of claim 10 wherein updating the curve representation comprises modifying supporting points of the curve representation to satisfy an objective function.