Granted Patent B2

US 12,229,950 · App. 17/438,990 · Granted Feb 18, 2025

Inventors: Torre Michelle Bydlon (Melrose, MA); Paul Thienphrapa (Cambridge, MA); Alvin Chen (Cambridge, MA)

Assignee: KONINKLIJKE PHILIPS N.V.

G06T7/0012A61B34/20G06T7/149G06T15/08A61B2034/2065G06T2207/10121G06T2207/30061

Abstract

Claims (107)

1. A controller for assisting navigation in an interventional procedure, comprising:

a memory that stores instructions, and

a processor that executes the instructions,

wherein, when executed by the processor, the instructions cause the controller to implement a process that includes:

obtaining a three-dimensional model generated prior to an interventional procedure based on segmenting pathways with a plurality of branches in a subject of the interventional procedure;

receiving current positional information from an optical shape sensing of a tracked device, wherein the tracked device is tracked using this optical shape sensing;

determining, during the interventional procedure, whether a current positional information of the tracked device is outside of the pathways in the three-dimensional model;

when the current positional information of the tracked device is outside of the pathways in the three-dimensional model, deforming the three-dimensional model to the current

positional information of the tracked device; and

highlighting a planned path through the pathways in the three-dimensional model and alerting a user when the tracked device moves away from the planned path.

2. A controller for assisting navigation in an interventional procedure, comprising:

a memory that stores instructions, and

a processor that executes the instructions,

wherein, when executed by the processor, the instructions cause the controller to implement a process that includes:

obtaining a three-dimensional model generated prior to an interventional procedure based on segmenting pathways with a plurality of branches in a subject of the interventional procedure; receiving current positional information from an optical shape sensing of a tracked device, wherein the tracked device is tracked using this optical shape sensing;

determining, during the interventional procedure, whether a current positional information of the tracked device is outside of the pathways in the three-dimensional model; and

when the current positional information of the tracked device is outside of the pathways in the three-dimensional model, deforming the three-dimensional model to the current positional information of the tracked device;

wherein the process implemented when the processor executes the instructions further comprises:

registering the tracked device in the pathways to the three-dimensional model;

calculating an offset between an immediately previous positional information of the tracked device and the current positional information of the tracked device relative to the

three-dimensional model;

transforming the three-dimensional model to the current positional information of the tracked device based on the offset, and

iteratively locally deforming each new branch of the three-dimensional model containing the tracked device each time the tracked device moves to a new branch.

3. The controller of claim 2 , wherein the process implemented when the processor executes the instructions further comprises:

tracking the tracked device in two dimensions based on X-ray imaging; and

registering the three-dimensional model to a two-dimensional X-ray image space.

4. The controller of claim 3 , wherein the process implemented when the processor executes the instructions further comprises:

projecting the three-dimensional model to overlay the three-dimensional model onto the two-dimensional X-ray image space as the tracked device is navigated through the pathways

under guidance of the tracking based on X-ray imaging; and

identifying locations of the tracked device in a fluoroscopic image based on the X-ray imaging.

5. The controller of claim 2 , wherein the process implemented when the processor executes the instructions further comprises:

creating a virtual bronchoscopic view based on positions of a fixed point of the tracked device;

determining closest locations of a planned path to the positions of the fixed point of the tracked device; and

automatically updating the bronchoscopic view as the tracked device is moved through the three-dimensional model.

6. The controller of claim 2 , wherein the three-dimensional model has been segmented to form a mesh of the subject in a coordinate system, wherein the optical shape sensing of the tracked device is registered in said coordinate system, said process further comprises for each point (i) of the mesh:

a) calculating the distance between the mesh point (i) and optical shape sensing device points (j:j+N) sensed along the optical shape sensing of the tracked device;

b) determining the one of said optical shape sensing device points (j:j+N) closest to the mesh point (i) via a distance minimization calculation;

c) adding said calculated offset of this optical shape sensing device point to mesh point (i);

d) storing the new mesh point (i) coordinate;

e) repeating (a) to (d) for all mesh points to get finally said deformed three- dimensional model.

7. A controller for assisting navigation in an interventional procedure, comprising:

a memory that stores instructions, and

a processor that executes the instructions,

wherein, when executed by the processor, the instructions cause the controller to implement a process that includes;

obtaining a three-dimensional model generated prior to an interventional procedure based on segmenting pathways with a plurality of branches in a subject of the interventional procedure; receiving current positional information from an optical shape sensing of a tracked device, wherein the tracked device is tracked using this optical shape sensing;

determining, during the interventional procedure, whether a current positional information of the tracked device is outside of the pathways in the three-dimensional model; and

when the current positional information of the tracked device is outside of the pathways in the three-dimensional model, deforming the three-dimensional model to the current positional information of the tracked device, wherein the process implemented when the processor executes the instructions further comprises:

registering the tracked device in the pathways to the three-dimensional model; and

when the current positional information of the tracked device is outside of the pathways in the three-dimensional model, the deforming the three-dimensional model comprises deforming

only a local branch of the pathways that contains the tracked device or two local branches of the pathways nearest to the tracked device,

wherein the pathways are through a lung, a vascular system or a gastro-intestinal system and the tracked device is navigated to a location in the lung, vascular system or gastro-intestinal

system, prior to the interventional procedure.

8. A controller for assisting navigation in an interventional procedure, comprising:

a memory that stores instructions, and

a processor that executes the instructions

wherein, when executed by the processor, the instructions cause the controller to implement a process that includes:

obtaining a three-dimensional model generated prior to an interventional procedure based on segmenting pathways with a plurality of branches in a subject of the interventional procedure; receiving current positional information from an optical shape sensing of a tracked device, wherein the tracked device is tracked using this optical shape sensing;

determining, during the interventional procedure, whether a current positional information of the tracked device is outside of the pathways in the three-dimensional model; and

when the current positional information of the tracked device is outside of the pathways in the three-dimensional model, deforming the three-dimensional model to the current positional information of the tracked device, wherein the process implemented when the processor executes the instructions further comprises:

when the current positional information of the tracked device is outside of the pathways in the three-dimensional model, the deforming of the three-dimensional model comprises deforming

only a local branch of the pathways that contains the tracked device or two local branches of the pathways nearest to the tracked device, wherein the pathways are through a lung,

a vascular system or a gastro-intestinal system;

deforming the three-dimensional model to the current positional information of the tracked device to correct for lung, vascular system or gastro-intestinal system motion or a lung,

vascular system or a gastro-intestinal system deformation, and

continuously tracking the tracked device while the lung, the vascular system or the gastro- intestinal system is maneuvered.

9. A controller for assisting navigation in an interventional procedure, comprising:

a memory that stores instructions, and

a processor that executes the instructions,

wherein, when executed by the processor, the instructions cause the controller to implement a process that includes:

obtaining a three-dimensional model generated prior to an interventional procedure based on segmenting pathways with a plurality of branches in a subject of the interventional procedure; receiving current positional information from an optical shape sensing of a tracked device, wherein the tracked device is tracked using this optical shape sensing;

determining, during the interventional procedure, whether a current positional information of the tracked device is outside of the pathways in the three-dimensional model; and

when the current positional information of the tracked device is outside of the pathways in the three-dimensional model, deforming the three-dimensional model to the current positional information of the tracked device, wherein the process implemented when the processor executes the instructions further comprises:

labelling each branch of a path in the pathways as a progressive sequence through the three-dimensional model;

registering the three-dimensional model to the tracked device;

updating the three-dimensional model to live anatomy of the subject; and

as the tracked device is navigated, presenting labels on a display for branches of the pathways of the three-dimensional model that are proximate to the tracked device.

10. A controller for assisting navigation in an interventional procedure, comprising:

a memory that stores instructions, and

a processor that executes the instructions,

wherein, when executed by the processor, the instructions cause the controller to implement a process that includes:

obtaining a three-dimensional model generated prior to an interventional procedure based on segmenting pathways with a plurality of branches in a subject of the interventional procedure; receiving current positional information from an optical shape sensing of a tracked device, wherein the tracked device is tracked using this optical shape sensing;

determining, during the interventional procedure, whether a current positional information of the tracked device is outside of the pathways in the three-dimensional model; and

when the current positional information of the tracked device is outside of the pathways in the three-dimensional model, deforming the three-dimensional model to the current positional information of the tracked device, wherein the process implemented when the processor executes the instructions further comprises:

deforming each of a plurality of branches of the pathways in the three-dimensional model based on a trajectory of the tracked device as the tracked device approaches each of the plurality of branches.

11. A controller for assisting navigation in an interventional procedure, comprising:

a memory that stores instructions, and

a processor that executes the instructions,

wherein, when executed by the processor, the instructions cause the controller to implement a process that includes:

obtaining a three-dimensional model generated prior to an interventional procedure based on segmenting pathways with a plurality of branches in a subject of the interventional procedure; receiving current positional information from an optical shape sensing of a tracked device, wherein the tracked device is tracked using this optical shape sensing;

determining, during the interventional procedure, whether a current positional information of the tracked device is outside of the pathways in the three-dimensional model; and

when the current positional information of the tracked device is outside of the pathways in the three-dimensional model, deforming the three-dimensional model to the current positional information of the tracked device, wherein the process implemented when the processor executes the instructions further comprises:

continuously acquiring ultrasound images of the pathways as the tracked device is navigated on the pathways;

registering an ultrasound probe and the three-dimensional model when a branch of the pathways is initially obtained based on the ultrasound images;

virtually marking a current location of the ultrasound probe on the three-dimensional model, and continuously acquiring ultrasound images and registering each branch of the pathways to the three-dimensional model as the ultrasound probe is navigated.

12. A controller for assisting navigation in an interventional procedure, comprising:

a memory that stores instructions, and

a processor that executes the instructions,

wherein, when executed by the processor, the instructions cause the controller to implement a process that includes:

obtaining a three-dimensional model generated prior to an interventional procedure based on segmenting pathways with a plurality of branches in a subject of the interventional procedure; receiving current positional information from an optical shape sensing of a tracked device, wherein the tracked device is tracked using this optical shape sensing;

determining, during the interventional procedure, whether a current positional information of the tracked device is outside of the pathways in the three-dimensional model; and

when the current positional information of the tracked device is outside of the pathways in the three-dimensional model, deforming the three-dimensional model to the current positional information of the tracked device, wherein the process implemented when the processor executes the instructions further comprises:

continuously acquiring ultrasound images of the pathways as the tracked device is navigated on the pathways; and

virtually marking the three-dimensional model each time a branch of the pathways is visualized.

13. The controller of claim 12 , wherein the process implemented when the processor executes the instructions further comprises:

automatically determining the position of a tip of an ultrasound probe used to acquire the ultrasound images by calculating a diameter and wall thickness of a pathway from the ultrasound

images and comparing the diameter and wall thickness to the three-dimensional model; and

optimizing the position of the tip of the ultrasound probe based on previous locations of the ultrasound probe.

Assignments (1)

Continuity (2)

References Cited (12)