System and method for catheter detection in fluoroscopic images and updating displayed position of catheter

1. A method for detecting a catheter in fluoroscopic data, the method comprising:

acquiring fluoroscopic data from a fluoroscopic sweep of a target area, the fluoroscopic data including 2D fluoroscopic frames of the target area captured from different perspectives;

performing an initial catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data;

performing a secondary catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data at least by considering the catheter tip candidates of the initial catheter detection;

eliminating false-positive catheter tip candidates of the secondary catheter detection by reconstructing a 3D position of the catheter tip and finding an intersecting point of rays corresponding to each 2D frame; and

reweighing the catheter tip candidates of the secondary catheter detection based on a distance of the catheter tip candidate from a projected 3D point.

2. The method according to claim 1 , wherein performing an initial catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data includes applying a shallow neural network operator.

3. The method according to claim 1 , wherein performing a secondary catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data includes applying a deep neural network operator.

4. The method according to claim 1 , further comprising iteratively repeating eliminating false-positive detections by reconstructing a 3D position of the catheter tip and finding an intersecting point of rays corresponding to each 2D frame.

5. The method according to claim 1 , further comprising displaying a user interface for manually selecting the catheter tip in a 2D fluoroscopic frame of the fluoroscopic data prior to performing an initial catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data.

6. The method according to claim 1 , wherein reweighing the catheter tip candidates of the secondary catheter detection based on a distance of the catheter tip candidate from a projected 3D point includes decreasing a weight of a pixel corresponding to a candidate when the distance of the catheter tip candidate is far from the projected 3D point.

7. A method for detecting a catheter in fluoroscopic data during a surgical navigation procedure, the method comprising:

tracking an electromagnetic position of a catheter using electromagnetic coordinates during a navigation procedure of the catheter to a target area;

displaying the tracked electromagnetic position of the catheter on a display of a 3D rendering;

acquiring fluoroscopic data from a fluoroscopic sweep of the target area, the fluoroscopic data including 2D fluoroscopic frames of the target area captured from different perspectives;

performing an initial catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data;

performing a secondary catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data at least by considering the catheter tip candidates of the initial catheter detection;

constructing fluoroscopic-based three dimensional volumetric data of the target area from the acquired fluoroscopic data, the fluoroscopic-based three dimensional volumetric data including a three-dimensional construction of a soft-tissue target in the target area;

acquiring, for each 2D frame of the fluoroscopic data, data of a position of the catheter relative to the three-dimensional construction of the soft-tissue target;

registering the acquired data of the position of the catheter relative to the three-dimensional construction of the soft-tissue with the electromagnetic position of the catheter; and

displaying the position of the catheter on the display of the 3D rendering based on the registering the acquired data of the position of the catheter relative to the three-dimensional construction of the soft-tissue with the electromagnetic position of the catheter.

8. The method according to claim 7 , further comprising eliminating false-positive catheter tip candidates of the secondary catheter detection by reconstructing a 3D position of the catheter tip and finding an intersecting point of rays corresponding to each 2D frame.

9. The method according to claim 7 , wherein performing an initial catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data includes applying a shallow neural network operator.

10. The method according to claim 7 , wherein performing a secondary catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data includes applying a deep neural network operator.

11. The method according to claim 7 , further comprising reweighing the catheter tip candidates of the secondary catheter detection based on a distance of the catheter tip candidate from a projected 3D point.

12. A system for performing an electromagnetic surgical navigation procedure, the system comprising:

an electromagnetic tracking system having electromagnetic tracking coordinates;

a catheter including a sensor configured to couple to the electromagnetic tracking system for detecting a position of the catheter in the electromagnetic tracking coordinates; and

a computing device operably coupled to the electromagnetic tracking system and the catheter, the computing device configured to:

display a navigation path to guide navigation of the catheter to a target area;

display the position of the catheter in the electromagnetic coordinates on a 3D rendering;

acquire fluoroscopic data from a fluoroscopic sweep of the target area, the fluoroscopic data including 2D fluoroscopic frames of the target area captured from different perspectives;

perform an initial catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data;

perform a secondary catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data at least by considering the catheter tip candidates of the initial catheter detection;

eliminate false-positive catheter tip candidates of the secondary catheter detection by reconstructing a 3D position of the catheter tip and finding an intersecting point of rays corresponding to each 2D frame; and

reweigh the catheter tip candidates of the secondary catheter detection based on a distance of the catheter tip candidate from a projected 3D point.

13. The system according to claim 12 , wherein the computing device is configured to perform an initial catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data by applying a shallow neural network operator.

14. The system according to claim 12 , wherein the computing device is configured to perform a secondary catheter detection for catheter tip candidates in each 2D frame of the fluoroscopic data by applying a deep neural network operator.

15. The system according to claim 12 , wherein the computing device is further configured to:

construct fluoroscopic-based three dimensional volumetric data of the target area from the acquired fluoroscopic data, the fluoroscopic-based three dimensional volumetric data including a three-dimensional construction of a soft-tissue target in the target area.

16. The system according to claim 15 , wherein the computing device is further configured to:

acquire, for each 2D frame of the fluoroscopic data, data of a position of the catheter relative to the three-dimensional construction of the soft-tissue;

register the acquired data of the position of the catheter relative to the three-dimensional construction of the soft-tissue with the position of the catheter in the electromagnetic coordinates; and

display the position of the catheter on the display of the 3D rendering based on the registration of the acquired data of the position of the catheter relative to the three-dimensional construction of the soft-tissue with the position of the catheter in the electromagnetic coordinates.