IP Library › Granted Patent US 12,478,436
Granted Patent B2
US 12,478,436 · App. 17/456,458 · Granted Nov 25, 2025

Smart image navigation for intracardiac echocardiography

Inventors: Rui Liao (Princeton Junction, NJ); Young-Ho Kim (West Windsor, NJ); Jarrod Collins (Houston, TX); Abdoul Aziz Amadou (London, GB); Sebastien Piat (Lawrence Township, NJ); Ankur Kapoor (Plainsboro, NJ); Tommaso Mansi (Plainsboro, NJ); Noha El-Zehiry (Plainsboro, NJ); Sasa Grbic (Plainsboro, NJ); Dorin Comaniciu (Princeton, NJ); Xianjun S. Zheng (Plainsboro, NJ); Bo Liu (New Brunswick, NJ); Zhoubing Xu (Plainsboro, NJ); Jin-Hyeong Park (Princeton, NJ)
Assignee: Siemens Medical Solutions USA, Inc.
A61B34/20A61B34/25A61B34/32A61B2034/2063
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,478,436
App. No.
17/456,458
Granted
Nov 25, 2025
Kind
B2
Abstract

Systems and methods for automatically navigating a catheter in a patient are provided. An image of a current view of a catheter in a patient is received. A set of actions of a robotic navigation system for navigating the catheter from the current view towards a target view is determined using a machine learning based network. The catheter is automatically navigated in the patient from the current view towards the target view using the robotic navigation system based on the set of actions.

Claims (40)

1 . A computer-implemented method comprising:

receiving an image of a current view of a catheter in a patient and an image of a target view of the catheter in the patient;

determining a set of actions of a robotic navigation system for navigating the catheter from the current view towards the target view by:

performing a first registration between the image of the target view of the catheter and the image of the current view of the catheter,

performing a second registration between the image of the target view of the catheter and an image of a view of the catheter navigated to a predefined position in the patient, and

determining the set of actions for navigating the catheter from the current view towards the target view based on the first registration and the second registration by: 1) predicting, by a machine learning based network, a reward for each of a set of possible actions and 2) determining the set of actions from the set of possible actions based on the rewards; and

automatically navigating the catheter in the patient from the current view towards the target view using the robotic navigation system based on the set of actions.

2 . The computer-implemented method of claim 1 , wherein

the image of the target view of the catheter comprises a preoperative medical image of the target view of the catheter in the patient.

3 . The computer-implemented method of claim 2 , wherein the preoperative medical image is a medical image acquired for planning the same medical procedure for which the catheter is automatically navigated.

4 . The computer-implemented method of claim 1 , further comprising:

comparing the image of the current view to an image of the target view to determine a similarity measure; and

repeating the receiving, the determining, and the automatically navigating until a similarity threshold is satisfied.

5 . The computer-implemented method of claim 1 , further comprising:

performing at least one of classification of current view of the catheter or identification of one or more anatomical objects of interest in the current view of the catheter; and

identifying the at least one of the classification of the current view of the catheter or the identification of the one or more anatomical objects of interest in the current view of the catheter.

6 . An apparatus comprising:

a memory storing computer program instructions; and

at least one processor configured to execute the computer program instructions, the computer program instructions configured to cause the at least one processor to perform operations comprising:

receiving an image of a current view of a catheter in a patient and an image of a target view of the catheter in the patient;

determining a set of actions of a robotic navigation system for navigating the catheter from the current view towards the target view by:

performing a first registration between the image of the target view of the catheter and the image of the current view of the catheter,

performing a second registration between the image of the target view of the catheter and an image of a view of the catheter navigated to a predefined position in the patient, and

determining the set of actions for navigating the catheter from the current view towards the target view based on the first registration and the second registration by: 1) predicting, by a machine learning based network, a reward for each of a set of possible actions and 2) determining the set of actions from the set of possible actions based on the rewards; and

automatically navigating the catheter in the patient from the current view towards the target view using the robotic navigation system based on the set of actions.

7 . The apparatus of claim 6 , wherein

the image of the target view of the catheter comprises a preoperative medical image of the target view of the catheter in the patient.

8 . The apparatus of claim 7 , wherein the preoperative medical image is a medical image acquired for planning the same medical procedure for which the catheter is automatically navigated.

9 . A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:

receiving an image of a current view of a catheter in a patient and an image of a target view of the catheter in the patient;

determining a set of actions of a robotic navigation system for navigating the catheter from the current view towards the target view by:

performing a first registration between the image of the target view of the catheter and the image of the current view of the catheter,

performing a second registration between the image of the target view of the catheter and an image of a view of the catheter navigated to a predefined position in the patient, and

determining the set of actions for navigating the catheter from the current view towards the target view based on the first registration and the second registration by: 1) predicting, by a machine learning based network, a reward for each of a set of possible actions and 2) determining the set of actions from the set of possible actions based on the rewards; and

automatically navigating the catheter in the patient from the current view towards the target view using the robotic navigation system based on the set of actions.

10 . The non-transitory computer readable medium of claim 9 , wherein

the image of the target view of the catheter comprises a preoperative medical image of the target view of the catheter in the patient.

11 . The non-transitory computer readable medium of claim 9 , the operations further comprising:

comparing the image of the current view to an image of the target view to determine a similarity measure; and

repeating the receiving, the determining, and the automatically navigating until a similarity threshold is satisfied.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2022
From: SIEMENS HEALTHCARE GMBH
To: SIEMENS MEDICAL SOLUTIONS USA, INC.
Reel/Frame 061787/0336 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2022
From: SIEMENS MEDICAL SOLUTIONS USA, INC.
To: SIEMENS HEALTHCARE GMBH
Reel/Frame 058934/0423 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2022
From: AMADOU, ABDOUL AZIZ
To: SIEMENS HEALTHCARE LIMITED
Reel/Frame 058839/0135 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2022
From: SIEMENS HEALTHCARE LIMITED
To: SIEMENS MEDICAL SOLUTIONS USA, INC.
Reel/Frame 058840/0367 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 31, 2022
From: LIAO, RUI; KIM, YOUNG-HO; COLLINS, JARROD; PIAT, SEBASTIEN; KAPOOR, ANKUR; MANSI, TOMMASO; EL-ZEHIRY, NOHA; COMANICIU, DORIN; GRBIC, SASA; PARK, JIN-HYEONG; LIU, BO; XU, ZHOUBING; ZHENG, XIANJUN S.
To: SIEMENS MEDICAL SOLUTIONS USA, INC.
Reel/Frame 058832/0860 →
Continuity (1)
Related Publication 20230157761A1 · May 25, 2023
References Cited (21)
US 20140147027A1 · Jain et al. · 2014 [cited by applicant]
US 20180032841A1 · Kluckner · 2018 [cited by examiner]
US 20180353240A1 · Florent et al. · 2018 [cited by applicant]
US 20200297444A1 · Camarillo · 2020 [cited by examiner]
US 20210000446A1 · Toporek et al. · 2021 [cited by applicant]
US 20210145412A1 · Mansi et al. · 2021 [cited by applicant]
JP 2014518123A · 2014 [cited by applicant]
JP 2019501703A · 2019 [cited by applicant]
JP 2021515647A · 2021 [cited by applicant]
Loschak et al., “A four degree of freedom robot for positioning ultrasound imaging catheters,” Journal of Mechanisms and Robotics, vol. 8, No. 5, 2016, pp. 051016-1-051016-9. [cited by applicant]
Liao et al., “An Artificial Agent for Robust Image Registration,” AAAI Conference on Artificial Intelligence. 2017. pp. 4168-4175. [cited by applicant]
Alkhouli, “Intracardiac Echocardiography in Structural Heart Disease Interventions”, JACC: Cardiovascular Interventions, 2018, pp. 2133-2147. [cited by applicant]
Hester et al., “Deep Q-learning from Demonstrations”, AAAI Conference on Artificial Intelligence. 2018, 8 pgs. [cited by applicant]
Zei, “Basic Intracardiac Ultrasound (ICE) in Electrophysiology”, Brigham and Women's Hospital, https://www.youtube.com/watch?v=kQLC8rUpnSo), 2020. [cited by applicant]
Enriquez et al., “Use of Intracardiac Echocardiography in Interventional Cardiology Working With the Anatomy Rather Than Fighting It”, Circulation, 2018. pp. 2278-2294. [cited by applicant]
Lu et al., “Automatic view planning for cardiac MRI acquisition,” 2011, Medical Image Computing and Computer- Assisted Intervention, pp. 479-486. [cited by applicant]
Alansary et al., “Automatic view planning with multi-scale deep reinforcement learning agents,” 2018, Medical Image Computing and Computer Assisted Intervention, pp. 277-285. [cited by applicant]
Hou et al., “Predicting slice-to-volume transformation in presence of arbitrary subject motion,” 2017, Medical Image Computing and Computer-Assisted Intervention, pp. 296-304. [cited by applicant]
Zhang et al., “Task Driven Generative Modeling for Unsupervised Domain Adaptation: Application to X-Ray Image Segmentation,” 2018, International Conference on Medical Image Computing and Computer-Assisted Intervention, … [cited by applicant]
Miao et al., “Dilated FCN for Multi-Agent 2D/3D Medical Image Registration,” 2018, Thirty-Second AAAI Conference on Artificial Intelligence, pp. 4694-4701. [cited by applicant]
Toth et al., “3D/2D model-to-image registration by imitation learning for cardiac procedures,” 2018, International Journal of Computer Assisted Radiology and Surgery vol. 13, pp. 1141-1149. [cited by applicant]
Cited By (1)
US 12,727,852