IP Library › Granted Patent US 12,721,592
Granted Patent B2
US 12,721,592 · App. 18/841,950 · Granted Sep 1, 2026

Autonomous robotic point of care ultrasound imaging

Inventors: Axel Krieger (Alexandria, VA); Thorsten Roger Fleiter (Baltimore, MD); Lidia Al-Zogbi (Baltimore, MD)
Assignees: THE JOHNS HOPKINS UNIVERSITY; UNIVERSITY OF MARYLAND, BALTIMORE
A61B8/4245A61B8/0875A61B8/4218A61B8/5207
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,721,592
App. No.
18/841,950
Granted
Sep 1, 2026
Kind
B2
Abstract

A system for and method of autonomously robotically acquiring an ultrasound image of an organ within a bony obstruction of a patient is presented. The techniques include acquiring an electronic three-dimensional patient-specific representation of the bony obstruction of the patient; obtaining an electronic representation of a target location in or on the organ within the bony obstruction of the patient; determining, automatically, position and orientation of an ultrasound probe to acquire an image of the target location; and directing, autonomously, and by a robot, the ultrasound probe on the patient to acquire the image of the target location based on the position and orientation.

Claims (39)

1 . A method of autonomously robotically acquiring an ultrasound image of an organ within a bony obstruction of a patient, the method comprising:

acquiring an electronic three-dimensional patient-specific representation of the bony obstruction of the patient;

obtaining an electronic representation of a target imaging location in or on the organ within the bony obstruction of the patient;

determining, automatically, a position and orientation of an ultrasound probe to acquire an image of the target imaging location, wherein the determining comprises determining the position and orientation based on a weighted function of a plurality of material densities, wherein the weighted function comprises a non-zero weight for bone material, a non-zero weight for air material, and a non-zero weight for soft tissue material, and wherein the weighted function comprises weights for pixels of the electronic three-dimensional patient-specific representation in between a point of contact of the ultrasound probe and the target imaging location; and

directing, autonomously, and by a robot, the ultrasound probe on the patient to acquire the image of the target imaging location based on the position and orientation.

2 . The method of claim 1 , further comprising outputting the image of the target imaging location.

3 . The method of claim 1 , wherein the bony obstruction comprises a ribcage, and wherein the organ comprises at least one of: a lung, a heart, a spleen, a liver, a pancreas, or a kidney.

4 . The method of claim 1 , wherein the acquiring comprises acquiring a three-dimensional radiological scan of the patient.

5 . The method of claim 1 , wherein the acquiring comprises acquiring a machine learning representation of the bony obstruction of the patient based on a topographical image of the patient.

6 . The method of claim 1 , wherein the obtaining comprises obtaining a human specified location in the electronic three-dimensional representation of the bony obstruction of the patient.

7 . The method of claim 1 , further comprising:

measuring a force on the ultrasound probe; and

determining a position of the ultrasound probe, based on the force, relative to the bony obstruction of the patient.

8 . The method of claim 1 ,

wherein the plurality of material densities comprises a bone density.

9 . The method of claim 8 ,

wherein the organ comprises a lung, and

wherein the plurality of material densities further comprises a density of air.

10 . The method of claim 1 , wherein an image of the target imaging location is acquired without requiring proximity of a technician to the patient.

11 . A system for autonomously robotically acquiring an ultrasound image of an organ within a bony obstruction of a patient, the system comprising:

an electronic processor that executes instructions to perform operations comprising:

acquiring an electronic three-dimensional patient-specific representation of the bony obstruction of the patient,

obtaining an electronic representation of a target imaging location in or on the organ within the bony obstruction of the patient, and

determining, automatically, a position and orientation of an ultrasound probe to acquire an image of the target imaging location, wherein the determining comprises determining the position and orientation based on a weighted function of a plurality of material densities, wherein the weighted function comprises a non-zero weight for bone material, a non-zero weight for air material, and a non-zero weight for soft tissue material, and wherein the weighted function comprises weights for pixels of the electronic three-dimensional patient-specific representation in between a point of contact of the ultrasound probe and the target imaging location; and

a robot communicatively coupled to the electronic processor, the robot comprising an effector couplable to an ultrasound probe, the robot configured to direct the ultrasound probe on the patient to acquire the image of the target imaging location based on the position and orientation.

12 . The system of claim 11 , wherein the operations further comprise outputting the image of the target imaging location.

13 . The system of claim 11 , wherein the bony obstruction comprises a ribcage, and wherein the organ comprises at least one of: a lung, a heart, a spleen, a liver, a pancreas, or a kidney.

14 . The system of claim 11 , wherein the acquiring comprises acquiring a three-dimensional radiological scan of the patient.

15 . The system of claim 11 , wherein the acquiring comprises acquiring a machine learning representation of the bony obstruction of the patient based on a topographical image of the patient.

16 . The system of claim 11 , wherein the obtaining comprises obtaining a human specified location in the electronic three-dimensional representation of the bony obstruction of the patient.

17 . The system of claim 11 , wherein the operations further comprise:

measuring a force on the ultrasound probe; and

determining a position of the ultrasound probe, based on the force, relative to the bony obstruction of the patient.

18 . The system of claim 11 ,

wherein the plurality of material densities comprises a bone density.

19 . The system of claim 18 ,

wherein the organ comprises a lung, and

wherein the plurality of material densities further comprises a density of air.

20 . The system of claim 11 , wherein the robot is configured to acquire an image of the target imaging location without requiring proximity of a technician to the patient.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 29, 2024
From: KRIEGER, AXEL; AL-ZOGBI, LIDIA
To: THE JOHNS HOPKINS UNIVERSITY
Reel/Frame 068437/0582 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 29, 2024
From: FLEITER, THORSTEN ROGER
To: UNIVERSITY OF MARYLAND, BALTIMORE
Reel/Frame 068437/0745 →
Continuity (2)
Provisional Application 63315115 · Mar 1, 2022
Related Publication 20250160787A1 · May 22, 2025
References Cited (51)
US 10521927B2 · Teixeira et al. · 2019 [cited by applicant]
US 20080200806A1 · Liu · 2008 [cited by examiner]
US 20200194117A1 · Krieger et al. · 2020 [cited by applicant]
US 20210236773A1 · Dupont et al. · 2021 [cited by applicant]
US 20220087654A1 · Mine · 2022 [cited by examiner]
CN 110477956A · 2019 [cited by applicant]
CN 111916195A · 2020 [cited by applicant]
CN 112151169A · 2020 [cited by applicant]
CN 112612274A · 2021 [cited by applicant]
CN 113288204A · 2021 [cited by applicant]
CN 113855068A · 2021 [cited by examiner]
CN 113858219A · 2021 [cited by applicant]
CN-113855068-A machine translation (Year: 2021). [cited by examiner]
Wang, X. (Authorized officer), International Preliminary Report on Patentability in corresponding International Application No. PCT/US2023/014008 mailed on Sep. 12, 2024, 11 pages. [cited by applicant]
Matos, T. (Authorized officer), International Search Report and Written Opinion in corresponding International Application No. PCT/US2023/014008 mailed on May 23, 2023, 12 pages. [cited by applicant]
Abramowicz, Jacques S. et al. “World federation for ultrasound in medicine and biology position statement: how to perform a safe ultrasound examination and clean equipment in the context of COVID-19.” Ultrasound in Medi… [cited by applicant]
Adams, Scott J. et al. “Telerobotic sonography for remote diagnostic imaging: narrative review of current developments and clinical applications.” Journal of ultrasound in medicine 40.7 (2021): 1287-1306. [cited by applicant]
Baad, Michael et al. “Clinical significance of US artifacts.” Radiographics 37.5 (2017): 1408-1423. [cited by applicant]
Blaivas, Michael. “Lung ultrasound in evaluation of pneumonia.” Journal of Ultrasound in Medicine 31.6 (2012): 823-826. [cited by applicant]
Elsayad, Alaa M. “Completely unsupervised image segmentation using wavelet analysis and Gustafson-Kessel clustering.” 2008 5th International Multi-Conference on Systems, Signals and Devices. IEEE, 2008. [cited by applicant]
He, Yisheng et al. “Pvn3d: a deep point-wise 3d keypoints voting network for 6dof pose estimation.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020. [cited by applicant]
Huang, Gao et al. “Densely connected convolutional networks.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. [cited by applicant]
Ioffe, Sergey et al. “Batch normalization: Accelerating deep network training by reducing internal covariate shift.” International conference on machine learning. pmlr, 2015. [cited by applicant]
Kim, Yeoun Jae et al. “Development of a control algorithm for the ultrasound scanning robot (NCCUSR) using ultrasound image and force feedback.” The International Journal of Medical Robotics and Computer Assisted Surger… [cited by applicant]
Kingma, Diederik P. “Adam: a method for stochastic optimization.” arXiv preprint arXiv:1412.6980 (2014). [cited by applicant]
Lee, Ta-Chih et al. “Building skeleton models via 3-D medial surface axis thinning algorithms.” CVGIP: graphical models and image processing 56.6 (1994): 462-478. [cited by applicant]
Liu, Xingyu et al. “Keypose: Multi-view 3d labeling and keypoint estimation for transparent objects.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020. [cited by applicant]
Mathur, Bharat et al. “A semi-autonomous robotic system for remote trauma assessment.” 2019 IEEE 19th International conference on bioinformatics and bioengineering (BIBE). IEEE, 2019. [cited by applicant]
Mathur, Bharat et al. “Evaluation of control strategies for a tele-manipulated robotic system for remote trauma assessment.” 2019 Proceedings of the Conference on Control and its Applications. Society for Industrial and… [cited by applicant]
Mylonas, George P. et al. “Autonomous eFAST ultrasound scanning by a robotic manipulator using learning from demonstrations.” 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2013. [cited by applicant]
Narayana, P. A., et al. “The attenuation of ultrasound in biological fluids.” The Journal of the Acoustical Society of America 76.1 (1984): 1-4. [cited by applicant]
Papazov, Chavdar et al. “Real-time 3D head pose and facial landmark estimation from depth images using triangular surface patch features.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2… [cited by applicant]
Paszke, Adam et al. “Automatic differentiation in pytorch.” (2017). [cited by applicant]
Peng, Qian-Yi et al. “Findings of lung ultrasonography of novel corona virus pneumonia during the 2019-2020 epidemic.” Intensive care medicine 46 (2020): 849-850. [cited by applicant]
Pomerleau, François et al. “Comparing ICP variants on real-world data sets: Open-source library and experimental protocol.” Autonomous robots 34 (2013): 133-148. [cited by applicant]
Quigley, Morgan et al. “ROS: an open-source Robot Operating System.” ICRA workshop on open source software. vol. 3. No. 3.2. 2009. [cited by applicant]
Smith-Guerin, N. et al. “Clinical validation of a mobile patient-expert tele-echography system using ISDN lines.” 4th International IEEE EMBS Special Topic Conference on Information Technology Applications in Biomedicin… [cited by applicant]
Szalma, József et al. “The influence of the chosen in vitro bone simulation model on intraosseous temperatures and drilling times.” Scientific Reports 9.1 (2019): 11817. [cited by applicant]
Teixeira, Brian et al. “Generating synthetic x-ray images of a person from the surface geometry.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. [cited by applicant]
Tsai, Roger Y. et al. “A new technique for fully autonomous and efficient 3 d robotics hand/eye calibration.” IEEE Transactions on robotics and automation 5.3 (1989): 345-358. [cited by applicant]
Virga Salvatore et al. “Automatic force-compliant robotic ultrasound screening of abdominal aortic aneurysms.” 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2016. [cited by applicant]
Wang, Jing et al. “Application of a Robotic Tele-Echography System for COVID-19 Pneumonia.” Journal of Ultrasound in Medicine 40.2 (2021): 385-390. [cited by applicant]
Wu, Shengzheng et al. “Robot-assisted teleultrasound assessment of cardiopulmonary function on a patient with confirmed COVID-19 in a cabin hospital.” Advanced Ultrasound in Diagnosis and Therapy 4.2 (2020): 128-130. [cited by applicant]
Xu, Bing et al. “Empirical evaluation of rectified activations in convolutional network.” arXiv preprint arXiv:1505.00853 (2015). [cited by applicant]
Yang, Geng et al. “Keep healthcare workers safe: application of teleoperated robot in isolation ward for COVID-19 prevention and control.” Chinese Journal of Mechanical Engineering 33 (2020): 1-4. [cited by applicant]
Ye, Ruizhong et al. “Feasibility of a 5G-based robot-assisted remote ultrasound system for cardiopulmonary assessment of patients with coronavirus disease 2019.” Chest 159.1 (2021): 270-281. [cited by applicant]
Zeiler, M. D. “ADADELTA: an Adaptive Learning Rate Method.” arXiv preprint arXiv:1212.5701 (2012). [cited by applicant]
Al-Zogbi et al., “A 3-D Printed Patient-Specific Ultrasound Phantom for Fast Scan,” Ultrasound in Med. & Biol., vol. 47, No. 3, pp. 820-832, 2021. [cited by applicant]
Buda et al., “Lung ultrasound in the diagnosis of COVID-19 infection—a case series and review of the literature,” Advances in Medical Sciences 65 (2020) 378-385. [cited by applicant]
Etchison, “Letter to the Editor,” Sports Health, Sep.-Oct. 2011, 1 page. [cited by applicant]
Laugier et al., “Chapter 2—Introduction to the Physics of Ultrasound,” ResearchGate, Nov. 2010, 18 pages. [cited by applicant]