IP Library Granted Patent US 12,229,922
Granted Patent B1
US 12,229,922 · App. 17/722,347 · Granted Feb 18, 2025

Method and apparatus for a software enabled high resolution ultrasound imaging device

Inventor: Masoud Vaziri (Richardson, TX)
Assignee: OPTICS INNOVATION LLC
G06T5/50G06F18/251G06T7/0012G06T2207/10088G06T2207/10136
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,229,922
App. No.
17/722,347
Filed
Apr 17, 2022
Granted
Feb 18, 2025
Kind
B1
Art Unit
2485
USPC
348/78
Abstract

Various computational methods and techniques are presented to increase the lateral and axial resolution of an ultrasound imager in order to allow a medical practitioner to use an ultrasound imager in real time to obtain a 3D map of a portion of a body with a resolution that is comparable to an MRI machine.

Claims (37)

1. An imaging method for capturing an image of a slice of a volume, the method comprising:

acquiring, via an ultrasound imager, a first image corresponding to a first slice of the volume, the first image having a first resolution;

receiving, from a memory unit, a training image corresponding to a second slice of the volume, the training image having a second resolution that is higher than the first resolution, at least a portion of the second slice of the volume corresponding to at least a portion of the first slice of the volume; and

executing an image resolution enhancement procedure to increase the image resolution of a subset of the first image based at least in part on the training image.

2. The imaging method of claim 1 , wherein the increasing step comprises:

recognizing patterns in the first image and the training image; and

learning from the recognized patterns to increase the image resolution of the subset of the first image.

3. The imaging method of claim 1 , wherein the first slice of the volume is a subset of the second slice of the volume.

4. The imaging method of claim 1 , wherein the image resolution is increased in the increasing step via an iterative image resolution enhancement procedure.

5. The imaging method of claim 1 , further comprising the step of generating a 3D map of the volume using the first image.

6. The imaging method of claim 1 , further comprising the step of capturing the position and orientation of the ultrasound imager via a 9-axis accelerometer when the first image is captured.

7. The imaging method of claim 1 , wherein the image resolution is also increased in the increasing step based on the position and orientation of the ultrasound imager.

8. An imaging apparatus comprising:

an ultrasound imaging unit for capturing a first image corresponding to a first slice of a volume, the first image having a first resolution;

a memory unit for storing a training image corresponding to a second slice of the volume, the training image having a second resolution higher than the first resolution, the training image being captured with an MRI machine and prior to capturing the first image, at least a portion of the second slice of the volume corresponding to at least a portion of the first slice of the volume; and

a processor, in communication with the ultrasound imaging unit and the memory unit, the processor configured to:

execute an image resolution enhancement procedure to increase the image resolution of a subset of the first image based at least in part on the training image.

9. The imaging apparatus of claim 8 , wherein the image resolution enhancement procedure comprises:

recognizing patterns in the first image and the training image; and

learning from the recognized patterns to increase the image resolution of the subset of the first image.

10. The imaging apparatus of claim 8 , wherein the first slice of the volume is a subset of the second slice of the volume.

11. The imaging apparatus of claim 8 , wherein the image resolution enhancement procedure is an iterative image resolution enhancement procedure.

12. The imaging apparatus of claim 8 , wherein the processor is further configured to generate a 3D map of the volume using the first image.

13. The imaging apparatus of claim 8 , wherein the processor is further configured to capture the position and orientation of the ultrasound imager via a 9-axis accelerometer when the first image is captured.

14. The imaging apparatus of claim 8 , wherein the image resolution is also increased based on the position and orientation of the ultrasound imager.

15. An imaging apparatus comprising:

an ultrasound imaging unit for capturing a first image corresponding to a first slice of a first volume of a first human body, the first image having a first resolution;

a memory unit for storing a training image corresponding to a second slice of a second volume of a second human body, the training image having a second resolution higher than the first resolution, the training image being captured with an MRI machine and prior to capturing the first image, at least a portion of the first volume of the first human body corresponding to at least a portion of the second volume of the second human body; and

a processor, in communication with the ultrasound imaging unit and the memory unit, the processor configured to:

execute an image resolution enhancement procedure to increase the image resolution of a subset of the first image based at least in part on the training image.

16. The imaging apparatus of claim 15 , wherein the first and the second human bodies are the same.

17. The imaging apparatus of claim 15 , wherein the processor is further configured to generate a 3D map of the first volume using the at least one first image.

18. The imaging apparatus of claim 15 , wherein the processor is further configured to capture the position and orientation of the ultrasound imager via a 9-axis accelerometer when the first image is captured.

19. The imaging apparatus of claim 15 , wherein the image resolution is also increased based on the position and orientation of the ultrasound imager.

20. The imaging apparatus of claim 15 , further comprising capturing a second image via the ultrasound imager corresponding to a second slice of the first volume of the first human body, the ultrasound imager configured to operate at multiple frequencies and the first and the second images are taken at two different frequencies.

21. The imaging method of claim 1 , wherein the volume is a portion of a human body.

22. The imaging method of claim 1 , wherein the training image is acquired via an imager other than an ultrasound imager.

Assignments (3)
NUNC PRO TUNC ASSIGNMENT Recorded Nov 21, 2023
From: VAZIRI, MASOUD
To: U & LIFE CORPORATION
Reel/Frame 065640/0201 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 21, 2023
From: VAZIRI, MOJTABA
To: OPTICS INNOVATION LLC
Reel/Frame 065641/0781 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2022
From: VAZIRI, MASOUD, MR
To: VAZIRI, MOJTABA
Reel/Frame 061007/0534 →
Continuity (2)
Continuation 16507910 · Jul 10, 2019
Provisional Application 62696346 · Jul 11, 2018
References Cited (146)
US 4028725A · Lewis · 1977 [cited by applicant]
US 4907296A · Blecha · 1990 [cited by applicant]
US 5262871A · Wilder · 1993 [cited by applicant]
US 5856811A · Shih · 1999 [cited by applicant]
US 5859921A · Suzuki · 1999 [cited by applicant]
US 6163336A · Richards · 2000 [cited by applicant]
US 6198485B1 · Mack · 2001 [cited by applicant]
US 6307526B1 · Mann · 2001 [cited by applicant]
US 6434280B1 · Peleg · 2002 [cited by applicant]
US 6486799B1 · Still · 2002 [cited by applicant]
US 6661495B1 · Popovich · 2003 [cited by applicant]
US 6766067B2 · Freeman · 2004 [cited by applicant]
US 6850629B2 · Jeon · 2005 [cited by applicant]
US 7023464B1 · Harada · 2006 [cited by applicant]
US 7331671B2 · Hammond · 2008 [cited by applicant]
US 7391887B2 · Durnell · 2008 [cited by applicant]
US 7492926B2 · Kang · 2009 [cited by applicant]
US 7697024B2 · Currivan · 2010 [cited by applicant]
US 7715658B2 · Cho · 2010 [cited by applicant]
US 7894666B2 · Mitarai · 2011 [cited by applicant]
US 8014632B2 · Matsumoto · 2011 [cited by applicant]
US 8139089B2 · Doyle · 2012 [cited by applicant]
US 8159519B2 · Kurtz · 2012 [cited by applicant]
US 8305899B2 · Luo · 2012 [cited by applicant]
US 8432492B2 · Deigmoeller · 2013 [cited by applicant]
US 8872910B1 · Vaziri · 2014 [cited by applicant]
US 9230140B1 · Ackley · 2016 [cited by applicant]
US 9438491B1 · Van Broeck · 2016 [cited by applicant]
US 9438819B2 · Van Broeck · 2016 [cited by applicant]
US 9618746B2 · Browne · 2017 [cited by applicant]
US 9674490B2 · Koravadi · 2017 [cited by applicant]
US 9779311B2 · Lee · 2017 [cited by applicant]
US 9727790B1 · Vaziri · 2017 [cited by applicant]
US 9858676B2 · Bostick · 2018 [cited by applicant]
US 9864372B2 · Chen · 2018 [cited by applicant]
US 10039445B1 · Torch · 2018 [cited by applicant]
US 10064552B1 · Vaziri · 2018 [cited by applicant]
US 10708514B2 · Haltmaier · 2020 [cited by applicant]
US 11189017B1 · Baqai · 2021 [cited by applicant]
US 11287262B2 · Dooley · 2022 [cited by applicant]
US 20030122930A1 · Schofield · 2003 [cited by applicant]
US 20040212882A1 · Liang · 2004 [cited by applicant]
US 20040218834A1 · Bishop · 2004 [cited by applicant]
US 20060033992A1 · Solomon · 2006 [cited by applicant]
US 20070041663A1 · Cho · 2007 [cited by applicant]
US 20070115349A1 · Currivan · 2007 [cited by applicant]
US 20080010060A1 · Asano · 2008 [cited by applicant]
US 20080030592A1 · Border · 2008 [cited by applicant]
US 20080036875A1 · Jones · 2008 [cited by applicant]
US 20080198324A1 · Fuziak · 2008 [cited by applicant]
US 20080291295A1 · Kato · 2008 [cited by applicant]
US 20080297589A1 · Kurtz · 2008 [cited by applicant]
US 20090189974A1 · Deering · 2009 [cited by applicant]
US 20100053555A1 · Enriquez · 2010 [cited by applicant]
US 20100103276A1 · Border · 2010 [cited by applicant]
US 20100128135A1 · Filipovich · 2010 [cited by applicant]
US 20100157078A1 · Atanassov · 2010 [cited by applicant]
US 20100157079A1 · Atanassov · 2010 [cited by applicant]
US 20100208207A1 · Connell, II · 2010 [cited by applicant]
US 20100240988A1 · Varga · 2010 [cited by applicant]
US 20100254630A1 · Ali · 2010 [cited by applicant]
US 20100277619A1 · Scarff · 2010 [cited by applicant]
US 20100289941A1 · Ito · 2010 [cited by applicant]
US 20100290668A1 · Friedman · 2010 [cited by applicant]
US 20100290685A1 · Wein · 2010 [cited by examiner]
US 20110263946A1 · El Kaliouby · 2011 [cited by applicant]
US 20110279666A1 · Stromborn · 2011 [cited by applicant]
US 20120257005A1 · Browne · 2012 [cited by applicant]
US 20130106911A1 · Salsman · 2013 [cited by applicant]
US 20130121525A1 · Chen · 2013 [cited by applicant]
US 20130242057A1 · Hong · 2013 [cited by applicant]
US 20140146153A1 · Birnkrant · 2014 [cited by applicant]
US 20140267890A1 · Lelescu · 2014 [cited by applicant]
US 20140313335A1 · Koravadi · 2014 [cited by applicant]
US 20150009550A1 · Misago · 2015 [cited by applicant]
US 20150209002A1 · De Beni · 2015 [cited by examiner]
US 20160012280A1 · Ito · 2016 [cited by applicant]
US 20160179093A1 · Prokorov · 2016 [cited by applicant]
US 20160225192A1 · Jones · 2016 [cited by applicant]
US 20170007351A1 · Yu · 2017 [cited by applicant]
US 20170019599A1 · Muramatsu · 2017 [cited by applicant]
US 20170099479A1 · Browd · 2017 [cited by applicant]
US 20170142312A1 · Dal Mutto · 2017 [cited by applicant]
US 20170181802A1 · Sachs · 2017 [cited by applicant]
US 20170225336A1 · Deyle · 2017 [cited by applicant]
US 20170322410A1 · Watson · 2017 [cited by applicant]
US 20170360578A1 · Shin · 2017 [cited by examiner]
US 20180012413A1 · Jones · 2018 [cited by applicant]
US 20180188892A1 · Levac · 2018 [cited by applicant]
US 20180330473A1 · Foi · 2018 [cited by examiner]
US 20190175214A1 · Wood · 2019 [cited by examiner]
US 20190254754A1 · Johnson · 2019 [cited by applicant]
US 20190272336A1 · Ciecko · 2019 [cited by applicant]
US 20200041261A1 · Bernstein · 2020 [cited by applicant]
US 20200077033A1 · Chan · 2020 [cited by applicant]
US 20200117025A1 · Sauer · 2020 [cited by applicant]
US 20200330179A1 · Ton · 2020 [cited by applicant]
US 20210067764A1 · Shau · 2021 [cited by applicant]
Brattain et al. (Machine learning for medical ultrasound: status, methods, and future opportunities. Abdom Radiol (NY). Apr. 2018;43 (4):786-799. doi: 10.1007/s00261-018-1517-0. PMID: 29492605; PMCID: PMC5886811. [cited by examiner]
F. Coly et al. “Ultrasound Orientation Sensor”, BS thesis, Worcester Polytechnic Institute. Apr. 25, 2019. [cited by examiner]
“A High Speed Eye Tracking System with Robust Pupil Center Estimation Algorithm”, Proceedings of the 29th Ammal International Conference of the IEEE EMBS, 25 Kyon, France, pp. 3331-3334, Aug. 2007. [cited by applicant]
“A Novel Method of Video-Based Pupil Tracking”, Proceedings of the 2009 IEEE International Conference on Systems, Man and Cybernetics, San Antonio, Tex., USA—pp. 1255-1262, Oct. 2009 [cited by applicant]
Athanasios Papoulis; A New Algorithm in Spectral Analysis and Band-Limited Extrapolation; IEEE Transactions on Circuits and Systems, Sep. 1975; vol. CAS-22, No. 9; pp. 735-742. [cited by applicant]
Barbara Zitova et al.; Image Registration Methods: a Survey; Department of Image Processing; Institute of Information Theory and Automation Academy of Sciences of the Szech Republic; Image and Vision Computing; pp. 977-… [cited by applicant]
B. K. Gunturk, “Super-resolution imaging”, in Compu-tational Photography Methods and Applications, by R. Lukac, CRC Press, 2010 [Abstract Provided]. [cited by applicant]
Cheng et al. Developing a Real-Time Identify-and-Locate System for the Blind. Workshop on Computer Vision Applications for the Visually Impaired, James Coughlan and Roberto Manduchi, Oct. 2008, Marseille, France. [cited by applicant]
Danny Keren et al.; Image Sequence Enhancement Using Sub-pixel Displacements; Department of computer science; The Hebrew University of Jerusalem; 1988 IEEE; pp. 742-746. [cited by applicant]
D. Li, D. Winfield and D. Parkhurst, “Starburst: A Hybrid algorithm for video based eye tracking combining feature-based and model-based approaches”, Iowa State Univer-sity, Ames, Iowa. [cited by applicant]
Edward R. Dowski, Jr. et al.; Extended Depth of Field Through Wave-Front Coding; Apr. 10, 1995; Optical Society of America; vol. 34, No. 11; Applied Optics pp. 1859-1866. [cited by applicant]
Eran Gur and Zeev Zalevsky; Single-Image Digital Super-Resolution a Revised Gerchberg-Papoulis Algorithm; AENG International Journal of Computer Science; Nov. 17, 2007; pp. 1-5. [cited by applicant]
Extrema.m, http:/lwww.malhworks.com/matlabcentral/fileexchange/12275-extrema-m-extrema2-m, Sep. 14, 2006. [cited by applicant]
Eyelink User Manual, SR Research Lid., Copyright 2005-2008, 134 pages. [cited by applicant]
Eyelink Data Viewer User's Manual, SR Research Lid., Copyright 2002-2008, 149 pages. [cited by applicant]
Fritz Gleyo, Microsoft May Make Life-Sized Cortana In Person For HoloLens, (Dec. 14, 2015). [cited by applicant]
Guestrin et al. “General Theory of Remote Gaze Estimation Using the Pupil Center and Comeal Reflections”, IEEE Trans. Biomedical Eng., vol. 53, No. 6, pp. 1124-1133, June (2006). [cited by applicant]
Jessi Hempel, Project HoloLens: Our Exclusive Hands-On With Microsoft's Holographic Goggles, (Jan. 21, 2015). [cited by applicant]
John Bardsley et al.; Blind Iterative Restoration of Images With Spatially-Varying Blur, 9 pages. [cited by applicant]
J. Goodman, Introduction to Fourier Optics, rd edition, 160-165, McGraw-Hill, 1988. [cited by applicant]
Jordan Novet, Microsoft could build a life-sized Cortana for HoloLens, https://www.technologyrecord.com/Article/introducing-microsoft-hololens-development-edition-48296. [cited by applicant]
Kennet Kubala et al.; Reducing Complexity in Computational Imaging Systems; CDM Optics, Inc.; Sep. 8 J003; vol. 11, No. 18; Optics Express; pp. 2102-2108. [cited by applicant]
Lees et al. (Ultrasound Imaging in Three and Four Dimensions, Seminars in Ultrasound, CT, and MR/, vol. 22, No. 1 (February), 2001: pp. 85-105, (Year: 2001). [cited by applicant]
Lindsay James, Introducing Microsoft HoloLens Development Edition, https://blogs.windows.com/ :levices/2015/04/30/build-2015-a-closer-look-at-the-microsoft-hololens-hardware/. [cited by applicant]
Lisa Gottesfeld Brown; A Survey of Image Registration Techniques; Department of Computer Science; Columbia University; Jan. 12, 1992; pp. 1-60. [cited by applicant]
Malcolm et al. Combining topdown processes to guide eye movements during real-world scene search. Journal of Vision, 10(2):4, p. 1-11 (2010). [cited by applicant]
Maria E. Angelopoulou et al.; FPGA-based Real-lime Super-Resolution on an Adaptive Image Sensor; Department of Electrical and Electronic Engineering, Imperial College London; 9 pages. [cited by applicant]
Maria E. Angelopoulou et al.; Robust Real-Time Super-Resolution on FPGA and an Application to Video Enhancement; Imperial College London; ACM Journal Name; Sep. 2008; vol. V, No. N; pp. 1-27. [cited by applicant]
Moreno et al. Classification of visual and linguistic tasks using eye-movement features; Journal of Vision (2014) 14(3):11, 1-18. [cited by applicant]
Oliver Bowen et al.; Real-Time Image Super Resolution Using an FPGA; Department of Electrical and Electronic Engineering; Imperial College London; 2008 IEEE; pp. 89-94. [cited by applicant]
Patrick Vandewalle et al.; A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution; Ecole Polytechnique Federal de Lausanne, School of Computer and Communication Sciences; Depa… [cited by applicant]
P. C. Hansen, J. G. Nagy, D. P. O'Leary, Deblurring Images: matrices, Spectra and Filtering, SIAM (2006) [Abstract Provided]. [cited by applicant]
P. Milanfar, Super-Resolution Imaging, CRC Press (2011) [Abstract Provided]. [cited by applicant]
Pravin Bhat et al.; Using Photographs to Enhance Videos of a Static Scene; University of Washington; Microsoft Research; Adobe Systems; University of California; The Eurographics Association 2007; pp. 1-12. [cited by applicant]
R. W. Gerchberg, “Super-resolution through error energy reduction”, Optica Acta, vol. 21, No. 9, pp. 709-720,(1974). [cited by applicant]
S. Chaudhuri, Super-Resolution Imaging, Kluwer Aca-demic Publishers (2001) [Abstract Provided]. [cited by applicant]
Sang-Hyuck Lee et al.; Breaking Diffraction Limit of a Small F-Number Compact Camera Using Wavefront Coding; Center for Information Storage Device; Department of Mechanical Engineering, Yonsei University, Shinchondong, … [cited by applicant]
Suk Hwan Lim and Amnon Silverstein; Estimation and Removal of Motion Blur by Capturing Two Images With Different Exposures; HP Laboratories and NVidia Corp.; HPL-2008-170; Oct. 21, 2008; 8 pages. [cited by applicant]
S.-W Jung and S.-J. Ko, “Image deblurring using multi-exposed images” in Computational Photography 65 Methods and Applications, by R. Lukac, CRC Press, 2010. [Abstract Provided]. [cited by applicant]
Todd Holmdahl, Build 2015: A closer look at the Microsoft HoloLens hardware, https://blogs.windows.com/ :levices/2015/04/30/build-2015-a-closer-look-at-the-microsoft-hololens-hardware/. [cited by applicant]
Tod R. Lauer; Deconvolution With a Spatially-Variant PSF; National Optical Astronomy Observatory; Tucson, AZ; arXiv:astro-ph/0208247v1; Aug. 12, 2002; 7 pages. [cited by applicant]
V. Barmore, Iterative-Interpolation Super-Resolution Image Reconstruction, Springer (2009) [Abstract Provided]. [cited by applicant]
W. Thomas Cathey et al.; New Paradigm for Imaging Systems; Optical Society of America; Applied Optics; Oct. 10, 2002; vol. 41, No. 29; pp. 6080-6092. [cited by applicant]
William T. Freeman et al.; Example-Based Super-Resolution; Mitsubishi Electric Research Labs; March/April J002; IEEE Computer Graphics and Applications; pp. 56-65. [cited by applicant]
Sawhney, H. et al.; “Hybrid Stereo Camera: An IBR Approach for Synthesis of Very High Resolution Stereoscopic Image Sequences”; AC SIGGRAPH, pp. 451-460; 2001 (10 pages). [cited by applicant]
Zitnick, L. et al.; “Stereo for Image-Based Rendering Using Image Over-Segmentation”; International Journal of Computer Visions; 2006 (32 pages). [cited by applicant]
A Zandifar, R. Duraiswami, L.S. Davis, A video-based framework for the analysis of presentations/posters, 2003 (Year: 2003) 10 pages. [cited by applicant]
Z. Zalevsky, D. Mendlovic, Optical Superresolution, 2004 (Year: 2004) 261 pages. [cited by applicant]