IP Library › Granted Patent US 12,347,550
Granted Patent B2
US 12,347,550 · App. 18/480,392 · Granted Jul 1, 2025

Imaging discovery utility for augmenting clinical image management

Inventors: Mark Palmer (Oak Grove, MN); Kalvin Parman (Maple Grove, MN); Ryan P. Lahm (Lino Lakes, MN); Megan Samec (Little Canada, MN); Srinivasan S. Varahoor (Windsor, CA); Priya Nair (Hayward, CA); Julianne Spencer (New Brighton, MN); Walton W. Baxter (San Clemente, CA); Callie Cronin Myers (Minneapolis, MN)
Assignee: Medtronic, Inc.
G16H30/20G06F16/221G16H30/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,347,550
App. No.
18/480,392
Granted
Jul 1, 2025
Kind
B2
Abstract

Novel tools and techniques are provided for implementing an imaging discovery utility for augmenting clinical image management. In some embodiments, in response to receiving a request for a first medical image file(s) from a requesting device, a non-relational (“NoSQL”) data management system (“DMS”) may access a NoSQL database containing, inter alia, a plurality of medical image files that are mirrored copies of a plurality of medical image files stored in a relational (“SQL”) database, the medical image files each being organized in an image-centric hierarchy with image data being at a top level and patient information associated with the image data being at a lower level. Based on a successful search of the NoSQL database based on search terms in the request, the NoSQL DMS may identify a corresponding second medical image(s) in the SQL database, and may retrieve and send (to the requesting device) the identified second medical image(s).

Claims (55)

1. A method, comprising:

determining that contents of one of a non-relational (“NoSQL”) database, via a NoSQL data management system (“DMS”), or a relational (“SQL”) database, via a SQL DMS, have changed;

autonomously mirroring the changed contents in the other of the NoSQL database or the SQL database, via the respective DMS, based on the determination, the NoSQL database containing a plurality of medical image files, a plurality of metadata tags associated with each medical image file, and a plurality of artifact files associated with two or more of the plurality of medical image files, the NoSQL database containing mirrored copies of the plurality of medical image files that are stored in the SQL database, wherein the plurality of medical image files contained in the NoSQL database and in the SQL database are each organized in an image-centric hierarchy with image data being at a top level of the image-centric hierarchy and patient information associated with the image data being at a level below the top level of the image-centric hierarchy;

receiving, with the NoSQL DMS and from a requesting device, a request for the plurality of medical image files, the request comprising one or more search terms;

based on a determination that at least one metadata tag associated with the plurality of medical image files contains keywords that correspond to or match at least one search term among the one or more search terms:

identifying, with the NoSQL DMS, corresponding medical image files of the plurality of medical image files as stored in the SQL database;

retrieving, with at least one of the NoSQL DMS or the SQL DMS and from the SQL database, the plurality of medical image files; and

sending, with the at least one of the NoSQL DMS or the SQL DMS, the retrieved plurality of medical image files to the requesting device.

2. The method of claim 1 , further comprising:

receiving, with the NoSQL DMS and from the requesting device, a request for at least one first medical image file, the request comprising one or more search terms;

accessing, with the NoSQL DMS, the NoSQL database that is managed by the NoSQL DMS; and

performing, with the NoSQL DMS, a search of the NoSQL database, by searching for each of the one or more search terms in the plurality of metadata tags associated with each medical image file among the plurality of medical image files in the NoSQL database.

3. The method of claim 2 , further comprising:

wherein identifying further includes identifying, with the NoSQL DMS, corresponding one or more second medical image files as stored in the SQL database, wherein the search of the NoSQL database serves as an index search of the SQL database;

wherein retrieving further includes retrieving, with at least one of the NoSQL DMS or the SQL DMS and from the SQL database, the one or more second medical image files; and

wherein sending further includes sending, with the at least one of the NoSQL DMS or the SQL DMS, the retrieved one or more second medical image files to the requesting device.

4. The method of claim 2 , wherein the requesting device comprises at least one of a laptop computer, a desktop computer, a tablet computer, a smart phone, a mobile phone, or a web client.

5. The method of claim 3 , further comprising:

receiving, with a computing system and from the requesting device, the request for the at least one first medical image file; and

querying, with the computing system, the NoSQL DMS for the requested at least one first medical image file, by relaying the request to the NoSQL DMS.

6. The method of claim 3 , further comprising:

generating, with a model generation system, one or more three-dimensional (“3D”) models based on at least one thirteenth medical image file among the retrieved one or more second medical image files.

7. The method of claim 1 , wherein the plurality of metadata tags comprises flexible image tagging that is customizable to include one or more first medical image metadata and one or more second medical image metadata.

8. The method of claim 7 , wherein the one or more first medical image metadata comprise one or more digital imaging and communications in medicine (“DICOM”)-compliant metadata.

9. The method of claim 1 , further comprising:

receiving, with the NoSQL DMS and from a requesting device, a request to modify a metadata tag associated with a medical image file that is stored and mirrored in each of the NoSQL database and the SQL database; and

presenting, via one of the NoSQL DMS or the SQL DMS, a user interface (“UI”) with options for dynamically modifying metadata tags associated with the medical image file.

10. The method of claim 9 , wherein the options to dynamically modify metadata tags includes at least one selected from the group consisting of dynamically adding group specific metadata tags to metadata fields of the medical image file, dynamically modifying group specific metadata tags in metadata fields of the medical image file, dynamically deleting group specific metadata tags in metadata fields of the medical image file, dynamically defining new group specific metadata fields of the medical image file, dynamically modifying group specific metadata fields of the medical image file, dynamically deleting group specific metadata fields of the medical image file, and dynamically linking a metadata field with one or more artifact files.

11. A method, comprising:

accessing, with a non-relational (“NoSQL”) data management system (“DMS”), a NoSQL database that is managed by the NoSQL DMS, the NoSQL database containing a plurality of medical image files, a plurality of metadata tags associated with each medical image file, and a plurality of artifact files associated with two or more of the plurality of medical image files, the NoSQL database containing mirrored copies of the plurality of medical image files that are stored in a relational (“SQL”) database that is managed by a SQL DMS;

performing, with the NoSQL DMS, a search of the NoSQL database, by searching for each of one or more search terms in the plurality of metadata tags associated with each medical image file among the plurality of medical image files in the NoSQL database; and

based on a determination that at least one metadata tag associated with one or more second medical image files among the plurality of medical image files contains keywords that correspond to or match at least one search term among the one or more search terms:

identifying, with the NoSQL DMS, corresponding one or more second medical image files as stored in the SQL database, wherein the search of the NoSQL database serves as an index search of the SQL database;

retrieving, with at least one of the NoSQL DMS or the SQL DMS and from the SQL database, the one or more second medical image files; and

sending, with the at least one of the NoSQL DMS or the SQL DMS, the retrieved one or more second medical image files to a requesting device.

12. The method of claim 11 , further comprising:

receiving, with a computing system and from a requesting device, a request for at least one first medical image file; and

querying, with the computing system, the NoSQL DMS for the requested at least one first medical image file, by relaying the request to the NoSQL DMS.

13. The method of claim 12 , further comprising:

presenting, with the computing system, a user interface (“UI”) via the requesting device;

wherein the request is received via the UI; and

wherein sending the retrieved one or more second medical image files to the requesting device includes displaying the retrieved one or more second medical image files in the UI.

14. The method of claim 13 , further comprising:

converting, with the computing system, the retrieved one or more second medical image files into one or more digital imaging and communications in medicine (“DICOM”)-compliant search results;

wherein displaying the retrieved one or more second medical image files in the UI includes displaying the retrieved one or more second medical image files as the converted one or more DICOM-compliant search results in the UI.

15. The method of claim 14 , further comprising:

based on a determination that a user's mouse cursor is hovering over a DICOM-compliant search result, generating and displaying, with the computing system, a pop-up display indicating specific information or values associated with the DICOM-compliant search result, based at least in part on the one or more search terms.

16. The method of claim 13 , further comprising:

providing, with the computing system, one or more task options for each of the displayed retrieved one or more second medical image files in the UI;

wherein the one or more task options includes at least one selected from the group consisting of options to download particular one or more third medical image files among the one or more second medical image files, options to add particular one or more fourth medical image files among the one or more second medical image files to at least one project group folder, options to link together at least two fifth medical image files among the one or more second medical image files, options to export one or more sixth medical image files among the one or more second medical image files, options to share one or more seventh medical image files among the one or more second medical image files, options to upload one or more eighth medical image files, options to start a new search, options to save a current search, options to manage saved searches, options to view recent searches, options to export searches, options to open a new project folder, options to save a current project folder, options to manage project folders, options to complete a current project, options to view active projects, and options to export project folders.

17. The method of claim 16 , wherein two or more ninth medical image files among the one or more second medical image files that are obtained during a single imaging session are grouped together as a single imaging asset with at least one of shared metadata, common metadata, or linked metadata, wherein the one or more task options further includes at least one selected from the group consisting of options to preview the two or more ninth medical image files together in the UI, options to download the two or more ninth medical image files as a single set of image files, options to export the two or more ninth medical image files as a single set of image files, options to share the two or more ninth medical image files as a single set of image files, and options to edit metadata associated with each of the two or more ninth medical image files individually or as a group.

18. The method of claim 16 , wherein the one or more task options further includes at least one selected from the group consisting of options to perform auto-segmentation for one or more tenth medical image files among the one or more second medical image files and options to perform auto-measurements for one or more eleventh medical image files among the one or more second medical image files.

19. The method of claim 12 , wherein the computing system includes at least one selected from the group consisting of a processor of a NoSQL DMS, a processor of a SQL DMS, a processor of a hybrid NoSQL/SQL DMS, a controller for both the NoSQL DMS and the SQL DMS, a server computer over a network, a cloud-based computing system over a network, and a distributed computing system.

20. The method of claim 11 , further comprising:

generating, with a model generation system, one or more three-dimensional (“3D”) models based on at least one thirteenth medical image file among the retrieved one or more second medical image files.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2023
From: PALMER, MARK; PARMAN, KALVIN; LAHM, RYAN P.; HARRIS, MEGAN L.; VARAHOOR, SRINIVASAN; NAIR, PRIYA; SPENCER, JULIANNE H.; BAXTER, WALTON W.; CRONIN, CALLIE
To: MEDTRONIC, INC.
Reel/Frame 065131/0805 →
Continuity (3)
Continuation 17349361 · Jun 16, 2021
Provisional Application 63075413 · Sep 8, 2020
Related Publication 20240029865A1 · Jan 25, 2024
References Cited (103)
US 7187964B2 · Khoury · 2007 [cited by applicant]
US 7892165B2 · Nakamura · 2011 [cited by applicant]
US 7998095B2 · McAuley · 2011 [cited by applicant]
US 8460282B2 · McAuley · 2013 [cited by applicant]
US 8482859B2 · Border et al. · 2013 [cited by applicant]
US 8768022B2 · Miga et al. · 2014 [cited by applicant]
US 8792693B2 · Satish et al. · 2014 [cited by applicant]
US 8902254B1 · Laughlin et al. · 2014 [cited by applicant]
US 9128281B2 · Osterhout et al. · 2015 [cited by applicant]
US 9134534B2 · Border et al. · 2015 [cited by applicant]
US 9223134B2 · Miller et al. · 2015 [cited by applicant]
US 9232982B2 · Soler et al. · 2016 [cited by applicant]
US 9298884B1 · Ahmad · 2016 [cited by applicant]
US 9317743B2 · Datta et al. · 2016 [cited by applicant]
US 9452294B2 · Kaula et al. · 2016 [cited by applicant]
US 9492241B2 · Joskowicz et al. · 2016 [cited by applicant]
US 9526443B1 · Berme et al. · 2016 [cited by applicant]
US 9615788B2 · Kaula et al. · 2017 [cited by applicant]
US 9639953B2 · Moraviec · 2017 [cited by applicant]
US 9642606B2 · Charles et al. · 2017 [cited by applicant]
US 9659104B2 · Soon-Shiong et al. · 2017 [cited by applicant]
US 9687301B2 · Lee et al. · 2017 [cited by applicant]
US 9720505B2 · Gribetz et al. · 2017 [cited by applicant]
US 9740296B2 · Cohen et al. · 2017 [cited by applicant]
US 9767608B2 · Lee et al. · 2017 [cited by applicant]
US 9773312B2 · Lee · 2017 [cited by applicant]
US 9855103B2 · Tsekos et al. · 2018 [cited by applicant]
US 9866767B2 · Jones · 2018 [cited by applicant]
US 9870060B2 · Marggraff et al. · 2018 [cited by applicant]
US 9875540B2 · Blumhofer et al. · 2018 [cited by applicant]
US 9888973B2 · Olson et al. · 2018 [cited by applicant]
US 9911225B2 · Engel et al. · 2018 [cited by applicant]
US 9928588B2 · Vilsmeier · 2018 [cited by applicant]
US 9947104B2 · Seiler et al. · 2018 [cited by applicant]
US 9986983B2 · Weingarten et al. · 2018 [cited by applicant]
US 10026227B2 · Laughlin et al. · 2018 [cited by applicant]
US 10152796B2 · Guo et al. · 2018 [cited by applicant]
US 10180572B2 · Osterhout et al. · 2019 [cited by applicant]
US 20020198891A1 · Li et al. · 2002 [cited by applicant]
US 20070173861A1 · Strommer et al. · 2007 [cited by applicant]
US 20080033527A1 · Nunez et al. · 2008 [cited by applicant]
US 20100198346A1 · Keogh et al. · 2010 [cited by applicant]
US 20120188352A1 · Wittenberg et al. · 2012 [cited by applicant]
US 20130035757A1 · Zentgraf et al. · 2013 [cited by applicant]
US 20130117273A1 · Lee · 2013 [cited by examiner]
US 20130339366A1 · Khimich · 2013 [cited by examiner]
US 20140128726A1 · Quill et al. · 2014 [cited by applicant]
US 20140176661A1 · Smurro et al. · 2014 [cited by applicant]
US 20140279881A1 · Tan · 2014 [cited by examiner]
US 20140296704A1 · Alves De Inda et al. · 2014 [cited by applicant]
US 20150142783A1 · Bruce · 2015 [cited by examiner]
US 20160019716A1 · Huang et al. · 2016 [cited by applicant]
US 20160085774A1 · Bhamidipati et al. · 2016 [cited by applicant]
US 20160350303A1 · Fischer et al. · 2016 [cited by applicant]
US 20170021132A1 · Zentgraf et al. · 2017 [cited by applicant]
US 20170098333A1 · Varga · 2017 [cited by applicant]
US 20170103581A1 · Mullins et al. · 2017 [cited by applicant]
US 20170109484A1 · Herger et al. · 2017 [cited by applicant]
US 20170206419A1 · Mullins · 2017 [cited by applicant]
US 20170221387A1 · Lampotang et al. · 2017 [cited by applicant]
US 20170258585A1 · Marquez et al. · 2017 [cited by applicant]
US 20170293805A1 · Kontschieder et al. · 2017 [cited by applicant]
US 20170295236A1 · Kulkarni · 2017 [cited by examiner]
US 20170323148A1 · Sarkar et al. · 2017 [cited by applicant]
US 20170340396A1 · Romo et al. · 2017 [cited by applicant]
US 20180012416A1 · Jones et al. · 2018 [cited by applicant]
US 20190183576A1 · Fahim et al. · 2019 [cited by applicant]
US 20190188309A1 · Anderson · 2019 [cited by examiner]
US 20190298450A1 · Dasi et al. · 2019 [cited by applicant]
US 20190350671A1 · Varshney et al. · 2019 [cited by applicant]
US 20190362556A1 · Ben-Dor et al. · 2019 [cited by applicant]
US 20190384764A1 · Taylor · 2019 [cited by applicant]
US 20200060765A1 · Fahim et al. · 2020 [cited by applicant]
US 20200129136A1 · Harding et al. · 2020 [cited by applicant]
US 20210369393A1 · Braido et al. · 2021 [cited by applicant]
US 20210369394A1 · Braido et al. · 2021 [cited by applicant]
CN 107077511A · 2017 [cited by examiner]
CN 107515867A · 2017 [cited by examiner]
CN 108009296A · 2018 [cited by examiner]
CN 108984598A · 2018 [cited by examiner]
CN 111739613A · 2020 [cited by examiner]
KR 101715019B1 · 2017 [cited by examiner]
TR 201720331A2 · 2018 [cited by examiner]
WO 2001020552A1 · 2001 [cited by applicant]
WO 2009148317A1 · 2009 [cited by applicant]
WO 2021243310A1 · 2021 [cited by applicant]
WO 2021243311A1 · 2021 [cited by applicant]
WO 2021243313A1 · 2021 [cited by applicant]
WO 2021243314A1 · 2021 [cited by applicant]
WO 2022055588A1 · 2022 [cited by applicant]
Costa et al., “Medical imaging archiving: A comparison between several NoSQL solutions”, Jun. 2014 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), (Year: 2014). [cited by examiner]
International Search Report & Written Opinion, International Application No. PCT/US2021/035033 dated Sep. 14, 2021 (16 pages). [cited by applicant]
International Search Report & Written Opinion, International Application No. PCT/US2021/035034 dated Sep. 16, 2021 (16 pages). [cited by applicant]
Zhang Q et al: “Dynamic real-time 4D cardiac MDCT image display using GPU-accelerated volume rendering”, Computerized Medical Imaging and Graphics, Pergamon Press, New York, NY, US, vol. 33, No. 6, Sep. 1, 2009, 16 page… [cited by applicant]
Azizian Mahdi et al: “Intraoperative 3D stereo visualization for image-guided cardiac ablation”, Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling, Spie, vol. 7964, No. 1, Mar. 2, 2011, 8 pages. [cited by applicant]
Linte CA et al: “Virtual and Augmented Medical Imaging Environments: Enabling Technology for Minimally Invasive Cardiac Interventional Guidance”, IEEE Reviews in Biomedical Engineering, IEEE, USA, vol. 3, Jan. 1, 2010, … [cited by applicant]
Robb A. Using patient specific anatomic models, IEEE Engineering in Medicine and Biology Magazine, IEEE Service Center, Pisacataway, NJ, US, vol. 15, No. 2, Mar. 1, 1996, 10 pages. [cited by applicant]
International Search Report & Written Opinion, International Application No. PCT/US2021/035036 dated Septmember 16, 2021 (16 pages). [cited by applicant]
International Search Report & Written Opinion, International Application No. PCT/US2021/035037 dated Sep. 16, 2021 (16 pages). [cited by applicant]
Suematsu et al., “Robotic-assisted closure of atrial septal defect under real-time three-dimensional echo guide in vitro study”, CP Journal of cardio-Thoracic Surgery, vol. 32, No. 4, dated Sep. 6, 2007, 4 pages. [cited by applicant]
International Search Report & Written Opinion, International Application No. PCT/US2021/037655 dated Oct. 4, 2021 (19 pages). [cited by applicant]
Silva et al. “Medical Imaging archiving: A comparison between several NoSQL solutions.” IEEE EMBS International Conference on Information Technology Applications in Biomedicine (ITAB) [online], 2014 [Retrieved on Aug. 1… [cited by applicant]
Costa et al., “Medical imaging archiving: A comparison between several NoSQL solutions”, IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Jun. 2014. [cited by applicant]