IP Library › Granted Patent US 12,272,444
Granted Patent B2
US 12,272,444 · App. 18/180,119 · Granted Apr 8, 2025

Computer vision technologies for rapid detection

Inventors: Kevin Ward (Glen Allen, VA); Daniel Francis Taylor (Belleville, MI); Michael W Sjoding (Ann Arbor, MI); Christopher Elliot Gillies (Ann Arbor, MI)
Assignee: REGENTS OF THE UNIVERSITY OF MICHIGAN
G16H30/40G06F17/18G06T7/0012G06T11/20G06V10/25G06V10/454G06V10/7784G06V10/82G16H10/60G06T2207/10116G06T2207/20081G06T2207/20084G06T2207/30061G06T2210/12G06V10/248G06V10/987
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,272,444
App. No.
18/180,119
Granted
Apr 8, 2025
Kind
B2
Abstract

A computing system includes a processor; and a memory having stored thereon an adjustment application comprising computer-executable instructions that, when executed, cause the computing system to: display a graphical user interface including a digital medical image of a patient; superimpose a bounding box; receive an adjustment of an area of interest; and provide an adjusted digital medical image. A non-transitory computer-readable medium includes computer-executable instructions that, when executed via one or more processors, cause a computer to: display a graphical user interface including a digital medical image of a patient; superimpose a bounding box; receive an adjustment of an area of interest; and provide an adjusted digital medical image. A computer-implemented method includes: displaying a graphical user interface including a digital medical image of a patient; superimposing a bounding box; receiving an adjustment of an area of interest; and providing an adjusted digital medical image.

Claims (77)

1. A computing system for facilitating a clinician adjustment of a digital medical image corresponding to a patient used in conjunction with a machine learning model, comprising:

one or more processors; and

one or more memories having stored thereon an adjustment application comprising computer-executable instructions that, when executed, cause the computing system to:

train, via the one or more processors, the machine learning model to learn to detect acute respiratory distress syndrome based on an opacity of the digital medical image;

display, via the one or more processors, a graphical user interface including the digital medical image corresponding to the patient;

superimpose, over the digital medical image corresponding to the patient, a bounding box containing one or more areas of interest;

receive, via the one or more processors, an adjustment of at least one of the bounding box or the one or more areas of interest; and

provide, via the one or more processors, an adjusted digital medical image corresponding to the patient to the machine learning model, the adjusted digital medical image including the adjustment of at least one of the bounding box or the one or more areas of interest.

2. The computing system of claim 1 , the adjustment application comprising further instructions that, when executed by the one or more processors, cause the computing system to:

capture, via an imaging device, the digital medical image corresponding to the patient.

3. The computing system of claim 1 , wherein the adjustment of at least one of the bounding box or the one or more areas of interest is at least one of a repositioning, a rotation, an extension, an erasure, or a redrawing.

4. The computing system of claim 1 ,

wherein the machine learning model is a deep learning model being trained, and

the adjustment application comprising further instructions that, when executed by the one or more processors, cause the computing system to:

inject the adjusted digital medical image corresponding to the patient to an image processing pipeline;

reprocess the adjusted digital medical image; and

provide the adjusted digital medical image to the deep learning model to facilitate training of the deep learning model.

5. The computing system of claim 1 ,

wherein the machine learning model is a trained deep learning model being operated, and

the adjustment application comprising further instructions that, when executed by the one or more processors, cause the computing system to:

provide the adjusted digital medical image to the trained deep learning model to improve predictive accuracy of the deep learning model.

6. The computing system of claim 1 ,

wherein the digital medical image is a chest X-ray image, and

wherein the one or more areas of interest include one or more heat maps generated using a Gradient-weighted Class Activation Mapping (Grad-CAM) algorithm.

7. The computing system of claim 1 ,

wherein the digital medical image corresponds to a projectional radiography image, computed tomography (CT) scanning image or a magnetic resonance imaging (MRI) scanning image.

8. The computing system of claim 1 , wherein the machine learning model is trained to detect at least one of acute respiratory distress syndrome, trauma, sepsis, pneumonia, aspiration, congestive heart failure, hemothorax, pneumothorax, atelectasis, collapsed lung, pleural effusion or tuberculosis.

9. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed via one or more processors, cause a computer to:

train, via one or more processors, the machine learning model to learn to detect acute respiratory distress syndrome based on an opacity of the digital medical image;

display, via one or more processors, a graphical user interface including a digital medical image corresponding to a patient;

superimpose, over the digital medical image corresponding to the patient, a bounding box containing one or more areas of interest;

receive, via one or more processors, an adjustment of at least one of the bounding box or the one or more areas of interest; and

provide, via one or more processors, an adjusted digital medical image corresponding to the patient to a machine learning model, the adjusted digital medical image including the adjustment of at least one of the bounding box or the one or more areas of interest.

10. The non-transitory computer-readable medium of claim 9 , having stored thereon computer-executable instructions that, when executed via one or more processors, cause a computer to:

capture, via an imaging device, the digital medical image corresponding to the patient.

11. The non-transitory computer-readable medium of claim 9 , wherein the adjustment of at least one of the bounding box or the one or more areas of interest is at least one of a repositioning, a rotation, an extension, an erasure, or a redrawing.

12. The non-transitory computer-readable medium of claim 9 ,

wherein the machine learning model is a deep learning model being trained, and

the computer-readable medium having stored thereon computer-executable instructions that, when executed via one or more processors, cause a computer to:

inject the adjusted digital medical image corresponding to the patient to an image processing pipeline;

reprocess the adjusted digital medical image; and

provide the adjusted digital medical image to the deep learning model to facilitate training of the deep learning model.

13. The non-transitory computer-readable medium of claim 9 ,

wherein the machine learning model is a trained deep learning model being operated, and

the computer-readable medium having stored thereon computer-executable instructions that, when executed via one or more processors, cause a computer to:

provide the adjusted digital medical image to the trained deep learning model to improve predictive accuracy of the deep learning model.

14. The non-transitory computer-readable medium of claim 9 ,

wherein the digital medical image is a chest X-ray image, and

wherein the one or more areas of interest include one or more heat maps generated using a Gradient-weighted Class Activation Mapping (Grad-CAM) algorithm.

15. The non-transitory computer-readable medium of claim 9 , wherein the machine learning model is trained to detect at least one of acute respiratory distress syndrome, trauma, sepsis, pneumonia, aspiration, congestive heart failure, hemothorax, pneumothorax, atelectasis, collapsed lung, pleural effusion or tuberculosis.

16. The non-transitory computer-readable medium of claim 9 ,

wherein the digital medical image corresponds to a projectional radiography image, computed tomography (CT) scanning image or a magnetic resonance imaging (MRI) scanning image.

17. A computer-implemented method for facilitating a clinician adjustment of a digital medical image corresponding to a patient used in conjunction with a machine learning model, the method comprising:

training, via one or more processors, the machine learning model to learn to detect acute respiratory distress syndrome based on an opacity of the digital medical image;

displaying, via one or more processors, a graphical user interface including the digital medical image corresponding to a patient;

superimposing, over the digital medical image corresponding to the patient, a bounding box containing one or more areas of interest;

receiving, via one or more processors, an adjustment of at least one of the bounding box or the one or more areas of interest; and

providing, via one or more processors, an adjusted digital medical image corresponding to the patient to a machine learning model, the adjusted digital medical image including the adjustment of at least one of the bounding box or the one or more areas of interest.

18. The computer-implemented method of claim 17 , further comprising:

capture, via an imaging device, the digital medical image corresponding to the patient.

19. The computer-implemented method of claim 17 , wherein the adjustment of at least one of the bounding box or the one or more areas of interest is at least one of a repositioning, a rotation, an extension, an erasure, or a redrawing.

20. The computer-implemented method of claim 17 ,

wherein the machine learning model is a deep learning model being trained, and

further comprising:

injecting the adjusted digital medical image corresponding to the patient to an image processing pipeline;

reprocessing the adjusted digital medical image; and

providing the adjusted digital medical image to the deep learning model to facilitate training of the deep learning model.

21. The computer-implemented method of claim 17 ,

wherein the machine learning model is a trained deep learning model being operated, and

further comprising:

providing the adjusted digital medical image to the trained deep learning model to improve predictive accuracy of the deep learning model.

22. The computer-implemented method of claim 17 ,

wherein the digital medical image is a chest X-ray image, and

wherein the one or more areas of interest include one or more heat maps generated using a Gradient-weighted Class Activation Mapping (Grad-CAM) algorithm.

23. The computer-implemented method of claim 17 , wherein the machine learning model is trained to detect at least one of acute respiratory distress syndrome, trauma, sepsis, pneumonia, aspiration, congestive heart failure, hemothorax, pneumothorax, atelectasis, collapsed lung, pleural effusion or tuberculosis.

24. The computer-implemented method of claim 17 ,

wherein the digital medical image corresponds to a projectional radiography image, computed tomography (CT) scanning image or a magnetic resonance imaging (MRI) scanning image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2023
From: WARD, KEVIN; TAYLOR, DANIEL FRANCIS; SJODING, MICHAEL W.; GILLIES, CHRISTOPHER ELLIOT
To: REGENTS OF THE UNIVERSITY OF MICHIGAN
Reel/Frame 063864/0010 →
Continuity (3)
Continuation 17082145 · Oct 28, 2020
Provisional Application 62951695 · Dec 20, 2019
Related Publication 20230230680A1 · Jul 20, 2023
References Cited (24)
US 9589374B1 · Gao et al. · 2017 [cited by applicant]
US 10290101B1 · Podilchuk et al. · 2019 [cited by applicant]
US 11176415B2 · Irshad · 2021 [cited by examiner]
US 20180310828A1 · DiMaio et al. · 2018 [cited by applicant]
US 20190156484A1 · Nye et al. · 2019 [cited by applicant]
US 20190164285A1 · Nye et al. · 2019 [cited by applicant]
US 20200098154A1 · Marinier · 2020 [cited by examiner]
US 20200161005A1 · Lyman et al. · 2020 [cited by applicant]
US 20200211695A1 · Zheng et al. · 2020 [cited by applicant]
US 20210366110A1 · Oosake · 2021 [cited by examiner]
WO WO2019217701A1 · 2019 [cited by examiner]
WO WO2019245597A1 · 2019 [cited by examiner]
WO WO2020157761A1 · 2020 [cited by examiner]
Benenson, Rodrigo, Stefan Popov, and Vittorio Ferrari. “Large-scale interactive object segmentation with human annotators.” Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019. [cited by examiner]
Wang, Guotai, et al. “Interactive medical image segmentation using deep learning with image-specific fine tuning.” IEEE transactions on medical imaging 37.7 (2018): 1562-1573. [cited by examiner]
Khaizi, Aida Syafiqah Ahmad, et al. “A mini review on the design of interactive tool for medical image segmentation.” 2017 International Conference on Engineering Technology and Technopreneurship (ICE2T). IEEE, 2017. [cited by examiner]
Tan, Yanhao, Ke Lu, and Jian Xue. “An Automated Lung Nodule Segmentation Method Based On Nodule Detection Network and Region Growing.” Proceedings of the 1st ACM International Conference on Multimedia in Asia. 2019. [cited by examiner]
“What is CheXpert?” (2019). Retreived from the Internet at: <URL:https://stanfordmlgroup.github.io/competitions/chexpert/>. [cited by applicant]
Bellani et al., “Epidemiology, patterns of care, and mortality for patients with acute respiratory distress syndrome in intensive care units in 50 countries,” Jama, 315(8):788-800 (2016). [cited by applicant]
Ronneberger et al., “U-net: Convolutional networks for biomedical image segmentation.” International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 2015. [cited by applicant]
Selvaraju et al., “Grad-cam: Visual explanations from deep networks via gradient-based localization,” Proceedings of the IEEE International Conference on Computer Vision. 2017. [cited by applicant]
Zompatori et al., “Overview of current lung imaging in acute respiratory distress syndrome,” Eur Respiratory Soc. (2014). [cited by applicant]
Leiner, T., Rueckert, D., Suinesiaputra, A. et al. Machine learning in cardiovascular magnetic resonance: basic concepts and applications. J Cardiovasc Magn Reson 21, 61 (2019). [cited by applicant]
U.S. Appl. No. 17/082,145, entitled Computer Vision Technologies for Rapid Detection, filed Oct. 28, 2020. [cited by applicant]
Cited By (1)
US 1,099,918