IP Library › Granted Patent US 12,062,429

Granted Patent B2

US 12,062,429 · App. 17/270,150 · Granted Aug 13, 2024

Salient visual explanations of feature assessments by machine learning models

Inventors: Ze He (Cambridge, MA); Binyam Gebrekidan Gebre (Rosmalen, NL); Christine Menking Swisher (San Diego, CA)

Assignee: KONINKLIJKE PHILIPS N.V.

G16H30/40 G06N3/105 G06T7/0012 G06V10/235 G06V10/462 G06T2200/24 G06T2207/20081 G06V2201/031

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12,062,429
App. No.: 17/270,150
Granted: Aug 13, 2024
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Various embodiments of the present disclosure are directed to a salient medical imaging controller ( 80 ) employing an artificial intelligence engine ( 40 ) and a graphical user interface ( 70 ). In operation, the artificial intelligence engine ( 40 ) includes one or more machine learning models ( 42 ) trained to render a feature assessment of a medical image. The graphical user interface ( 70 ) provides a user interaction with the artificial intelligence engine ( 40 ) to manipulate a salient visualization of the feature assessment of the medical image by the machine learning model(s) ( 42 ).

Claims (49)

1. A medical image saliency controller, comprising:

a memory storing an artificial intelligence engine and a graphical user interface, wherein the artificial intelligence engine includes at least one machine learning model; and

a processor in communication with the memory, the processor being configured to:

apply the at least one machine learning model to a medical image data representative of at least one feature of a volumetric medical image to render a feature assessment of the volumetric medical image; and

control a display of the graphical user interface providing a user interaction with the artificial intelligence engine to manipulate a salient visualization of the feature assessment of the volumetric medical image by the at least one machine learning model;

wherein generating the salient visualization comprises reslicing a planar salient image from the assessed volumetric medical image based on a relevancy level of each feature of the assessed volumetric medical image to the feature assessment.

2. The medical image saliency controller of claim 1 , wherein the control of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image includes:

the processor being further configured to control the display of the graphical user interface providing a user specification of a relevancy threshold for each feature of the volumetric medical image to the salient visualization of the feature assessment of the volumetric medical image.

3. The medical image saliency controller of claim 1 , wherein the control of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image includes:

the processor being further configured to control the display of the graphical user interface providing a user specification of an alternative feature assessment of the volumetric medical image by the at least one machine learning model.

4. The medical image saliency controller of claim 1 , wherein the control of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image includes:

the processor being further configured to control the display of the graphical user interface providing a user specification of a masking of at least one of the features of the volumetric medical image.

5. The medical image saliency controller of claim 1 , wherein the control of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image includes:

the processor being further configured to control the display of the graphical user interface providing a user specification of a revision of at least one of the features of the volumetric medical image.

6. The medical image saliency controller of claim 1 ,

wherein the memory further stores an image display engine; and

wherein the processor is further configured to:

control a display of a salient image generated by the image display engine, the salient image being illustrative of the salient visualization of the feature assessment of the volumetric medical image.

7. The medical image saliency controller of claim 6 , wherein the control of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image includes:

the processor being further configured to control the display of the graphical user interface including at least one of the salient image and the volumetric medical image.

8. The medical image saliency controller of claim 6 , wherein the salient visualization of the feature assessment of the medical image includes at least one of a heat map, a feature segmentation and an activation diagram.

9. A non-transitory machine-readable storage medium encoded with instructions for execution by at least one processor, the non-transitory machine-readable storage medium comprising instructions to:

apply an artificial intelligence engine including at least one machine learning model to a medical image data representative of at least one feature of a volumetric medical image to render a feature assessment of the volumetric medical image; and

control a display of a graphical user interface providing a user interaction with the artificial intelligence engine to manipulate a salient visualization of the feature assessment of the volumetric medical image by the at least one machine learning model

10. The non-transitory machine-readable storage medium of claim 9 , wherein the instructions to control of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image include:

instructions to control of the display of the graphical user interface providing a user specification of a relevancy threshold for each feature of the volumetric medical image to the salient visualization of the feature assessment of the volumetric medical image.

11. The non-transitory machine-readable storage medium of claim 9 , wherein the instructions to control of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image include:

instructions to control the display of the graphical user interface providing a user specification of an alternative feature assessment of the volumetric medical image by the at least one machine learning model.

12. The non-transitory machine-readable storage medium of claim 9 , wherein the instructions to control of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image include:

instructions to control the display of the graphical user interface providing a user specification of a masking of at least one of the features of the volumetric medical image.

13. The non-transitory machine-readable storage medium of claim 9 , wherein the instructions to control of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the medical volumetric image include:

instructions to control the display of the graphical user interface providing a user specification of a revision of at least one of the features of the volumetric medical image.

14. The non-transitory machine-readable storage medium of claim 9 , wherein the non-transitory machine-readable storage medium further comprises instructions to:

control a display of a salient image generated by an image display engine, the salient image being illustrative of the salient visualization of the feature assessment of the volumetric medical image.

15. The non-transitory machine-readable storage medium of claim 14 , wherein the instructions to control of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the medical image include:

instructions to control the display of the graphical user interface including at least one of the salient image and the volumetric medical image.

16. A medical image saliency method, comprising:

applying an artificial intelligence engine including at least one machine learning model to a medical image data representative of at least one feature of a volumetric medical image to render a feature assessment of the medical image; and

controlling a display of the graphical user interface providing a user interaction with the artificial intelligence engine to manipulate a salient visualization of the feature assessment of the volumetric medical image by the at least one machine learning model

17. The medical image saliency method of claim 16 , wherein the controlling of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image includes:

controlling the display of the graphical user interface providing a user specification of a relevancy threshold for each feature of the medical image to the salient visualization of the feature assessment of the volumetric medical image.

18. The medical imaging saliency method of claim 16 , wherein the controlling of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the medical image includes:

controlling the display of the graphical user interface providing a user specification of an alternative feature assessment of the medical image by the at least one least one machine learning model.

19. The medical image saliency method of claim 16 , wherein the controlling of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image includes:

controlling the display of the graphical user interface providing a user specification of a masking of at least one of the features of the volumetric medical image.

20. The medical image saliency method of claim 16 , wherein the controlling of the display of the graphical user interface providing the user interaction with the artificial intelligence engine to manipulate the salient visualization of the feature assessment of the volumetric medical image includes:

controlling the display of the graphical user interface providing a user specification of a revision of at least one of the features of the volumetric medical image.

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 22, 2021

From: HE, ZE; GEBRE, BINYAM GEBREKIDAN; SWISHER, CHRISTINE MENKING

To: KONINKLIJKE PHILIPS N.V.

Reel/Frame 055351/0805 →

Continuity (2)

Provisional Application 62720229 · Aug 21, 2018

Related Publication 20210327563A1 · Oct 21, 2021