IP Library Granted Patent US 10,607,122
Granted Patent B2
US 10,607,122 · App. 15/831,057 · Granted Mar 31, 2020

Systems and user interfaces for enhancement of data utilized in machine-learning based medical image review

Inventors: Aviad Zlotnick (Mitzpeh Netifah, IL); Alon Hazan (Zikhron Ya'akov, IL); Murray A. Reicher (Rancho Santa Fe, CA)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06K9/66G06F3/04845G06K9/6254G06K9/6263G06K9/6267G06K2209/05
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,607,122
App. No.
15/831,057
Granted
Mar 31, 2020
Kind
B2
Abstract

Systems and techniques are disclosed for improvement of machine learning systems based on enhanced training data. An example method includes generating an interactive classification user interface concurrently displaying a first group of medical images and a second group of medical images, each group depicting objects associated with a respective classification. User input indicating movement of medical images from the first group to the second group is detected. The moved medical images are classified according to the second group. The re-classified medical images are provided to a machine learning system, with the machine learning system updating based on analysis of object characteristics of the re-classified medical images to increase accuracies associated with automated assignment of classifications.

Claims (33)

1. A method for facilitating proper classification of features in medical images comprising:

generating, for display on a display of a computing system, an interactive classification user interface concurrently displaying a first group of medical images and a second group of medical images, wherein each of the first group of medical images depicts an object associated with a first classification and each of the second group of medical images depicts an object associated with a second classification;

detecting user input indicating movement of one or more medical images from the first group to the second group;

in response to the user input, re-classifying the one or more medical images as associated with the second classification; and

providing the re-classifications to a machine learning system.

2. The method of claim 1 , wherein the first group of medical images is displayed as a grid.

3. The method of claim 1 , wherein the first group of medical images is displayed as a stack.

4. The method of claim 1 , wherein each medical image is associated with a user who classified the medical image, and wherein the method further comprises:

accessing information indicating users associated with the moved medical images; and

generating respective notifications for presentation to the indicated users identifying the re-classification.

5. The method of claim 1 , wherein, the machine learning system updates based on analysis of object characteristics of the re-classified one or more medical images to increase an accuracy associated with automated assignment of classifications.

6. A method for facilitating proper classification of features in medical images comprising:

generating, for display on a display of a computing system, an interactive classification user interface concurrently displaying two groups of medical images, each group having been assigned a disparate classification relating to objects included in the medical images, and each object having been extracted from a larger medical image;

the interactive classification user interface enabling a user to adjust a classification of each medical image to correspond to a different group, such that the medical image is re-classified according to the different classification; and

providing the re-classifications to a machine learning system, the machine learning system updating based on the re-classifications to increase an accuracy associated with assigning classifications.

7. The method of claim 6 , wherein borders of the objects included in the medical images are highlighted, the highlighting being automatically generated by the machine learning system.

8. The method of claim 6 , wherein the interactive classification user interface displays each group of medical images as a respective grid.

9. The method of claim 6 , wherein the interactive classification user interface displays each group of medical images as a respective stack.

10. The method of claim 6 , wherein the interactive classification user interface receives input from the user on a touch-screen display, the input representing the user dragging a first image included in a first group to a second group, such that the first image is re-classified.

11. The method of claim 6 , wherein the interactive classification user interface is configured to receive input via a touch-sensitive display, and wherein received input directed to a particular medical image causes the interactive classification user interface to update to present a full image context related to the particular medical image.

12. The method of claim 6 , wherein a re-classification assigned by the user is stored, and wherein an initial user who classified the associated medical image is alerted.

13. The method of claim 6 , wherein a threshold number of users assign classifications to the medical images, and wherein a final classification is determined for each medical image prior to providing the classifications to the machine learning system.

14. The method of claim 6 , wherein the classifications relate to a diagnosis associated with an object, a shape of an object, or a change in size or character of an object over time.

15. Non-transitory computer storage-media storing instructions that when executed by a system of one or more computers, cause the one or more computers to perform operations comprising:

providing a visual concurrent display of two groups of medical images, each group being associated with a respective classification, wherein the classification relates to objects included in the medical images;

providing a user interface enabling a reviewing user to re-classify medical images, such that one or more medical images are assigned to a different group;

responsive to the reviewing user utilizing the user interface to indicate that a particular medical image is to be re-classified, updating the particular image to be assigned to the different group and causing storage of the update.

16. The computer storage-media of claim 15 , wherein the operations further comprise:

training a machine learning system based on the re-classification.

17. The computer storage-media of claim 15 , wherein each group is presented as a respective grid of medical images.

18. The computer storage-media of claim 15 , wherein each group is presented as a respective stack of medical images.

19. The computer storage-media of claim 15 , wherein an initial reviewing user who classified the particular medical image is alerted.

20. The computer storage-media of claim 15 , wherein a notification is provided to a third reviewing user, the notification enabling activation of an application on a user device of the third reviewing user to determine a final classification for the particular medical image.

Assignments (4)
SECURITY INTEREST Recorded Oct 1, 2025
From: MERATIVE US L.P.; MERGE HEALTHCARE INCORPORATED
To: TCG SENIOR FUNDING L.L.C., AS COLLATERAL AGENT
Reel/Frame 072808/0442 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2022
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: MERATIVE US L.P.
Reel/Frame 061496/0752 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 11, 2018
From: HAZAN, ALON
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 044603/0366 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2017
From: REICHER, MURRAY A.; ZLOTNICK, AVIAD
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 044321/0713 →
Continuity (1)
Related Publication 20190171914A1 · Jun 6, 2019