IP Library Granted Patent US 11,164,309
Granted Patent B2
US 11,164,309 · App. 16/379,839 · Granted Nov 2, 2021

Image analysis and annotation

Inventors: Marwan Sati (Mississauga, CA); David Richmond (Newton, MA)
Assignee: International Business Machines Corporation
G06T7/0012G06K9/4671G16H30/40G16H50/20G16H70/60G06K2209/05G06T2207/20072G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,164,309
App. No.
16/379,839
Granted
Nov 2, 2021
Kind
B2
Abstract

An embodiment of the invention may include a method, computer program product and computer system for object detection and identification. The method, computer program product and computer system may include computing device which may receive an image from an imaging device. The image may be a medical image. The computing device may detect one or more potential indicators of disease in the image using a first algorithm and determine areas of potential disease in the image using an artificial intelligence algorithm. The computing device may determine a correlation between the determined areas of potential disease in the image and the one or more potential indicators of disease for the image. The computing device may, in response to determining a positive correlation, identify one or more of the potential indicators of disease for annotation and generate a report indicating one or more potential indicators of disease was found in the image.

Claims (55)

1. A method for improving image analysis and annotation, the method comprising:

receiving, by a computing device, an image from an imaging device, wherein the image is a medical image;

detecting, by the computing device, one or more potential indicators of disease in the image using a Computer Aided Detection algorithm;

determining, by the computing device, a disease probability map having areas of potential disease in the image using an artificial intelligence algorithm, wherein the artificial intelligence algorithm is a deep learning algorithm that detects one or more salients in the image and generates the disease probability map;

determining, by the computing device, a positive correlation between the determined areas of potential disease in the image and the one or more potential indicators of disease for the image, wherein the positive correlation is determined when the one or more potential indicators of the disease is within the disease probability map;

reducing false-positives by generating a heat map based on the areas of potential disease in the image;

identifying one or more findings by cross-correlating the heat map and the one or more potential indicators of disease in the image using the Computer Aided Detection algorithm; and

generating, using a natural language processing, a report in a natural language that includes recommendations for the one or more findings based on the disease probability map.

2. The method as in claim 1 , further comprising:

in response to determining the positive correlation, identifying, by the computing device, one or more of the potential indicators of disease for annotation; and

generating, by the computing device, the report indicating one or more potential indicators of disease was found in the image.

3. The method as in claim 1 , further comprising:

in response to determining a negative, generating, by the computing device, the report indicating no potential indicators of disease were found in the image.

4. The method as in claim 1 , further comprising:

determining, by the computing device, a discrepancy between the areas of potential disease and the one or more potential indicators of disease; and

generating, by the computing device, the report indicating further review of the image is required.

5. The method as in claim 1 , wherein the one or more potential indicators of disease in the image are further detected using a computer aided detection algorithm, the computer aided detection algorithm generating markings for the one or more potential indicators of disease on the image.

6. The method as in claim 1 , wherein the areas of potential disease in the image are determined using a saliency detection algorithm.

7. A computer program product for improving image analysis and annotation, the computer program product comprising:

a computer-readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions comprising:

program instructions to receive, by a computing device, an image from an imaging device, wherein the image is a medical image;

program instructions to detect, by the computing device, one or more potential indicators of disease in the image using a Computer Aided Detection algorithm;

program instructions to determine, by the computing device, a disease probability map having areas of potential disease in the image using an artificial intelligence algorithm, wherein the artificial intelligence algorithm is a deep learning algorithm that detects one or more salients in the image and generates the disease probability map;

program instructions to determine, by the computing device, a positive correlation between the determined areas of potential disease in the image and the one or more potential indicators of disease for the image, wherein the positive correlation is determined when the one or more potential indicators of the disease is within the disease probability map;

program instructions to reduce false-positives by generating a heat map based on the areas of potential disease in the image;

program instructions to identify one or more findings by cross-correlating the heat map and the one or more potential indicators of disease in the image using the Computer Aided Detection algorithm; and

program instructions to generate, using a natural language processing, a report in a natural language that includes recommendations for the one or more findings based on the disease probability map.

8. The computer program product as in claim 7 , further comprising:

in response to determining the positive correlation, program instructions to identify, by the computing device, one or more of the potential indicators of disease for annotation; and

program instructions to generate, by the computing device, the report indicating one or more potential indicators of disease was found in the image.

9. The computer program product as in claim 7 , further comprising:

in response to determining a negative, program instructions to generate, by the computing device, the report indicating no potential indicators of disease were found in the image.

10. The computer program product as in claim 7 , further comprising:

program instructions to determine, by the computing device, a discrepancy between the areas of potential disease and the one or more potential indicators of disease; and

program instructions to generate, by the computing device, the report indicating further review of the image is required.

11. The computer program product as in claim 7 , wherein the one or more potential indicators of disease in the image are detected using a computer aided detection algorithm, the computer aided detection algorithm generating markings for the one or more potential indicators of disease on the image.

12. The computer program product as in claim 7 , wherein the areas of potential disease in the image are determined using a saliency detection algorithm.

13. A computer system for improving image analysis and annotation, the system comprising:

one or more computer processors, one or more computer-readable storage media, and program instructions stored on one or more of the computer-readable storage media for execution by at least one of the one or more processors, the program instructions comprising:

program instructions to receive, by a computing device, an image from an imaging device, wherein the image is a medical image;

program instructions to detect, by the computing device, one or more potential indicators of disease in the image using a Computer Aided Detection algorithm;

program instructions to determine, by the computing device, a disease probability map having areas of potential disease in the image using an artificial intelligence algorithm, wherein the artificial intelligence algorithm is a deep learning algorithm that detects one or more salients in the image and generates the disease probability map;

program instructions to determine, by the computing device, a positive correlation between the determined areas of potential disease in the image and the one or more potential indicators of disease for the image, wherein the positive correlation is determined when the one or more potential indicators of the disease is within the disease probability map;

program instructions to reduce false-positives by generating a heat map based on the areas of potential disease in the image;

program instructions to identify one or more findings by cross-correlating the heat map and the one or more potential indicators of disease in the image using the Computer Aided Detection algorithm; and

program instructions to generate, using a natural language processing, a report in a natural language that includes recommendations for the one or more findings based on the disease probability map.

14. The system as in claim 13 , further comprising:

in response to determining the positive correlation, program instructions to identify, by the computing device, one or more of the potential indicators of disease for annotation; and

program instructions to generate, by the computing device, the report indicating one or more potential indicators of disease was found in the image.

15. The system as in claim 13 , further comprising:

in response to determining a negative, program instructions to generate, by the computing device, the report indicating no potential indicators of disease were found in the image.

16. The system as in claim 13 , further comprising:

program instructions to determine, by the computing device, a discrepancy between the areas of potential disease and the one or more potential indicators of disease; and

program instructions to generate, by the computing device, the report indicating further review of the image is required.

17. The system as in claim 13 , wherein one or more potential indicators of disease in the image are detected using a computer aided detection algorithm, the computer aided detection algorithm generating markings for the one or more potential indicators of disease on the image.

Assignments (3)
SECURITY INTEREST Recorded Oct 1, 2025
From: MERATIVE US L.P.; MERGE HEALTHCARE INCORPORATED
To: TCG SENIOR FUNDING L.L.C., AS COLLATERAL AGENT
Reel/Frame 072808/0442 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2022
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: MERATIVE US L.P.
Reel/Frame 061496/0752 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 10, 2019
From: SATI, MARWAN; RICHMOND, DAVID
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 048841/0259 →