IP Library Granted Patent US 11,205,515
Granted Patent B2
US 11,205,515 · App. 15/163,316 · Granted Dec 21, 2021

Annotation and assessment of images

Inventor: Murray A. Reicher (Rancho Santa Fe, CA)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G16H50/20G01R33/4812G06T7/0016G06T11/60G16H15/00G16H30/40G16H40/63G16H50/50G06T2207/20104G06T2207/30096G09G2380/08G16H20/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,205,515
App. No.
15/163,316
Granted
Dec 21, 2021
Kind
B2
Abstract

Annotations of medical images may be generated using one or more lexicons so that terminology is consistent across multiple exams, users, facilities, etc. Measurements of lesions may be provided using a bilinear measurement tool that allows easier bilinear measurements. Disease assessment models may be selected and applied as measurements are acquired in order to provide immediate determination of disease staging according to one or more selected assessment models.

Claims (76)

1. A computing system comprising:

one or more hardware processors configured to execute software instructions to cause the computing system to:

automatically determine an assessment model from a plurality of available assessment models stored in computer memory to apply in determining a disease stage of a patient, wherein the assessment model is associated with one or more sets of assessment rules and, the plurality of available assessment models including the RECIST model and an additional assessment model different from the RECIST model;

select a first set of assessment rules from the one or more sets of assessment rules based on characteristics of at least one of a disease of the patient and the patient determined based on DICOM header information of a least one of a plurality of previous exams associated with the patient;

access, from the plurality of previous exams associated with the patient, previous lesion measurement data that satisfies the first set of assessment rules;

determine, according to the first set of assessment rules, a baseline assessment score based on the previous lesion measurement data;

determine, according to the first set of assessment rules, a current assessment score based on current lesion measurement data from a latest exam associated with the patient, wherein the latest exam associated with the patient was obtained after the plurality of previous exams associated with the patient, the current lesion measurement data generated using a software-implemented measurement tool movable on at least one image included in the latest exam associated with the patient to select at least one location within the at least one image for automatically determining the measurement data based on pixels included in the at least one image;

compare the baseline assessment score with the current assessment score in accordance with the determined assessment model;

determine and output a disease stage based on the comparison of the baseline assessment score with the current assessment score in accordance with the determined assessment model and the first set of assessment rules; and

in response to determining that at least one of a longest diameter or a short axis of a first lesion measurement of the previous lesion measurement data does not satisfy the first set of assessment rules, provide an audible or visual indication to a user, wherein the audible or visual indication comprises setting a color of at least a portion of the measurement tool displayed on a user interface to a first color, wherein the at least a portion of the measurement tool that is set to the first color comprises at least one of a first line segment or a second line segment that do not meet the first set of assessment rules.

2. The computing system of claim 1 , wherein the one or more sets of assessment rules of the assessment model comprise at least the first set of assessment rules for use with a first category of lesions and a second set of assessment rules for use with a second category of lesions.

3. The computing system of claim 2 , wherein the first category of lesions comprises solid tumors.

4. The computing system of claim 3 , wherein the second category of lesions comprises lymph nodes.

5. The computing system of claim 1 , wherein the first set of assessment rules specifies at least one of a minimum longest diameter or a minimum short axis for a lesion measurement to be used by the assessment model.

6. The computing system of claim 1 , wherein determining the baseline assessment score comprises:

determining, from the previous lesion measurement data, a plurality of measurements of one or more lesions of a first previous exam of the plurality of previous exams;

for two or more of the plurality of measurements, determining respective longest diameters of the measurements;

summing up all the longest diameters of the measurements to determine a total of longest diameters of lesions in the first previous exam; and

using the total of the longest diameters in the first previous exam as the baseline assessment score.

7. The computing system of claim 6 , wherein determining the current assessment score comprises:

determining, from the current lesion measurement data, a second plurality of measurements of the one or more lesions of the latest exam;

for two or more of the second plurality of measurements, determining respective longest diameters of the second measurements;

summing up all of the longest diameters of the second measurements to determine a total of longest diameters of lesions in the latest exam; and

using the total of the longest diameters in the latest exam as the current assessment score.

8. The computing system of claim 1 , wherein determining the baseline assessment score comprises:

determining, from the previous lesions measurement data, a plurality of measurements of one or more lesions of a first previous exam of the plurality of previous exams;

for two or more of the plurality of measurements, determining respective short axis lengths of the measurements;

summing up all the short axis lengths of the measurements to determine a total of short axis lengths of lesions in the first previous exam; and

using the total of the short axis lengths in the first previous exam as the baseline assessment score.

9. The computing system of claim 8 , wherein determining the current assessment score comprises:

determining, from the current lesion measurement data, a second plurality of measurements of the one or more lesions of the latest exam;

for two or more of the second plurality of measurements, determining respective short axis lengths of the second measurements;

summing up all the short axis lengths of the second measurements to determine a total of short axis lengths of lesions in the latest exam; and

using the total of the short axis lengths in the latest exam as the current assessment score.

10. The computing system of claim 1 , wherein the one or more hardware processors are configured to execute software instructions to further cause the computing system to:

in response to an input from the user, override the first set of assessment rules to cause the computing system to use the first lesion measurement in determining the disease stage.

11. A computer-implemented method comprising:

by one or more processors executing program instructions:

automatically determining an assessment model from a plurality of available assessment models stored in computer memory to apply in determining a disease stage of a patient, the plurality of available assessment models including the RECIST model and the Gottlieb model;

selecting a first set of assessment rules from the one or more sets of assessment rules based on characteristics of at least one of a disease of the patient and the patient determined based on DICOM header information of at least one of a plurality of previous exams associated with the patient;

accessing, from the plurality of previous exams associated with the patient, previous lesion measurement data that satisfies the first set of assessment rules;

determining, according to the first set of assessment rules, a baseline assessment score based on the previous lesion measurement data;

determining, according to the first set of assessment rules, a current assessment score based on current lesion measurement data from a latest exam associated with the patient, wherein the latest exam associated with the patient was obtained after the plurality of previous exams associated with the patient, the current lesion measurement data generated using a software-implemented measurement tool movable on at least one image included in the latest exam associated with the patient to select at least one location within the at least one image for automatically determining the measurement data based on pixels included in the at least one image;

comparing the baseline assessment score with the current assessment score in accordance with the determined assessment model;

determining and outputting a disease stage based on the comparison of the baseline assessment score with the current assessment score in accordance with the determined assessment model and the first set of assessment rules; and

in response to determining that at least one of a longest diameter or a short axis of a first lesion measurement of the previous lesion measurement data does not satisfy the first set of assessment rules, provide an audible or visual indication to a user, wherein the audible or visual indication comprises setting a color of at least a portion of the measurement tool displayed on a user interface to a first color, wherein the at least a portion of the measurement tool that is set to the first color comprises at least one of a first line segment or a second line segment that do not meet the first set of assessment rules.

12. The computer-implemented method of claim 11 , wherein the first set of assessment rules specifies at least one of a minimum longest diameter or a minimum short axis for a lesion measurement to be used by the assessment model.

13. The computer-implemented method of claim 11 , wherein determining the baseline assessment score comprises:

determining, from the previous lesion measurement data, a plurality of measurements of one or more lesions of a first previous exam of the plurality of previous exams;

for two or more of the plurality of measurements, determining respective longest diameters of the measurements;

summing up all the longest diameters of the measurements to determine a total of longest diameters of lesions in the first previous exam; and

using the total of the longest diameters in the first previous exam as the baseline assessment score.

14. A computer-implemented method of claim 13 , wherein determining the current assessment score comprises:

determining, from the current lesion measurement data, a second plurality of measurements of the one or more lesions of the latest exam;

for two or more of the second plurality of measurements, determining respective longest diameters of the second measurements;

summing up all the longest diameters of the second measurements to determine a total of longest diameters of lesions in the latest exam; and

using the total of the longest diameters in the latest exam as the current assessment score.

15. The computer-implemented method of claim 11 , wherein determining the baseline assessment score comprises:

determining, from the previous lesion measurement data, a plurality of measurements of one or more lesions of a first previous exam of the plurality of previous exams;

for two or more of the plurality of measurements, determining respective short axis lengths of the measurements;

summing up all the short axis lengths of the measurements to determine a total of short axis lengths of lesions in the first previous exam; and

using the total of the short axis lengths in the first previous exam as the baseline assessment score.

16. The computer-implemented method of claim 15 , wherein determining the current assessment score comprises:

determining, from the current lesion measurement data, a second plurality of measurements of the one or more lesions of the latest exam;

for two or more of the second plurality of measurements, determining respective short axis lengths of the second measurements;

summing up all the short axis lengths of the second measurements to determine a total of short axis lengths of lesions in the latest exam; and

using the total of the short axis lengths in the latest exam as the current assessment score.

17. A system for determining a disease state of a lesion, the system comprising:

one or more hardware processors configured to execute software instructions to cause the system to:

receive, from a user, a selection of a plurality of assessment models to be applied to lesion measurements of a patient from a plurality of available assessment models stored in computer memory;

retrieving one or more assessment rules associated with each of the plurality of assessment models from the computer memory;

automatically receive an update to the one or more assessment rules of one of the plurality of assessment models via a push or a pull architecture;

apply the update to the one or more assessment rules;

automatically apply each of the plurality of assessment models to one or more of the lesion measurements of the patient to provide disease staging according to each of the plurality of assessment models;

provide the determined disease staging to the user via a user interface; and

in response to determining that at least one of the one or more lesion measurements does not satisfy at least one of the one or more assessment rules, provide an audible or visual indication to a user, wherein the audible or visual indication comprises setting a color of at least a portion of a software-implemented measurement tool, movable on at least one image displayed on a user interface, to a first color, wherein the at least a portion of the measurement tool that is set to the first color comprises at least one of a first line segment or a second line segment that do not meet the first set of assessment rules.

Assignments (5)
SECURITY INTEREST Recorded Oct 1, 2025
From: MERATIVE US L.P.; MERGE HEALTHCARE INCORPORATED
To: TCG SENIOR FUNDING L.L.C., AS COLLATERAL AGENT
Reel/Frame 072808/0442 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2022
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: MERATIVE US L.P.
Reel/Frame 061496/0752 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 16, 2021
From: MERGE HEALTHCARE SOLUTIONS INC.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 055617/0985 →
NUNC PRO TUNC ASSIGNMENT Recorded Mar 7, 2019
From: D.R. SYSTEMS, INC.
To: MERGE HEALTHCARE SOLUTIONS INC.
Reel/Frame 048631/0123 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 22, 2016
From: REICHER, MURRAY A.
To: D.R. SYSTEMS, INC.
Reel/Frame 039500/0925 →
Continuity (3)
Division 13300239 · Nov 18, 2011
Provisional Application 61415679 · Nov 19, 2010
Related Publication 20170046498A1 · Feb 16, 2017