IP Library Granted Patent US 12,051,497
Granted Patent B2
US 12,051,497 · App. 16/708,803 · Granted Jul 30, 2024

Systems and methods for validating and correcting automated medical image annotations

Inventors: Leo J. Grady (Millbrae, CA); Romain Moreau-Gobard (Redwood City, CA); Michiel Schaap (Mountain View, CA)
Assignee: HeartFlow, Inc.
G16H30/40A61B6/032A61B6/468G06F3/04845G06F40/169G06T7/0012G06T17/00G06T19/00G06T19/003G06V40/103G16H30/20G06T2207/30048G06T2207/30101G06T2207/30196G06T2219/004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,051,497
App. No.
16/708,803
Granted
Jul 30, 2024
Kind
B2
Abstract

Systems and methods are disclosed for manipulating image annotations. One method includes receiving an image of an individual's anatomy; automatically determining, using a processor, one or more annotations for anatomical features identified in the image of the individual's anatomy; determining a dependency or hierarchy between at least two of the one or more annotations for anatomical features identified in the image of the individual's anatomy; and generating, based on the dependency or hierarchy, a workflow prompting a user to manipulate the one or more annotations for anatomical features identified in the image of the individual's anatomy.

Claims (52)

1. A computer-implemented method of manipulating medical image annotations, the method comprising:

receiving at least one image of an individual's anatomy;

automatically determining, via a processor, a plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy;

determining a dependency or hierarchy between at least a first annotation of a first anatomical feature and a second annotation of a second anatomical feature of the plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy, wherein the dependency or hierarchy defines that the second annotation of the second anatomical feature is based on a prior identification of, or modification to, the first annotation of the first anatomical feature;

determining, via the processor, a red flag task, based on the determined dependency or hierarchy, associated with the at least one image of the individual's anatomy, the red flag task indicating a need for validation by a user of a user interface element; and

automatically presenting, in a multi-planar reconstructed view, the plurality of annotations to the user via the user interface element in an order based on the determined dependency or hierarchy between at least the first annotation of the first anatomical feature and the second annotation of the second anatomical feature of the plurality of annotations, wherein the first annotation of the first anatomical feature is presented to the user before the second annotation of the second anatomical feature is presented to the user, as a result of the determination of the dependency or hierarchy that defines that the second annotation of the second anatomical feature is based on the prior identification of, or modification to, the first annotation of the first anatomical feature.

2. The method of claim 1 , wherein the red flag task enables the user to assess the at least one image of the individual's anatomy.

3. The method of claim 1 , wherein the red flag task enables the user to determine an adequacy of the at least one image to be processed into a patient-specific 3D model.

4. The method of claim 3 , wherein the patient-specific 3D model illustrates a lumen or myocardium.

5. The method of claim 1 , wherein the red flag task enables the user to identify one or more anatomical anomalies in the at least one image.

6. The method of claim 1 , further comprising obtaining, via the processor, one or more user interactions with the at least one image.

7. The method of claim 6 , further comprising generating, via the processor, a model of the individual's anatomy based on the one or more user interactions with the at least one image.

8. The method of claim 1 , further comprising creating one or more visualizations for each of the plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy.

9. The method of claim 8 , wherein the one or more visualizations enables the user to view and validate the plurality of annotations.

10. The method of claim 1 , further comprising receiving an input from a user related to the first annotation of the first anatomical feature before presenting the second annotation of the second anatomical feature to the user.

11. The method of claim 1 , further comprising determining that at least one of the plurality of annotations is a critical annotation and representing the critical annotation to the user for a second time after presenting the critical annotation to the user for a first time.

12. A system for manipulating medical image annotations, the system comprising:

at least one data storage device storing instructions for manipulating medical image annotations; and

at least one processor configured to execute the instructions to perform a method including:

receiving at least one image of an individual's anatomy;

automatically determining, via a processor, a plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy;

determining a dependency or hierarchy between at least a first annotation of a first anatomical feature and a second annotation of a second anatomical feature of the plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy, wherein the dependency or hierarchy defines that the second annotation of the second anatomical feature is based on a prior identification of, or modification to, the first annotation of the first anatomical feature;

determining, via the processor, a red flag task, based on the determined dependency or hierarchy, associated with the at least one image of the individual's anatomy, the red flag task indicating a need for validation by a user of a user interface element; and

automatically presenting, in a multi-planar reconstructed view, the plurality of annotations to the user via the user interface element in an order based on the determined dependency or hierarchy between at least the first annotation of the first anatomical feature and the second annotation of the second anatomical feature of the plurality of annotations, wherein the first annotation of the first anatomical feature is presented to the user before the second annotation of the second anatomical feature is presented to the user, as a result of the determination of the dependency or hierarchy that defines that the second annotation of the second anatomical feature is based on the prior identification of, or modification to, the first annotation of the first anatomical feature.

13. The system of claim 12 , wherein the red flag task enables the user to assess the at least one image of the individual's anatomy.

14. The system of claim 12 , wherein the red flag task enables the user to determine an adequacy of the at least one image to be processed into a patient-specific 3D model.

15. The system of claim 12 , wherein the red flag task enables the user to identify one or more anatomical anomalies in the at least one image.

16. A non-transitory computer readable medium for use on a computer system containing computer-executable programming instructions for performing a method of manipulating medical image annotations, the method comprising:

receiving at least one image of an individual's anatomy;

automatically determining, via a processor, plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy;

determining a dependency or hierarchy between at least a first annotation of a first anatomical feature and a second annotation of a second anatomical feature of the plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy, wherein the dependency or hierarchy defines that the second annotation of the second anatomical feature is based on a prior identification of, or modification to, the first annotation of the first anatomical feature;

determining, via the processor, a red flag task, based on the determined dependency or hierarchy, associated with the at least one image of the individual's anatomy, the red flag task indicating a need for validation by a user of a user interface element; and

automatically presenting, in a multi-planar reconstructed view, the plurality of annotations to the user via the user interface element in an order based on the determined dependency or hierarchy between at least the first annotation of the first anatomical feature and the second annotation of the second anatomical feature of the plurality of annotations, wherein the first annotation of the first anatomical feature is presented to the user before the second annotation of the second anatomical feature is presented to the user, as a result of the determination of the dependency or hierarchy that defines that the second annotation of the second anatomical feature is based on the prior identification of, or modification to, the first annotation of the first anatomical feature.

17. The non-transitory computer readable medium of claim 16 , wherein the red flag task enables the user to assess the at least one image of the individual's anatomy.

18. The non-transitory computer readable medium of claim 16 , wherein the red flag task enables the user to determine an adequacy of the at least one image to be processed into a patient-specific 3D model.

19. The non-transitory computer readable medium of claim 16 , wherein the red flag task enables the user to identify one or more anatomical anomalies in the at least one image.

20. A computer-implemented method of manipulating medical image annotations, the method comprising:

receiving at least one image of an individual's anatomy;

automatically determining, via a processor, a plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy;

determining a dependency or hierarchy between at least a first annotation of a first anatomical feature and a second annotation of a second anatomical feature of the plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy, wherein the dependency or hierarchy defines that the second annotation of the second anatomical feature is based on a prior identification of, or modification to, the first annotation of the first anatomical feature;

determining, via the processor, a red flag task, based on the determined dependency or hierarchy, associated with the at least one image of the individual's anatomy, the red flag task indicating a need for validation by a user of a user interface element;

determining that at least one of the plurality of annotations is a critical annotation and representing the critical annotation to the user for a second time after presenting the critical annotation to the user for a first time; and

automatically presenting, in a multi-planar reconstructed view, the plurality of annotations to the user via the user interface element in an order based on the determined dependency or hierarchy between at least the first annotation of the first anatomical feature and the second annotation of the second anatomical feature of the plurality of annotations.

21. A system for manipulating medical image annotations, the system comprising:

at least one data storage device storing instructions for manipulating medical image annotations; and

at least one processor configured to execute the instructions to perform a method including:

receiving at least one image of an individual's anatomy;

automatically determining, via a processor, a plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy;

determining a dependency or hierarchy between at least a first annotation of a first anatomical feature and a second annotation of a second anatomical feature of the plurality of annotations associated with anatomical features identified in the at least one image of the individual's anatomy, wherein the dependency or hierarchy defines that the second annotation of the second anatomical feature is based on a prior identification of, or modification to, the first annotation of the first anatomical feature;

determining, via the processor, a red flag task, based on the determined dependency or hierarchy, associated with the at least one image of the individual's anatomy, the red flag task indicating a need for validation by a user of a user interface element;

determining that at least one of the plurality of annotations is a critical annotation and representing the critical annotation to the user for a second time after presenting the critical annotation to the user for a first time; and

automatically presenting, in a multi-planar reconstructed view, the plurality of annotations to the user via the user interface element in an order based on the determined dependency or hierarchy between at least the first annotation of the first anatomical feature and the second annotation of the second anatomical feature of the plurality of annotations.

Assignments (5)
RELEASE OF SECURITY INTEREST Recorded Sep 11, 2025
From: HAYFIN SERVICES LLP
To: HEARTFLOW, INC.
Reel/Frame 072876/0775 →
RELEASE OF SECURITY INTEREST Recorded Jun 21, 2024
From: HAYFIN SERVICES LLP
To: HEARTFLOW, INC.
Reel/Frame 067801/0032 →
SECURITY INTEREST Recorded Jun 18, 2024
From: HEARTFLOW, INC.
To: HAYFIN SERVICES LLP
Reel/Frame 067775/0966 →
SECURITY INTEREST Recorded Jan 20, 2021
From: HEARTFLOW, INC.
To: HAYFIN SERVICES LLP
Reel/Frame 055037/0890 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 15, 2020
From: GRADY, LEO; MOREAU-GOBARD, ROMAIN; SCHAAP, MICHIEL
To: HEARTFLOW, INC.
Reel/Frame 051523/0351 →