IP Library Granted Patent US 11,742,070
Granted Patent B2
US 11,742,070 · App. 16/725,649 · Granted Aug 29, 2023

System and method for controlling user repeatability and reproducibility of automated image annotation correction

Inventors: Leo Grady (Millbrae, CA); Michiel Schaap (Redwood City, CA)
Assignee: HeartFlow, Inc.
G16H30/40G06F3/04842G06F3/04845G06F16/5866G06T7/0012G06T7/97G06T11/60G16H30/20G06T2207/10004G06T2207/30004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,742,070
App. No.
16/725,649
Granted
Aug 29, 2023
Kind
B2
Abstract

Systems and methods are disclosed for controlling image annotation. One method includes acquiring a digital representation of image data and generating a set of image annotations for the digital representation of the image data. The method also may include determining an association between members of the set of image annotations and generating one or more groups of members based on the association. A representative annotation from the one or more groups may also be determined, presented for selection, and the selection may be recorded in memory.

Claims (51)

1. A computer-implemented method of controlling image annotation, using at least one computer system, the method comprising using a processor for:

receiving a digital representation of image data corresponding to anatomic features of a patient;

segmenting the digital representation to determine a first candidate coronary vessel tree by a method controlled by one or more parameters;

segmenting the digital representation to generate a plurality of alternative candidate coronary vessel trees corresponding to the first candidate coronary vessel tree:

generating a plurality of groups of alternative candidate coronary vessel trees, each group of the plurality of groups comprising a subset of the plurality of alternative candidate coronary vessel trees;

determining, by the at least one computer system, a representative alternative candidate coronary vessel tree for each of the groups of alternative candidate coronary vessel trees, the representative alternative candidate coronary vessel tree for each group being determined as either:

an alternative candidate coronary vessel tree of the group closest to an average alternative candidate coronary vessel tree of the group, or

a generated average coronary vessel tree of the group; and

presenting, by the at least one computer system, the first candidate coronary vessel tree and each representative alternative candidate coronary vessel tree for selection and/or correction by a user.

2. The computer-implemented method of claim 1 ,

wherein the plurality of alternative candidate coronary vessel trees are generated by one or more of randomly selecting settings of the one or more parameters, applying a plurality of image segmentation techniques, perturbing the digital representation by rigid transformation, or perturbing the digital representation by an addition of Gaussian noise to the digital representation.

3. The computer-implemented method of claim 1 , further comprising:

determining, by the at least one computer system, one or more associations between alternative candidate coronary vessel trees of the plurality of alternative candidate coronary vessel trees,

wherein the plurality of groups of alternative candidate coronary vessel trees are generated based on the determined one or more associations.

4. The computer-implemented method of claim 1 , wherein the digital representation of image data is acquired electronically via a network.

5. The computer-implemented method of claim 1 , wherein each representative coronary vessel tree identifies anatomic features identified within the digital representation of image data.

6. The computer-implemented method of claim 1 , wherein the generated first candidate coronary vessel tree is generated using one or more of a level set, a graph based method, an active shape model, an active appearance model.

7. The computer-implemented method of claim 3 , wherein each of the one or more associations between alternative candidate coronary vessel trees of the plurality of alternative candidate coronary vessel trees is determined by computing a dice coefficient.

8. The computer-implemented method of claim 1 , wherein generating the plurality of groups of alternative candidate coronary vessel trees includes generating the plurality of groups of alternative candidate coronary vessel trees via a cluster algorithm.

9. The computer-implemented method of claim 8 , wherein the cluster algorithm includes spectral partitioning and isoperimetric portioning.

10. The computer-implemented method of claim 1 , wherein the image data is a portion of a body organ.

11. A system of controlling image annotation, the system comprising:

a data storage device storing instructions for controlling image segmentation; and

a processor configured to execute the instruction to perform a method including:

receiving a digital representation of image data corresponding to anatomic features of a patient;

segmenting the digital representation to determine a first candidate coronary vessel tree by a method controlled by one or more parameters;

segmenting the digital representation to generate a plurality of alternative candidate coronary vessel trees corresponding to the first candidate coronary vessel tree:

generating a plurality of groups of alternative candidate coronary vessel trees, each group of the plurality of groups comprising a subset of the plurality of alternative candidate coronary vessel trees;

determining a representative alternative candidate coronary vessel tree for each of the groups of alternative candidate coronary vessel trees, the representative alternative candidate coronary vessel tree for each group being determined as either:

an alternative candidate coronary vessel tree of the group closest to an average alternative candidate coronary vessel tree of the group, or

a generated average coronary vessel tree of the group; and

presenting the first candidate coronary vessel tree and each representative alternative candidate coronary vessel tree for selection or correction by a user.

12. The system of claim 11 , wherein the plurality of alternative candidate coronary vessel trees are generated by one or more of randomly selecting settings of the one or more parameters, applying a plurality of image segmentation techniques, perturbing the digital representation by rigid transformation, or perturbing the digital representation by an addition of Gaussian noise to the digital representation.

13. The system of claim 11 , wherein the processor is further configured to electronically acquire the digital representation of image data via a network.

14. The system of claim 11 , wherein the processor is further configured to determine one or more associations between alternative candidate coronary vessel trees of the plurality of alternative candidate coronary vessel trees,

wherein the plurality of groups of alternative candidate coronary vessel trees are generated based on the determined one or more associations.

15. The system of claim 11 , wherein the processor is further configured to automatically generate the first candidate coronary vessel tree using one or more of a level set, a graph based method, an active shape model, an active appearance model.

16. The system of claim 14 , wherein the processor is configured to determine each of the one or more associations between alternative candidate coronary vessel trees of the plurality of alternative candidate of coronary vessel trees by computing a dice coefficient.

17. The system of claim 14 , wherein processor is configured to generate the plurality of groups of alternative candidate coronary vessels trees via a cluster algorithm.

18. The system of claim 17 , wherein the cluster algorithm includes spectral partitioning and isoperimetric portioning.

19. A non-transitory computer-readable medium for use on a computer system containing computer-executable programming instructions for controlling image annotation, the instructions being executable by the computer system for:

receiving a digital representation of image data corresponding to anatomic features of a patient;

segmenting the digital representation to determine a first candidate coronary vessel tree by a method controlled by one or more parameters;

segmenting the digital representation to generate a plurality of alternative candidate coronary vessel trees corresponding to the first candidate coronary vessel tree:

generating a plurality of groups of alternative candidate coronary vessel trees, each group of the plurality of groups comprising a subset of the plurality of alternative candidate coronary vessel trees;

determining, by the computer system, a representative alternative candidate coronary vessel tree for each of the groups of alternative candidate coronary vessel trees, the representative alternative candidate coronary vessel tree for each group being determined as either:

an alternative candidate coronary vessel tree of the group closest to an average alternative candidate coronary vessel tree of the group, or

a generated average coronary vessel tree of the group; and

presenting, by the computer system, the first candidate coronary vessel tree and each representative alternative candidate coronary vessel tree for selection or correction by a user.

20. The non-transitory computer readable medium of claim 19 , the instructions being executable by the computer system further comprising determining one or more associations between alternative candidate coronary vessel trees of the plurality of alternative candidate coronary vessel trees,

wherein the plurality of groups of alternative candidate coronary vessel trees are generated based on the determined one or more associations.

Assignments (5)
RELEASE OF SECURITY INTEREST Recorded Sep 11, 2025
From: HAYFIN SERVICES LLP
To: HEARTFLOW, INC.
Reel/Frame 072876/0775 →
RELEASE OF SECURITY INTEREST Recorded Jun 21, 2024
From: HAYFIN SERVICES LLP
To: HEARTFLOW, INC.
Reel/Frame 067801/0032 →
SECURITY INTEREST Recorded Jun 18, 2024
From: HEARTFLOW, INC.
To: HAYFIN SERVICES LLP
Reel/Frame 067775/0966 →
SECURITY INTEREST Recorded Jan 20, 2021
From: HEARTFLOW, INC.
To: HAYFIN SERVICES LLP
Reel/Frame 055037/0890 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 3, 2020
From: GRADY, LEO; SCHAAP, MICHIEL
To: HEARTFLOW, INC.
Reel/Frame 051407/0878 →
Continuity (5)
Continuation 15834488 · Dec 7, 2017
Continuation 15415264 · Jan 25, 2017
Continuation 14470314 · Aug 27, 2014
Provisional Application 61882512 · Sep 25, 2013
Related Publication 20200134897A1 · Apr 30, 2020
Cited By (18)
US 12,245,882 US 12,283,046 US 12,299,885 US 12,324,695 US 12,324,696 US 12,380,560 US 12,396,695 US 12,406,365 US 12,440,180 US 12,499,539 US 12,555,228 US 12,558,048 US 12,599,352 US 12,620,092 US 12,635,965 US 12,669,911 US 12,712,082 US 12,714,382