IP Library Granted Patent US 10,803,592
Granted Patent B2
US 10,803,592 · App. 15/975,197 · Granted Oct 13, 2020

Systems and methods for anatomic structure segmentation in image analysis

Inventors: Leo Grady (Millbrae, CA); Peter Kersten Petersen (Palo Alto, CA); Michiel Schaap (Redwood City, CA); David Lesage (Redwood City, CA)
Assignee: Heartflow, Inc.
G06T7/174G06T7/0012G06T7/12G06T7/149G06K2209/05G06T2200/04G06T2207/10072G06T2207/10081G06T2207/10088G06T2207/10104G06T2207/10108G06T2207/10132G06T2207/20081G06T2207/20112G06T2207/30004G06T2207/30101
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,803,592
App. No.
15/975,197
Granted
Oct 13, 2020
Kind
B2
Abstract

Systems and methods are disclosed for anatomic structure segmentation in image analysis, using a computer system. One method includes: receiving an annotation and a plurality of keypoints for an anatomic structure in one or more images; computing distances from the plurality of keypoints to a boundary of the anatomic structure; training a model, using data in the one or more images and the computed distances, for predicting a boundary in the anatomic structure in an image of a patient's anatomy; receiving the image of the patient's anatomy including the anatomic structure; estimating a segmentation boundary in the anatomic structure in the image of the patient's anatomy; and predicting, using the trained model, a boundary location in the anatomic structure in the image of the patient's anatomy by generating a regression of distances from keypoints in the anatomic structure in the image of the patient's anatomy to the estimated boundary.

Claims (41)

1. A computer-implemented method of anatomic structure segmentation in image analysis, the method comprising:

receiving an annotation and a plurality of keypoints for an anatomic structure in one or more images;

defining a mapping of image coordinates in the one or more images to a Euclidian space, where the image coordinates are in rays in the Euclidean space and each ray includes one of the keypoints;

computing, based on the mapping, distances from the plurality of keypoints to a boundary of the anatomic structure, wherein, the computing of the distances includes computing distances from the respective keypoint on each of the rays to the boundary of the anatomic structure;

training a model, using data in the one or more images and the computed distances, for predicting a boundary in the anatomic structure in an image of a patient's anatomy;

receiving the image of the patient's anatomy including the anatomic structure;

estimating a segmentation boundary in the anatomic structure in the image of the patient's anatomy; and

predicting, using the trained model, a boundary location in the anatomic structure in the image of the patient's anatomy by generating a regression of distances from keypoints in the anatomic structure in the image of the patient's anatomy to the estimated boundary.

2. The computer-implemented method of claim 1 , wherein the annotation for the anatomic structure is in a format of a mesh, a voxel, an implicit surface representation, or a point cloud.

3. The computer-implemented method of claim 1 , wherein locations of the received keypoints in the anatomic structure are known.

4. The computer-implemented method of claim 1 , further comprising fitting shape model to the anatomic structure to determine the locations of the plurality of keypoints.

5. The computer-implemented method of claim 1 , wherein the image coordinates are continuous.

6. The computer-implemented method of claim 1 , prior to training the model, further comprising determining image intensities along rays associated with the computed distances.

7. The computer-implemented method of claim 1 , wherein predicting the boundary location comprises predicting an indirect representation of the distances from keypoints in the anatomic structure in the image of the patient's anatomy to the estimated boundary.

8. The computer-implemented method of claim 1 , wherein estimating the segmentation boundary in the anatomic structure in the image of the patient's anatomy comprises retrieving a set of keypoints in the anatomic structure.

9. The computer-implemented method of claim 1 , wherein the predicted boundary location in the anatomic structure in the image of the patient's anatomy is a sub-voxel accurate boundary location.

10. The computer-implemented method of claim 1 , further comprising constructing a 3-dimensional surface based at the predicted boundary location.

11. The computer-implemented method of claim 1 , further comprising outputting the predicted boundary location to an electronic storage medium.

12. The computer-implemented method of claim 1 , wherein the anatomic structure comprises a blood vessel and the patient's anatomy comprises a blood vessel of the patient's vasculature.

13. The computer-implemented method of claim 1 , wherein the annotation for the anatomic structure comprises a vessel lumen boundary, a vessel lumen central line, a vessel lumen surface, or a combination thereof.

14. The computer-implemented method of claim 1 , wherein the generated regression is a continuous value.

15. A system for anatomic structure segmentation in image analysis, the system comprising:

a data storage device storing instructions for anatomic structure segmentation; and

a processor configured to execute the instructions to perform a method comprising:

receiving an annotation and a plurality of keypoints for an anatomic structure in one or more images;

defining a mapping of image coordinates in the one or more images to a Euclidian space, where the image coordinates are in rays in the Euclidean space and each ray includes one of the keypoints;

computing, based on the mapping, distances from the plurality of keypoints to a boundary of the anatomic structure, wherein, the computing of the distances includes computing distances from the respective keypoint on each of the rays to the boundary of the anatomic structure;

training a model, using data in the one or more images and the computed distances, for predicting a boundary in the anatomic structure in an image of a patient's anatomy;

receiving the image of the patient's anatomy including the anatomic structure;

estimating a segmentation boundary in the anatomic structure in the image of the patient's anatomy; and

predicting, using the trained model, a boundary location in the anatomic structure in the image of the patient's anatomy by generating a regression of distances from keypoints in the anatomic structure in the image of the patient's anatomy to the estimated boundary.

16. The system of claim 15 , wherein the anatomic structure comprises a blood vessel and the patient's anatomy comprises a blood vessel of the patient's vasculature.

17. The system of claim 15 , wherein the predicted boundary location in the anatomic structure in the image of the patient's anatomy is a sub-voxel accurate boundary location.

18. A non-transitory computer readable medium for use on a computer system containing computer-executable programming instructions for anatomic structure segmentation in image analysis, the method comprising:

receiving an annotation and a plurality of keypoints for an anatomic structure in one or more images;

defining a mapping of image coordinates in the one or more images to a Euclidian space, where the image coordinates are in rays in the Euclidean space and each ray includes one of the keypoints;

computing, based on the mapping, distances from the plurality of keypoints to a boundary of the anatomic structure, wherein, the computing of the distances includes computing distances from the respective keypoint on each of the rays to the boundary of the anatomic structure;

training a model, using data in the one or more images and the computed distances, for predicting a boundary in the anatomic structure in an image of a patient's anatomy;

receiving the image of the patient's anatomy including the anatomic structure;

estimating a segmentation boundary in the anatomic structure in the image of the patient's anatomy; and

predicting, using the trained model, a boundary location in the anatomic structure in the image of the patient's anatomy by generating a regression of distances from keypoints in the anatomic structure in the image of the patient's anatomy to the estimated boundary.

Assignments (5)
RELEASE OF SECURITY INTEREST Recorded Sep 11, 2025
From: HAYFIN SERVICES LLP
To: HEARTFLOW, INC.
Reel/Frame 072876/0775 →
RELEASE OF SECURITY INTEREST Recorded Jun 21, 2024
From: HAYFIN SERVICES LLP
To: HEARTFLOW, INC.
Reel/Frame 067801/0032 →
SECURITY INTEREST Recorded Jun 18, 2024
From: HEARTFLOW, INC.
To: HAYFIN SERVICES LLP
Reel/Frame 067775/0966 →
SECURITY INTEREST Recorded Jan 20, 2021
From: HEARTFLOW, INC.
To: HAYFIN SERVICES LLP
Reel/Frame 055037/0890 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 5, 2019
From: GRADY, LEO; PETERSEN, PETER KERSTEN; SCHAAP, MICHIEL; LESAGE, DAVID
To: HEARTFLOW, INC.
Reel/Frame 049380/0806 →