IP Library Granted Patent US 10,388,020
Granted Patent B2
US 10,388,020 · App. 14/394,097 · Granted Aug 20, 2019

Methods and systems for segmentation of organs and tumors and objects

Inventors: Xiaotao Guo (Bronx, NY); Binsheng Zhao (Forest Hills, NY); Lawrence H. Schwartz (New York, NY)
Assignee: The Trustees of Columbia University in the City of New York
G06T7/12G06K9/0014G06K9/34G06T7/11G06T7/143G06T7/149G06T2200/24G06T2207/10076G06T2207/10081G06T2207/10088G06T2207/10104G06T2207/10136G06T2207/20104G06T2207/20152G06T2207/30024G06T2207/30056G06T2207/30096
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,388,020
App. No.
14/394,097
Granted
Aug 20, 2019
Kind
B2
Abstract

Techniques for segmentation of organs and tumors and cells in image data include revising a position of a boundary by evaluating an evolution equation that includes differences of amplitude values for voxels on the boundary from a statistical metric of amplitude of voxels inside, and from a statistical metric of amplitude of voxels outside, for a limited region that lies within a distance r of the boundary. The distance r is small compared to a perimeter of the first boundary. Some techniques include determining a revised position of multiple boundaries by evaluating an evolution equation that includes differences in a first topographical distance from a first marker and a second topographical distance from a second marker for each voxel on the boundary, and also includes at least one other term related to boundary detection.

Claims (88)

1. A method comprising:

obtaining, on a processor, image data that indicates amplitude values at each of a plurality of voxels;

determining a watershed transform of the plurality of voxels;

obtaining, on a processor, initial boundary data that indicates an initial position of a first boundary associated with a first object represented by the image data and an initial position of a different second boundary associated with a different second object represented by the image data;

determining a first marker that indicates a first amplitude value inside the first object; determining a second marker that indicates a second amplitude value inside the second object;

determining, based on the watershed transform, a predecessor map that labels voxels each with data indicating a different voxel that is a predecessor on a path of steepest descent from a watershed line of the watershed transform, wherein one voxel on the watershed line satisfies that it is on the watershed line separating catchment basins of at least two minima and it is an upstream voxel of the other voxels;

determining, on a processor, a first topographical distance between a voxel on at least one boundary and the first marker;

determining, on a processor, a second topographical distance between the voxel and the second marker, wherein determining at least one of the first or the second topographical distance further comprises determining at least one of the first or the second topographical distance based at least in part on the predecessor map;

moving at least one of the first boundary and the second boundary, thereby forming a moved boundary, by evaluating an evolution equation that includes a boundary force that includes a difference in the first topographical distance and the second topographical distance for each voxel on the at least one boundary, and also includes at least one other term related to boundary detection; and

presenting automatically on a display device the moved boundary.

2. The method as recited in claim 1 wherein:

said moving the at least one of the first boundary and the second boundary further comprises evaluating an evolution equation that includes a first difference of amplitude values for voxels on the at least one of the first boundary and the second boundary from a statistical metric of amplitude of voxels inside the at least one of the first boundary and the second boundary and a second difference of amplitude values for voxels on the at least one of the first boundary and the second boundary from a statistical metric of amplitude of voxels outside the at least one of the first boundary and the second boundary for a limited region that lies within a distance r of the boundary, and

r is small compared to a perimeter of at least one of the first boundary and the second boundary.

3. The method as recited in claim 1 , wherein obtaining the initial boundary data further comprises obtaining the initial boundary data by propagating a boundary determined on a different subset of the voxels of the image data.

4. The method as recited in claim 1 , wherein obtaining the initial boundary data further comprises receiving a boundary for which at least a subset of the boundary is based on manually edited data received in response to a prompt for a manual edit presented in a graphical user interface for operation by a human user.

5. The method as recited in claim 1 , wherein obtaining image data that indicates amplitude values at each of a plurality of voxels further comprises obtaining image data that indicates amplitude values at each of the plurality of voxels and that excludes voxels outside a region of interest.

6. The method as recited in claim 5 , wherein the region of interest is based on data received in response to a prompt for a manual region of interest presented in a graphical user interface for operation by a human user.

7. The method as recited in claim 1 , wherein:

the initial boundary data is determined in a first subset of the image data; and

the method further comprising propagating the position of the at least one of the first boundary and the second boundary to a different subset of the voxels of the image data.

8. A method comprising:

obtaining, on a processor, image data that indicates amplitude values at each of a plurality of voxels;

obtaining, on a processor, initial boundary data that indicates an initial position of a first boundary associated with a first object represented by the image data and an initial position of a different second boundary associated with a different second object represented by the image data;

determining a first marker that indicates a first amplitude value inside the first object;

determining a second marker that indicates a second amplitude value inside the second object;

determining, on a processor, a first topographical distance between a voxel on at least one boundary and the first marker;

determining, on a processor, a second topographical distance between the voxel and the second marker;

moving at least one of the first boundary and the second boundary, thereby forming a moved boundary, by evaluating an evolution equation that includes a boundary force that includes a difference in the first topographical distance and the second topographical distance for each voxel on the at least one boundary, and also includes at least one other term related to boundary detection,

wherein the difference in the first topographical distance and the second topographical distance for each voxel on the at least one boundary is multiplied by a term that depends on distance between the first boundary and the second boundary; and

presenting automatically on a display device the moved boundary.

9. The method as recited in claim 8 , wherein:

said moving the at least one of the first boundary and the second boundary further comprises evaluating an evolution equation that includes a first difference of amplitude values for voxels on the at least one of the first boundary and the second boundary from a statistical metric of amplitude of voxels inside the at least one of the first boundary and the second boundary and a second difference of amplitude values for voxels on the at least one of the first boundary and the second boundary from a statistical metric of amplitude of voxels outside the at least one of the first boundary and the second boundary for a limited region that lies within a distance r of the boundary, and

r is small compared to a perimeter of at least one of the first boundary and the second boundary.

10. The method as recited in claim 8 , wherein obtaining the initial boundary data further comprises obtaining the initial boundary data by propagating a boundary determined on a different subset of the voxels of the image data.

11. The method as recited in claim 8 , wherein obtaining the initial boundary data further comprises receiving a boundary for which a least of subset of the boundary is based on manually edited data received in response to a prompt for a manual edit presented in a graphical user interface for operation by a human user.

12. The method as recited in claim 8 , wherein obtaining image data that indicates amplitude values at each of a plurality of voxels further comprises obtaining image data that indicates amplitude values at each of the plurality of voxels and that excludes voxels outside a region of interest.

13. The method as recited in claim 12 , wherein the region of interest is based on data received in response to a prompt for a manual region of interest presented in a graphical user interface for operation by a human user.

14. The method as recited in claim 8 , wherein:

the initial boundary data is determined in a first subset of the image data; and

the method further comprising propagating the position of the at least one of the first boundary and the second boundary to a different subset of the voxels of the image data.

15. A non-transitory computer-readable medium carrying one or more sequences of instructions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the steps of:

obtaining image data that indicates amplitude values at each of a plurality of voxels;

obtaining initial boundary data that indicates an initial position of first boundary associated with a first object represented by the image data and an initial position of a different second boundary associated with a different second object represented by the image data;

determining a watershed transform for the plurality of voxels;

determining a first marker that indicates a first amplitude value inside the first object;

determining a second marker that indicates a second amplitude value inside the second object;

determining, based on the watershed transform, a predecessor map that labels voxels each with data indicating a different voxel that is a predecessor on a path of steepest descent from a watershed line of the watershed transform, wherein one voxel on the watershed line satisfies that it is on the watershed line separating catchment basins of at least two minima and it is an upstream voxel of the other voxels;

determining a first topographical distance between each voxel and the first marker;

determining a second topographical distance between each voxel and the second marker, wherein determining at least one of the first or the second topographical distance further comprises determining at least one of the first or the second topographical distance based at least in part on the predecessor map;

moving at least one of the first boundary and the second boundary, thereby forming a moved boundary, by evaluating an evolution equation that includes a boundary force that includes a difference in the first topographical distance and the second topographical distance for each voxel on the at least one boundary, and also includes at least one other term related to boundary detection; and

presenting on a display device the moved boundary.

16. A system comprising:

at least one processor; and

at least one memory including one or more sequences of instructions,

the at least one memory and the one or more sequences of instructions configured to, with the at least one processor, cause an apparatus to perform at least the following,

obtain image data that indicates amplitude values at each of a plurality of voxels;

determine a watershed transform for the plurality of voxels;

obtain initial boundary data that indicates an initial position of first boundary associated with a first object represented by the image data and an initial position of a different second boundary associated with a different second object represented by the image data;

determine a first marker that indicates a first amplitude value inside the first object,

determine a second marker that indicates a second amplitude value inside the second object;

determine, based on the watershed transform, a predecessor map that labels voxels each with data indicating a different voxel that is a predecessor on a path of steepest descent from a watershed line of the watershed transform, wherein one voxel on the watershed line satisfies that it is on the watershed line separating catchment basins of at least two minima and it is an upstream voxel of the other voxels;

determine a first topographical distance between each voxel and the first marker;

determine a second topographical distance between each voxel and the second marker, wherein determining at least one of the first or the second topographical distance further comprises determining at least one of the first or the second topographical distance based at least in part on the predecessor map; and

move at least one of the first boundary and the second boundary, thereby forming a moved boundary, by evaluating an evolution equation that includes a boundary force that includes a difference in the first topographical distance and the second topographical distance for each voxel on the at least one boundary, and also includes at least one other term related to boundary detection; and

presenting on a display device the moved boundary.

17. A non-transitory computer-readable medium carrying one or more sequences of instructions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the steps of:

obtaining image data that indicates amplitude values at each of a plurality of voxels;

obtaining initial boundary data that indicates an initial position of a first boundary associated with a first object represented by the image data and an initial position of a different second boundary associated with a different second object represented by the image data;

determining a first marker that indicates a first amplitude value inside the first object;

determining a second marker that indicates a second amplitude value inside the second object;

determining a first topographical distance between a voxel on at least one boundary and the first marker;

determining a second topographical distance between the voxel and the second marker;

moving at least one of the first boundary and the second boundary, thereby forming a moved boundary, by evaluating an evolution equation that includes a boundary force that includes a difference in the first topographical distance and the second topographical distance for each voxel on the at least one boundary, and also includes at least one other term related to boundary detection; and

presenting on a display device the moved boundary,

wherein the difference in the first topographical distance and the second topographical distances for each voxel on the at least one boundary is multiplied by a term that depends on distance between the first boundary and the second boundary.

18. A system comprising:

at least one processor; and

at least one memory including one or more sequences of instructions,

the at least one memory and the one or more sequences of instructions configured to, with the at least one processor, cause an apparatus to perform at least the following,

obtain image data that indicates amplitude values at each of a plurality of voxels;

obtain initial boundary data that indicates an initial position of a first boundary associated with a first object represented by the image data and an initial position of a different second boundary associated with a different second object represented by the image data;

determine a first marker that indicates a first amplitude value inside the first object;

determine a second marker that indicates a second amplitude value inside the second object;

determine a first topographical distance between a voxel on at least one boundary and the first marker;

determine a second topographical distance between the voxel and the second marker;

moving at least one of the first boundary and the second boundary, thereby forming a moved boundary, by evaluating an evolution equation that includes a boundary force that includes a difference in the first topographical distance and the second topographical distance for each voxel on the at least one boundary, and also includes at least one other term related to boundary detection; and

presenting automatically on a display device the moved boundary,

wherein the difference in the first topographical distance and the second topographical distance for each voxel on the at least one boundary is multiplied by a term that depends on distance between the first boundary and the second boundary.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE ADDRESS OF THE RECEIVING PARTY PREVIOUSLY RECORDED AT REEL: 034242 FRAME: 0902. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Apr 20, 2015
From: ZHAO, BINSHENG; SCHWARTZ, LAWRENCE H.; GUO, XIAOTAO
To: THE TRUSTEES OF COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK
Reel/Frame 035463/0871 →
CONFIRMATORY LICENSE Recorded Dec 1, 2014
From: COLUMBIA UNIV NEW YORK MORNINGSIDE
To: NATIONAL INSTITUTES OF HEALTH (NIH), U.S. DEPT. OF HEALTH AND HUMAN SERVICES (DHHS), U.S. GOVERNMENT
Reel/Frame 034498/0176 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 23, 2014
From: ZHAO, BINSHENG; SCHWARTZ, LAWRENCE H.; GUO, XIAOTAO
To: THE TRUSTEES OF COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK
Reel/Frame 034242/0902 →
Continuity (2)
Provisional Application 61622917 · Apr 11, 2012
Related Publication 20150078640A1 · Mar 19, 2015
Cited By (3)
US 12,307,675 US 12,567,142 US 12,725,232