IP Library Granted Patent US 10,019,796
Granted Patent B2
US 10,019,796 · App. 15/270,061 · Granted Jul 10, 2018

System and method for blood vessel analysis and quantification in highly multiplexed fluorescence imaging

Inventors: Yousef Al-Kofahi (Niskayuna, NY); Anup Sood (Clifton Park, NY); Fiona Ginty (Niskayuna, NY); Qing Li (Niskayuna, NY); Christopher James Sevinsky (Watervliet, NY); Alberto Santamaria-Pang (Niskayuna, NY)
Assignee: GENERAL ELECTRIC COMPANY
G06T7/0012G06T7/0042G06T7/0087G06T11/60G06T2207/10004G06T2207/10056G06T2207/10064G06T2207/20076G06T2207/20144G06T2207/30101G06T2207/30172G06T2207/30204
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,019,796
App. No.
15/270,061
Granted
Jul 10, 2018
Kind
B2
Abstract

Disclosed are novel computer-implemented methods for creating a blood vessel map of a biological tissue. The methods comprise the steps of, accessing image data corresponding to multi-channel multiplexed image of a fluorescently stained biological tissue manifesting expression levels of a primary marker and at least one auxiliary marker of blood vasculature, and extracting features of blood vessels using the primary marker as an input to create a single channel segmentation of the blood vessels. The method further comprises the steps of extracting features of blood vessels using the auxiliary marker to create auxiliary channels as a second input and apply multi-channel blood vessel enhancement. Blood vessel maps are created using the features and tracing the blood vasculature by iteratively extending the centerlines of the initial segmentation using statistical models and geometric rules.

Claims (286)

1. A computer-implemented method for creating a blood vessel map of a biological tissue, the method comprising the steps of:

accessing image data corresponding to a multi-channel multiplexed image of a biological tissue fluorescent stained to manifest expression levels of a primary marker and at least one auxiliary marker, each marker marking blood vasculature;

extracting features of blood vessels using the primary marker as an input to create an initial single channel segmentation of the blood vessels as a first input;

extracting features of blood vessels using the at least one auxiliary marker to create auxiliary channel segmentations as a second input and applying multi-channel blood vessel enhancement to the first and second inputs;

computing a probability map using the extracted features;

tracing the blood vasculature by iteratively extending the centerlines of the initial single channel segmentation using statistical models and geometric rules; and

generating a corresponding blood vessel map.

2. The method of claim 1 further comprising rendering on a visual display device a representation of the blood vessels within the tissue sample.

3. The method of claim 1 where the primary marker is CD31.

4. The method of claim 1 where the auxiliary marker is CD34, ColIV, SMA, von Willebrand Factor or a combination thereof.

5. The method of claim 1 where the biological tissue is a biopsy or surgical sample.

6. The method of claim 1 where generating a blood vessel map further comprises at least one of enhancing the blood vessels' morphology or suppressing the non-vessel signal in the image data.

7. The method of claim 6 where the enhancing comprises using a Frangi's filter.

8. The method of claim 7 where using the Frangi's filter comprises the steps of transforming the image data into pixel scale such that:

I is an image and G_σ is a Gaussian filter where σ represents its scale;

L_σ is an Gaussian filtered image (I*G_σ) at scale σ; and

H(x,y) is a Hessian matrix at pixel (x,y), computed from the second order partial derivatives as:

H

σ

(

x

,

y

)

=

[

2

L

σ

(

x

,

y

)

x

2

2

L

σ

(

x

,

y

)

x

y

2

L

σ

(

x

,

y

)

x

y

2

L

σ

(

x

,

y

)

y

2

]

where λ 1 and λ 2 be the eigenvalues of H σ such that |λ 1 |≥|λ 2 |;

computing Frangi's scale σ as:

V

p

σ

(

x

,

y

)

=

{

(

1

-

e

-

2

R

2

)

(

1

-

e

-

8

S

2

)

,

λ

1

<

0

and

λ

2

<

0

0

,

otherwise

;

and

R

=

λ

1

λ

2

and

S

=

λ

1

2

+

λ

2

2

;

and

and creating a map V p (x, y) using the pixel-wise maximum across scales as:

V p ( x,y )=max( V p σ ( x,y )|∀σ).

9. The method of claim 1 where tracing further comprises using the initial single channel segmentation and the probability map to extend the initial single channel segmentation to improve its continuity and completeness.

10. The method of claim 9 where the tracing is iterative.

11. The method of claim 10 where iterative tracing comprises

extracting the centerline of the initial single channel segmentation to create initial fragments;

identifying the end points of the initial fragments;

extending the end points of the initial fragments to generate a more complete blood vessel segmentation.

12. The method of claim 11 where extending the end points comprises creating a template to identify foreground and background probabilities to categorize features as blood vessels.

13. The method of claim 12 where creating a template comprises:

Creating a template T i of size M×N at each direction θ i and compute average foreground (Fg i ) and background (Bg i ) probabilities as;

Fg

i

=

1

T

i

(

x

,

y

)

T

i

F

v

(

x

,

y

)

;

Bg

i

=

1

T

i

(

x

,

y

)

T

i

B

v

(

x

,

y

)

;

and

where ∥T i ∥ represents the size/area of the template T i , and ‘M’ and ‘N’ are positive integers.

14. The method of claim 13 further comprising testing hypothesis (H 0 and H 1 ) to extend the end points where:

H 0 : template T i is background

H 1 : template T i is foreground; and

selecting the direction of extension for the end point a ratio test such that, the ratio test is

argmax

i

(

Fg

i

Bg

i

)

<

H

0

>

H

1

τ

(

τ

is

set

to

1.5

)

;

and

if T i satisfied the ratio test; then

extending the end point (x, y) by a predetermined number of pixels until the endpoints intersects another segment in the direction θ i .

15. The method of claim 8 where the map is used to analyze and quantify vasculature in a tissue sample.

16. The method of claim 10 where the biological tissue is a biopsy or surgical sample.

17. The method of claim 10 further comprising creating a blood vessel segmentation mask based on the identified blood vessels.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 11, 2021
From: GLOBAL LIFE SCIENCES SOLUTIONS USA LLC
To: LEICA MICROSYSTEMS CMS GMBH
Reel/Frame 057261/0128 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 1, 2020
From: GENERAL ELECTRIC COMPANY
To: GLOBAL LIFE SCIENCES SOLUTIONS USA LLC
Reel/Frame 053966/0133 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 20, 2016
From: AL-KOFAHI, YOUSEF; SOOD, ANUP; GINTY, FIONA; LI, QING; SEVINSKY, CHRISTOPHER JAMES; SANTAMARIA-PANG, ALBERTO
To: GENERAL ELECTRIC COMPANY
Reel/Frame 039793/0234 →
Continuity (2)
Provisional Application 62242363 · Oct 16, 2015
Related Publication 20170109880A1 · Apr 20, 2017