System and method for statistical shape model based segmentation of intravascular ultrasound and optical coherence tomography images
View Patent ↗A method for segmenting intravascular images includes acquiring a series of digitized images acquired from inside a vessel, each said image comprising a plurality of intensities corresponding to a 2-dimensional grid of pixels, providing a precomputed set of shapes for modeling contours of vessel wall boundaries, wherein a contour can be expressed as a sum of a mean shape and a inner product of shape modes and shape weights, initializing a boundary contour for one of said set of images, initializing said shape weights by projecting a contour into said shape modes, updating said shape weights from differential equations of said shape weights, and computing a contour by summing said mean shape and said inner product of shape modes and updated shape weights.
1. A method for segmenting intravascular images comprising the steps of:
acquiring a series of digitized images acquired from inside a vessel, each said image comprising a plurality of intensities corresponding to a 2-dimensional grid of pixels;
providing a precomputed set of shapes for modeling contours of vessel wall boundaries, wherein a contour can be expressed as a sum of a mean shape and a inner product of shape modes and shape weights;
initializing a boundary contour for one of said set of images;
initializing said shape weights by projecting a contour into said shape modes;
updating said shape weights from differential equations of said shape weights; and
computing a contour by summing said mean shape and said inner product of shape modes and updated shape weights.
2. The method of claim 1 , further comprising repeating said steps of updating said shape weights and computing a contour until said contour converges.
3. The method of claim 2 , wherein convergence of said contour occurs when and area enclosed by said contour begins to oscillate.
4. The method of claim 1 , wherein said shape weights are initialized according to w c =U c T (Φ init c −Φ mean c ), wherein w c is a vector of shape weights, Φ init c is the initialized boundary contour, Φ mean c is the mean shape for the boundary, and U c T a matrix representing the shape modes.
5. The method of claim 1 , wherein said shape weights are updated according to
w
c
(
t
+
1
)
=
w
c
(
t
)
+
α
∂
w
c
∂
t
(
t
)
,
wherein w c (t+1) is the updated shape weight, w c (t) is a current shape weight,
∂
w
c
∂
t
(
t
)
is a gradient of the current shape weight, and α is a step size.
6. The method of claim 5 , wherein said shape weights are updated according to
w
i
c
(
t
+
1
)
=
w
i
c
(
t
)
+
α
[
∂
w
i
c
∂
t
(
t
-
1
)
+
(
1
-
δ
)
∂
w
c
∂
t
(
t
)
]
,
wherein δ is a weighting factor, and wherein
∂
w
i
c
∂
t
(
t
-
1
)
is a previous weight shape gradient.
7. The method of claim 1 , wherein said contour is computed from said mean shape and said inner product of shape modes and updated shape weights according to
Φ
c
(
w
)
=
Φ
mean
c
+
∑
i
=
1
k
w
i
c
(
t
+
1
)
U
i
c
,
wherein w i c (t+1) is the i th component of the updated shape weight, U i c is the i th component of the shape modes, Φ mean c is the mean shape for the boundary, and Φ c (w) is the contour.
8. The method of claim 1 , further comprising preprocessing the images to remove acquisition artifacts, including finding a minimum image I min of said series of images, computing sums over rows of I min until a global maxima is found, determining an artifact location as where a row sum falls below 50%, omitting rows above said artifact from said segmentation, wherein said artifact ends at a first local minimum after said global maximum.
9. The method of claim 1 , wherein said precomputed set of shapes are computed from the steps of:
acquiring a training set of digitized in-vivo intravascular images;
forming a training set of boundary contour shapes from said training set of images;
aligning said shapes in a radial direction and computing a mean shape of the set of shapes;
subtracting the mean shape from the set of training shapes to form a set of shape variability matrices; and
determining a set of modes and weights wherein a shape model Φ is expressed as a sum of said mean shape and an inner product of the modes and weights according to
Φ
(
w
)
=
Φ
mean
+
∑
i
=
1
k
w
i
U
i
,
wherein Φ mean is said mean shape, w i are the weights, and U i are the modes.
10. The method of claim 1 , wherein said images are acquired via ultrasound, wherein said boundary contour is a lumen contour, wherein said boundary contour is initialized by shifting said mean shape in an angular direction to minimize a mean image intensity above the contour, and wherein said gradient
∂
w
∂
t
of said shape weight is calculated according to
∂
w
∂
t
=
∫
C
-
log
(
P
i
n
(
I
(
x
)
)
P
out
(
I
(
x
)
)
)
U
l
ⅆ
x
,
wherein C is the contour, I(x) is an image intensity at pixel x, U l is a lumen shape mode, and P in and P out are probability densities of an image point being, respectively, inside the contour and outside the contour.
11. The method of claim 10 , wherein said probability densities are estimated from a kernel density estimator
P
(
q
)
=
1
N
σ
2
π
∑
i
=
1
N
exp
(
(
q
-
q
i
)
2
2
σ
2
)
,
wherein N is a number of pixels inside or outside the contour, q is an intensity value inside or outside the contour, σis a corresponding variance, and q i is an individual intensity at pixel i.
12. The method of claim 10 , wherein said probability ,densities are functions of a modified intensity that is an averaged intensity over a column above a contour position x=(x, y):
I
modified
(
x
,
y
)
=
max
y
0
∈
[
0
,
y
]
1
y
-
y
0
+
1
∑
y
i
=
y
0
y
I
(
x
,
y
)
wherein the origin of the image (0,0) is a top left corner of said image.
13. The method of claim 1 , wherein said boundary contour is initialized by dividing an image into a plurality of columns, finding a position of a maximal gradient for every column wherein a contour is initialized, and passing a median filter over the initial contour wherein noise is eliminated.
14. The method of claim 13 , wherein said gradient of said shape weight
∂
w
∂
t
is calculated according to
∂
w
e
∂
t
=
∫
C
∇
G
(
x
)
U
a
ⅆ
x
,
wherein U a is a shape contour, and ∇G represents the smoothed oriented edge gradient taken as a difference between an average intensity of two oriented windows above and below said contour,
∇ G ( x )= G oriented ( x −βn)− G oriented ( x βn)
wherein β is a scaling factor to determine the extent of said oriented boxes above and below said contour, and n is the normal vector to the contour at position x, and
G
oriented
(
x
)
=
1
(
2
dx
+
1
)
dy
(
∑
i
=
-
dx
dx
∑
j
=
-
dy
-
1
I
(
x
+
iu
+
jn
)
-
∑
i
=
-
dx
dx
∑
j
=
1
dy
I
(
x
+
iu
+
jn
)
)
,
wherein 2dx+1 is the width and dy the height of said gradient window, x is a 2D image pixel, and u is the unit tangent vector to the contour at point x.
15. The method of claim 14 , wherein said boundary contour is one of a media/adventitia contour in an ultrasound image, or a lumen contour in an optical coherence tomography image.
16. A method for segmenting intravascular images comprising the steps of:
acquiring a training set of digitized in-vivo intravascular images, each said image comprising a plurality of intensities corresponding to a 2-dimensional grid of pixels;
forming a training set of boundary contour shapes from said training set of images;
aligning said shapes in a radial direction and computing a mean shape of the set of shapes;
subtracting the mean shape from the set of training shapes to form a set of shape variability matrices;
determining a set of modes and weights wherein a shape model Φ is expressed as a sum of said mean shape and an inner product of the modes and weights according to
Φ
(
w
)
=
Φ
mean
+
∑
i
=
1
k
w
i
U
i
,
wherein Φ mean is said mean shape, w i are the weights, and U i are the modes;
providing a digitized in-vivo intravascular testing image; and
segmenting a boundary contour in said testing image by using said shape model to initialize said boundary contour and to drive said segmentation.
17. The method of claim 16 , wherein segmenting said boundary contour comprises the steps of:
initializing a first shape Φ init ;
initializing said weights w from w =U T (Φ init −Φ mean ),
computing weight gradients
∂
w
∂
t
and updating said weights according to
w
(
t
+
1
)
=
w
(
t
)
+
α
∂
w
∂
t
(
t
)
,
wherein t is an iteration step number and α is a step size;
updating shapes from
Φ
(
w
)
=
Φ
mean
+
∑
i
=
1
k
w
i
(
t
+
1
)
U
i
,
wherein the sum is over all k modes; and
repeating said steps of computing weight gradients and updating weights until said shape contour converges.
18. The method of claim 17 , wherein said images are acquired via ultrasound and said boundary contour is a lumen, wherein said boundary contour is initialized by shifting said mean shape in an angular direction, and wherein said gradient
∂
w
∂
t
of said shape weight is calculated according to
∂
w
∂
t
=
∫
C
-
log
(
P
i
n
(
I
(
x
)
)
P
out
(
I
(
x
)
)
)
U
l
ⅆ
x
,
wherein C is the contour, I(x) is an image intensity at pixel x, U l is a lumen shape mode, wherein P in and P out are probability densities of an image point being, respectively, inside the contour and outside the contour, and said probability densities are estimated from
P
(
q
)
=
1
N
σ
2
π
∑
i
=
1
N
exp
(
(
q
-
q
i
)
2
2
σ
2
)
,
wherein N is a number of pixels inside or outside the contour, q is an intensity value inside or outside the contour, σ is a corresponding variance, and q i is an individual intensity at pixel i.
19. The method of claim 17 , wherein said boundary contour is one of a media/adventitia contour in an ultrasound image, or a lumen contour in an optical coherence tomography image, wherein said boundary contour is initialized by finding a position of a maximal gradient for every image column, wherein said gradient of said shape weight
∂
w
∂
t
is calculated according to
∂
w
a
∂
t
=
∫
C
∇
G
(
x
)
U
a
ⅆ
x
,
wherein U a is a shape contour, and ∇G represents the smoothed oriented edge gradient taken as a difference between an average intensity of two oriented windows above and below said contour,
∇ G ( x )= G oriented ( x −β)−G oriented ( x +βn)
where β is a scaling factor to determine the extent of said oriented boxes above and below said contour, and n is the normal vector to the contour at position x, and
G
oriented
(
x
)
=
1
(
2
dx
+
1
)
dy
(
∑
i
=
-
dx
dx
∑
j
=
-
dy
-
1
I
(
x
+
iu
+
jn
)
-
∑
i
=
-
dx
dx
∑
j
=
1
dy
I
(
x
+
iu
+
jn
)
)
,
wherein 2dx+1 is the width and dy the height of said gradient window, x is a 2D image pixel, and u is the unit tangent vector to the contour at point x.
20. A program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method steps for segmenting intravascular images, the method comprising the steps of:
acquiring a series of digitized images acquired from inside a vessel, each said image comprising a plurality of intensities corresponding to a 2-dimensional grid of pixels;
providing a precomputed set of shapes for modeling contours of vessel wall boundaries, wherein a contour can be expressed as a sum of a mean shape and a inner product of shape modes and shape weights;
initializing a boundary contour for one of said set of images;
initializing said shape weights by projecting a contour into said shape modes;
updating said shape weights from differential equations of said shape weights; and
computing a contour by summing said mean shape and said inner product of shape modes and updated shape weights.
21. The computer readable program storage device of claim 20 , the method further comprising repeating said steps of updating said shape weights and computing a contour until said contour converges.
22. The computer readable program storage device of claim 21 , wherein convergence of said contour occurs when and area enclosed by said contour begins to oscillate.
23. The computer readable program storage device of claim 20 , wherein said shape weights are initialized according to w c =U c T (Φ init c −Φ mean c ), wherein w c is a vector of shape weights, Φ init c is the initialized boundary contour, Φ mean c is the mean shape for the boundary, and U c T is a matrix representing the shape modes.
24. The computer readable program storage device of claim 20 , wherein said shape weights are updated according to
w
c
(
t
+
1
)
=
w
c
(
t
)
+
α
∂
w
c
∂
t
(
t
)
,
wherein w c (t+1) is the updated shape weight, w c (t) is a current shape weight,
∂
w
c
∂
t
(
t
)
is a gradient of the current shape weight, and α is a step size.
25. The computer readable program storage device of claim 24 , wherein said shape weights are updated according to
w
i
a
(
t
+
1
)
=
w
i
c
(
t
)
+
α
[
δ
∂
w
i
c
∂
t
(
t
-
1
)
+
(
1
-
δ
)
∂
w
i
c
∂
t
(
t
)
]
,
wherein δ is a weighting factor, and wherein
∂
w
i
c
∂
t
(
t
-
1
)
is a previous weight shape gradient.
26. The computer readable program storage device of claim 20 , wherein said contour is computed from said mean shape and said inner product of shape modes and updated shape weights according to
Φ
c
(
w
)
=
Φ
mean
c
+
∑
i
=
1
k
w
i
c
(
t
+
1
)
U
i
a
,
wherein w i c (t+1) is the i th component of the updated shape weight, U i c is the i th component of the shape modes, Φ mean c is the mean shape for the boundary, and Φ c (W) is the contour.
27. The computer readable program storage device of claim 20 , the method further comprising preprocessing the images to remove acquisition artifacts, including finding a minimum image I min of said series of images, computing sums over rows of I min until a global maxima is found, determining an artifact location as where a row sum falls below 50%, omitting rows above said artifact from said segmentation, wherein said artifact ends at a first local minimum after said global maximum.
28. The computer readable program storage device of claim 20 , wherein said precomputed set of shapes are computed from the steps of:
acquiring a training set of digitized in-vivo intravascular images;
forming a training set of boundary contour shapes from said training set of images;
aligning said shapes in a radial direction and computing a mean shape of the set of shapes;
subtracting the mean shape from the set of training shapes to form a set of shape variability matrices; and
determining a set of modes and weights wherein a shape model Φ is expressed as a sum of said mean shape and an inner product of the modes and weights according to
Φ
(
w
)
=
Φ
mean
+
∑
i
=
1
k
w
i
U
i
,
wherein Φ mean is said mean shape, w i are the weights, and U i are the modes.
29. The computer readable program storage device of claim 20 , wherein said images are acquired via ultrasound, wherein said boundary contour is a lumen contour, wherein said boundary contour is initialized by shifting said mean shape in an angular direction to minimize a mean image intensity above the contour, and wherein said gradient
∂
w
∂
t
of said shape weight is calculated according to
∂
w
∂
t
=
∫
C
-
log
(
P
i
n
(
I
(
x
)
)
P
out
(
I
(
x
)
)
)
U
l
ⅆ
x
,
wherein C is the contour, I(x) is an image intensity at pixel x, U l is a lumen shape mode, and P in and P out are probability densities of an image point being, respectively, inside the contour and outside the contour.
30. The computer readable program storage device of claim 29 , wherein said probability densities are estimated from a kernel density estimator
P
(
q
)
=
1
N
σ
2
π
∑
i
=
1
N
exp
(
(
q
-
q
i
)
2
2
σ
2
)
,
wherein N is a number of pixels inside or outside the contour, q is an intensity value inside or outside the contour, σ is a corresponding variance, and q i is an individual intensity at pixel i.
31. The computer readable program storage device of claim 29 , wherein said probability densities are functions of a modified intensity that is an averaged intensity over a column above a contour position x=(x, y):
I
modified
(
x
,
y
)
=
max
y
0
∈
[
0
,
y
]
1
y
-
y
0
+
1
∑
y
i
=
y
0
y
I
(
x
,
y
i
)
,
wherein the origin of the image (0,0) is a top left corner of said image.
32. The computer readable program storage device of claim 20 , wherein said boundary contour is initialized by dividing an image into a plurality of columns, finding a position of a maximal gradient for every column wherein a contour is initialized, and passing a median filter over the initial contour wherein noise is eliminated.
33. The computer readable program storage device of claim 32 , wherein said gradient of said shape weight
∂
w
∂
t
is calculated according to
∂
w
a
∂
t
=
∫
C
∇
G
(
x
)
U
a
ⅆ
x
,
wherein U a is a shape contour, and ∇G represents the smoothed oriented edge gradient taken as a difference between an average intensity of two oriented windows above and below said contour,
∇ G ( x )= G oriented ( x −βn)− G oriented ( x +βn)
where β is a scaling factor to determine the extent of said oriented boxes above and below said contour, and n is the normal vector to the contour at position x, and
G
oriented
(
x
)
=
1
(
2
dx
+
1
)
dy
(
∑
i
=
-
dx
dx
∑
j
=
-
dy
-
1
I
(
x
+
iu
+
jn
)
-
∑
i
=
-
dx
dx
∑
j
=
1
dy
I
(
x
+
iu
+
jn
)
)
,
wherein 2dx+1 is the width and dy the height of said gradient window, x is a 2D image pixel, and u is the unit tangent vector to the contour at point x.
34. The computer readable program storage device of claim 33 , wherein said boundary contour is one of a media/adventitia contour in an ultrasound image, or a lumen contour in an optical coherence tomography image.