System and method for learning-based 2D/3D rigid registration for image-guided surgery using Jensen-Shannon divergence
View Patent ↗A method of registering 3-dimensional digitized images to 2-dimensional digitized images during a medical procedure includes providing a pair of correctly-registered training images L={l r , l f } and their joint intensity distribution p l (i r , i f ), wherein i r and i f are reference and floating images, respectively, providing a pair of observed images O={o r , o f } and their joint intensity distribution p o (i r , i f ), mapping a marginal intensity distribution of the observed pair O={o r , o f } to a marginal intensity distribution of the training pair L={l r , l f }, and estimating a set of parameters T that registers image o f to image o r by maximizing a weighted sum of a Jensen-Shannon divergence (JSD) of a joint intensity distribution of the observed pair and a joint intensity distribution of the training pair and a similarity measure between the observed images.
1. A method of registering 3-dimensional digitized images to 2-dimensional digitized images during a medical procedure comprising the steps of:
providing a pair of correctly-registered training images L={l r , l f }, wherein l r and l f are reference and floating images, respectively, and their joint intensity distribution p l (i r , i f ), wherein i r and i f are reference and floating images, respectively;
providing a pair of observed images O={o r , o f }, wherein o r and o f are reference and floating images, respectively, and their joint intensity distribution p o (i r , i f );
mapping a marginal intensity distribution of the observed pair O={o r , o f } to a marginal intensity distribution of the training pair L={l r , l f }; and
estimating a set of parameters T that registers image o f to image o r by maximizing a weighted sum of a Jensen-Shannon divergence (JSD) of a joint intensity distribution of the observed pair and a joint intensity distribution of the training pair and a similarity measure between the observed images.
2. The method of claim 1 , wherein mapping the marginal intensity distribution of the observed pair to the marginal intensity distribution of the training pair comprises mapping an intensity i for observed image o r to intensity k for training image l r where k minimizes an absolute difference between the marginal histograms p o r (i) and p l r (k) of o r and l r respectively, where t 1 ≦k≦t 2 with t 1 and t 2 defined by C l r (t 1 −1)<C o r (i−1)≦C l r (t 1 ) and C l r (t 2 −1)<C o r (i)≦C l r (t 2 ) wherein C o r ( ) and C l r ( ) denote cumulative density functions of o r and l r respectively.
3. The method of claim 1 , wherein one of said pair of observed images is a 2D digitally reconstructed radiograph (DRR) generated from a 3-dimensional (3D) image of a medical target, and the other of said pair is a live 2-dimensional (2D) X-ray image of said medical target acquired in real-time from said medical target during a medical procedure.
4. The method of claim 3 , wherein said registration parameters T comprise six parameters that characterize a pose of said 3D image with respect to said 2D X-ray image.
5. The method of claim 4 , wherein maximizing said weighted sum comprises maximizing an expression equivalent to
α(1 −JSD ( p l ∥p o T ))+(1−α)[ MI ( o r , o f T ) or PI ( o r , o f T )],
wherein JISD(p l ∥p o T ) is a Jensen-Shannon divergence of the joint intensity distribution p l of the training images and of the joint intensity distribution p o T of the observed images wherein one of said observed images has been registered to the other via the parameters T, MI(o r ,o f T ) is a Mutual Information measure of the registered observed images, PI(o r ,o f T ) is a Pattern Intensity measure of the registered observed images, and α is a weighting factor with values between 0 and 1.
6. The method of claim 5 , wherein said expression is maximized using a Hill-climbing optimization method.
7. The method of claim 5 , wherein the Jensen-Shannon divergence is defined as
JSD
(
p
l
p
o
T
)
=
1
2
D
(
p
l
p
M
)
+
1
2
D
(
p
o
T
p
M
)
,
p
M
=
1
2
(
p
l
+
p
o
T
)
,
where
D
(
p
1
p
2
)
=
∫
∫
r
,
i
f
p
1
(
i
r
,
i
f
)
log
[
p
1
(
i
r
,
i
f
)
/
p
2
(
i
r
,
i
f
)
]
ⅆ
i
r
ⅆ
i
f
,
wherein i r and i f are the intensities of a reference image and a floating image, respectively, in each pair of training and observed images, p l is either p l or p o T and p 2 is p M .
8. The method of claim 5 , wherein the Mutual Information measure of the registered observed images is defined as
MI
(
o
r
,
o
f
T
)
=
∫
∫
r
,
i
f
p
o
T
(
i
r
,
i
f
)
log
⌊
p
o
T
(
i
r
,
i
f
)
/
(
p
o
r
(
i
r
)
p
o
f
T
(
i
f
)
)
⌋
ⅆ
i
r
ⅆ
i
f
,
wherein p o r (i r ) and p o r T (i f ) are the marginal distributions for o r and o f T , respectively.
9. The method of claim 5 , wherein the Pattern Intensity measure of the registered observed images is defined as
PI
(
o
r
,
o
f
T
)
=
∫
∫
x
,
y
σ
2
/
⌊
σ
2
+
(
o
d
(
x
,
y
)
-
o
d
(
v
,
w
)
)
2
⌋
ⅆ
x
ⅆ
y
,
wherein o d =o r −o f T =is a difference image, o d (x,y) denotes the intensity value of o d at pixel (x,y), wherein (v−x) 2 +(w−y) 2 <R 2 define the region centered at (x,y) with radius R, wherein σ and R are pre-defined constants.
10. The method of claim 1 , further comprising repeating the steps of providing a pair of observed images, mapping a marginal intensity distribution of the observed pair to a marginal intensity distribution of the training pair, and estimating a set of parameters T that registers the observed pair of images until a difference in parameter values T between iterations is sufficiently small.
11. A program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method steps for registering 3-dimensional digitized images to 2-dimensional digitized images during a medical procedure comprising the steps of:
providing a pair of correctly-registered training images L={l r , l f }, wherein l r and l f are reference and floating images, respectively, and their joint intensity distribution p l (i r , i f ), wherein i r and i f are reference and floating images, respectively;
providing a pair of observed images O={o r , o f }, wherein o r and o f are reference and floating images, respectively, and their joint intensity distribution p o (i r ,i f );
mapping a marginal intensity distribution of the observed pair O={o r , o f } to a marginal intensity distribution of the training pair L={l r , l f } and
estimating a set of parameters T that registers image o f to image o r by maximizing a weighted sum of a Jensen-Shannon divergence (JSD) of a joint intensity distribution of the observed pair and a joint intensity distribution of the training pair and a similarity measure between the observed images.
12. The computer readable program storage device of claim 11 , wherein mapping the marginal intensity distribution of the observed pair to the marginal intensity distribution of the training pair comprises mapping an intensity i for observed image o r to intensity k for training image l r where k minimizes an absolute difference between the marginal histograms p o r (i) and p l r (k) of o r and l r respectively, where t 1 <k<t 2 with t 1 and t 2 defined by C l r (t 1 −1)<C o r (i−1)≦C l r (t 1 ) and C l r (t 2 −1)<C o r (i)≦C l r (t 2 ) wherein C o r ( ) and C l r ( ) denote cumulative density functions of o r and l r respectively.
13. The computer readable program storage device of claim 11 , wherein one of said pair of observed images is a 2D digitally reconstructed radiograph (DRR) generated from a 3-dimensional (3D) image of a medical target, and the other of said pair is a live 2-dimensional (2D) X-ray image of said medical target acquired in real-time from said medical target during a medical procedure.
14. The computer readable program storage device of claim 13 , wherein said registration parameters T comprise six parameters that characterize a pose of said 3D image with respect to said 2D X-ray image.
15. The computer readable program storage device of claim 14 , wherein maximizing said weighted sum comprises maximizing an expression equivalent to
α(1 −JSD ( p l ∥p o T ))+(1−α)[ MI ( o r , o f T ) or PI ( o r , o f T )],
wherein JSD(p l ∥p o T ) is a Jensen-Shannon divergence of the joint intensity distribution p l of the training images and of the joint intensity distribution p o T of the observed images wherein one of said observed images has been registered to the other via the parameters T, MI(o r ,o f T ) is a Mutual Infoiniation measure of the registered observed images, PI(o r ,o f T ) is a Pattern Intensity measure of the registered observed images, and α is a weighting factor with values between 0 and 1.
16. The computer readable program storage device of claim 15 , wherein said expression is maximized using a Hill-climbing optimization method.
17. The computer readable program storage device of claim 15 , wherein the Jensen-Shannon divergence is defined as
JSD
(
p
l
p
o
T
)
=
1
2
D
(
p
l
p
M
)
+
1
2
D
(
p
o
T
p
M
)
,
p
M
=
1
2
(
p
l
+
p
o
T
)
,
where
D
(
p
1
p
2
)
=
∫
∫
r
,
i
f
p
1
(
i
r
,
i
f
)
log
[
p
1
(
i
r
,
i
f
)
/
p
2
(
i
r
,
i
f
)
]
ⅆ
i
r
ⅆ
i
f
,
wherein i r and i f are the intensities of a reference image and a floating image, respectively, in each pair of training and observed images, p l is either p l or p o T , and p 2 is p M .
18. The computer readable program storage device of claim 15 , wherein the Mutual Information measure of the registered observed images is defined as
MI
(
o
r
,
o
f
T
)
=
∫
∫
r
,
i
f
p
o
T
(
i
r
,
i
f
)
log
⌊
p
o
T
(
i
r
,
i
f
)
/
(
p
o
r
(
i
r
)
p
o
f
T
(
i
f
)
)
⌋
ⅆ
i
r
ⅆ
i
f
,
wherein p o r (i r )and p o f T (i f ) are the marginal distributions for o r and o f T , respectively.
19. The computer readable program storage device of claim 15 , wherein the Pattern Intensity measure of the registered observed images is defined as
PI
(
o
r
,
o
f
T
)
=
∫
∫
x
,
y
σ
2
/
⌊
σ
2
+
(
o
d
(
x
,
y
)
-
o
d
(
v
,
w
)
)
2
⌋
ⅆ
x
ⅆ
y
,
wherein o d =o r −o f T =is a difference image, o d (x,y) denotes the intensity value of o d a pixel (x,y), wherein (v−x) 2 +(w−y) 2 <R 2 define the region centered at (x,y) with radius R, wherein σ and R are pre-defined constants.
20. The computer readable program storage device of claim 11 , the method further comprising repeating the steps of providing a pair of observed images, mapping a marginal intensity distribution of the observed pair to a marginal intensity distribution of the training pair, and estimating a set of parameters T that registers the observed pair of images until a difference in parameter values T between iterations is sufficiently small.