Multimodal image registration using compound mutual information
View Patent ↗In a digital image registration method, a second digital image is transformed to be more like a first digital image using a modification function, resulting in a modified second digital image. A set of first features of each of the first and modified second digital images is determined. Zones are specified in each of the first and modified second digital images. A set of zonal features, of each of the zones is determined. Compound mutual information is computed from the first and zonal features. The modification function is altered. The transforming, determining, specifying, determining, computing, and altering are iterated until the compound mutual information is maximized.
1. A method for registering first and second digital images captured using first and second different capture modalities, said method comprising the steps of:
transforming said second digital image in the second modality to be more like said first digital image in the first modality using a modification function, to provide a modified second digital image;
determining a set of first features of each of said first and modified second digital images;
specifying mutually exclusive zones in each of said first and modified second digital images where each of the specified mutually exclusive zones is less than an entirety of each of said first and modified second digital images;
determining a set of zonal features of each of said specified mutually exclusive zones of each of said first and modified second digital images;
computing compound mutual information from said first features and said zonal features where compound mutual information comprises an aggregate measure of mutual information;
altering said modification function; and
iterating said transforming, determining, specifying, determining, computing, and altering steps until said compound mutual information is maximized,
wherein said computing further comprises:
(1) extracting a set of global marginal densities from a set of global features and a set of zonal marginal densities from said set of zonal features, and
(2) applying weighting coefficients to said sets of marginal densities to provided weighted marginal mutual information, and aggregating said weighted marginal mutual information in accordance with the formula:
I
(
A
,
B
)
=
∑
m
=
1
M
(
λ
m
∑
h
a
h
b
(
p
(
h
a
m
,
h
b
m
)
log
p
(
h
a
m
,
h
b
m
)
p
(
h
a
m
)
p
(
h
b
m
)
)
)
where
A and a signify the first digital image;
B and b signify the modified second digital image;
M is the total number of sets of global and zonal features;
p(h a m ), p(h b m ) and p(h a m , h b m ) are marginal densities derived from image features {h m } m=1 M ; and
λ m are weighting coefficients, where
∑
m
λ
m
=
1.
2. The method of claim 1 wherein said zonal features are different than said first features.
3. The method of claim 1 further comprising, following said iterating, using said modified second digital image in place of said second digital image.
4. The method of claim 1 wherein said first features are global.
5. The method of claim 4 further comprising, prior to said computing, determining at least one other set of global features of each of said first and modified second digital images.
6. The method of claim 5 wherein said first features are intensity/color histograms.
7. The method of claim 1 wherein said specified mutually exclusive zones define ring/shell shapes.
8. The method of claim 1 wherein said specified mutually exclusive zones each include a center pixel and a plurality of neighboring pixels, said neighboring pixels being encoded based on respective locations relative to said center pixel.
9. The method of claim 1 wherein said specified mutually exclusive zones are each rotation invariant relative to the center of the respective zone.
10. The method of claim 1 wherein said specifying and said determining of said set of zonal features are concurrent.
11. The method of claim 1 further comprising, prior to said computing, designating at least one set of other zones in each of said first and modified second digital images, and determining at least one other set of zonal features of each of said other zones of each of said first and modified second digital images.
12. The method of claim 11 wherein said at least one set of other zones of each of said sets of features are at different distances from respective center pixels.
13. The method of claim 11 wherein said at least one set of other zones define ring/shell shapes.
14. The method of claim 1 wherein at least one of said determining steps includes converting said first and modified second digital images from spatial domain to one of frequency domain and space-frequency domain.
15. The method of claim 1 wherein said modification function is a geometric transform.
16. The method of claim 1 further comprising maintaining said digital images in one of the frequency domain and the space-frequency domain during one or more of said transforming, determining, specifying, determining, computing, altering, and iterating steps.
17. The method of claim 1 further comprising continuously maintaining said digital images in one of frequency domain and space-frequency domain during all of said transforming, determining, specifying, determining, computing, altering, and iterating steps.
18. The method of claim 1 further comprising continuously maintaining said digital images in spatial domain during all of said transforming, determining, specifying, determining, computing, altering, and iterating steps.
19. The method of claim 1 wherein said zonal features are spatial neighborhood properties.
20. The method of claim 1 wherein said sets of global features are intensity histograms.
21. The method of claim 1 further comprising optimizing said altering.
22. The method of claim 21 wherein said optimizing utilizes a multidimensional direction set.
23. The method of claim 21 wherein said optimizing utilizes windowing and gradient decent.
24. The method of claim 1 wherein said first and second digital images are products of different modalities.
25. The method of claim 1 wherein at least one of said modalities is selected from the group consisting of: ultrasound images, infrared images, visible light image, X-ray images, MRI images, and PET images.
26. The method of claim 1 wherein said first and second images differ in at least one of: image dimension, pixel bit depth, noise characteristics, and spatial sampling density.
27. A system for registering first and second digital images captured using first and second different capture modalities, said method comprising the steps of:
means for transforming said second digital image in the second modality to be more like said first digital image in the first modality using a modification function, to provide a modified second digital image;
means for determining a set of first features of each of said first and modified second digital images;
means for specifying mutually exclusive zones in each of said first and modified second digital images where each of the specified mutually exclusive zones is less than an entirety of each of said first and modified second digital images;
means for determining a set of zonal features of each of said specified mutually exclusive zones of each of said first and modified second digital images, said zonal features being different than said first features;
means for computing compound mutual information from said first features and said zonal features where compound mutual information comprises an aggregate measure of mutual information;
means for altering said modification function; and
means for iterating said transforming, determining, specifying, determining, computing, and altering steps until said compound mutual information is maximized,
wherein said means for computing further comprises:
(1) means for extracting a set of global marginal densities from a set of global features and a set of zonal marginal densities from said set of zonal features, and
(2) means for applying weighting coefficients to said sets of marginal densities to provided weighted marginal mutual information, and aggregating said weighted marginal mutual information in accordance with the formula:
I
(
A
,
B
)
=
∑
m
=
1
M
(
λ
m
∑
h
a
h
b
(
p
(
h
a
m
,
h
b
m
)
log
p
(
h
a
m
,
h
b
m
)
p
(
h
a
m
)
p
(
h
b
m
)
)
)
where:
A and a signify the first digital image;
B and b signify the modified second digital image;
M is the total number of sets of global and zonal features;
p(h a m ), p(h b m ) and p(h a m , h b m ) are marginal densities derived from image features {h m } m=1 M ; and
λ m are weighting coefficients, where
∑
m
λ
m
=
1.
28. The method of claim 27 further comprising, following said iterating, using said modified second digital image in place of said second digital image.
29. A computer program product for registering first and second digital images captured using first and second different capture modalities, the computer program product comprising computer readable storage medium having a computer program stored thereon for performing the steps of:
transforming said second digital image in the second modality to be more like said first digital image in the first modality using a modification function, to provide a modified second digital image;
determining a set of first features of each of said first and modified second digital images;
specifying mutually exclusive zones in each of said first and modified second digital images where each specified mutually exclusive zone is less than an entirety of each of said first and modified second digital images;
determining a set of zonal features of each of said specified mutually exclusive zones of each of said first and modified second digital images;
computing compound mutual information from said first features and said zonal features where compound mutual information comprises an aggregate measure of mutual information;
altering said modification function; and
iterating said transforming, determining, specifying, determining, computing, and altering steps until said compound mutual information is maximized,
wherein said computing further comprises:
(1) extracting a set of global marginal densities from a set of global features and a set of zonal marginal densities from said set of zonal features, and
(2) applying weighting coefficients to said sets of marginal densities to provided weighted marginal mutual information, and aggregating said weighted marginal mutual information in accordance with the formula:
I
(
A
,
B
)
=
∑
m
=
1
M
(
λ
m
∑
h
a
h
b
(
p
(
h
a
m
,
h
b
m
)
log
p
(
h
a
m
,
h
b
m
)
p
(
h
a
m
)
p
(
h
b
m
)
)
)
where:
A and a signify the first digital image;
B and b signify the modified second digital image;
M is the total number of sets of global and zonal features;
p(h a m ), p(h b m ) and p(h a m , h b m ) are marginal densities derived from image features {h m } m=1 M ; and
λ m are weighting coefficients, where
∑
m
λ
m
=
1.
30. The method of claim 1 wherein said first and second different modalities comprise an X-Ray image capture modality and a magnetic resonance image capture modality.