Method and system for pericardium based model fusion of pre-operative and intra-operative image data for cardiac interventions
View Patent ↗A method and system for model based fusion pre-operative image data, such as computed tomography (CT), and intra-operative C-arm CT is disclosed. A first pericardium model is segmented in the pre-operative image data and a second pericardium model is segmented in a C-arm CT volume. A deformation field is estimated between the first pericardium model and the second pericardium model. A model of a target cardiac structure, such as a heart chamber model or an aorta model, extracted from the pre-operative image data is fused with the C-arm CT volume based on the estimated deformation field between the first pericardium model and the second pericardium model. An intelligent weighted average may be used improve the model based fusion results using models of the target cardiac structure extracted from pre-operative image data of patients other than a current patient.
1. A method for fusion of a model of a target cardiac structure extracted from a first medical image of a patient acquired using a first imaging modality with a second medical image of the patient acquired using a second imaging modality, comprising:
segmenting a first pericardium model in the first medical image;
segmenting a second pericardium model in the second medical image;
estimating a deformation field between the first pericardium model and the second pericardium model; and
fusing the model of the target cardiac structure extracted from the first medical image with the second medical image based on the estimated deformation field between the first pericardium model and the second pericardium model, wherein the target cardiac structure is a cardiac structure other than the pericardium and the pericardium is used as anchor structure to fuse the target cardiac structure to the second medical image.
2. The method of claim 1 , wherein the first medical image is a pre-operative image and the second medical image is an intra-operative image acquired at the time of a cardiac intervention.
3. The method of claim 1 , wherein the first medical image is a computed tomography volume and the second medical image is a C-arm computed tomography volume.
4. The method of claim 1 , wherein estimating a deformation field between the first pericardium model and the second pericardium model comprises:
estimating a deformation field between the first pericardium model and the second pericardium model using a thin plate spline (TPS) model.
5. The method of claim 1 , wherein fusing the model of the target cardiac structure extracted from the first medical image with the second medical image based on the estimated deformation field between the first pericardium model and the second pericardium model comprises:
generating a fused model of the target cardiac structure by transforming the model of the target cardiac structure extracted from the first medical image to the second medical image using the estimated deformation field.
6. The method of claim 1 , wherein fusing the model of the target cardiac structure extracted from the first medical image with the second medical image based on the estimated deformation field between the first pericardium model and the second pericardium model comprises:
generating a patient-specific aligned target model by transforming the model of the target cardiac structure extracted from the first medical image to the second medical image using the estimated deformation field; and
calculating a respective weight for each of a plurality of aligned target models, the plurality of aligned target models including the patient-specific aligned target model and one or more aligned target models generated from models of the target cardiac structure extracted from medical images of other patients acquired using the first medical imaging modality; and
generating a fused model of the target cardiac structure in the second medical image as a weighted average of the plurality of aligned target models using the respective weight generated for each of the plurality of aligned target models.
7. The method of claim 6 , wherein calculating a respective weight for each of a plurality of aligned target models comprises:
for each of the aligned target models, calculating a distance measure between a corresponding pericardium model segmented in a corresponding medical image acquired using the first medical imaging modality and the second pericardium model segmented in the second medical image; and
determining the respective weight for each of the plurality of aligned target models based on the calculated distance measure between the corresponding pericardium model and the second pericardium model.
8. A method for fusing a target anatomical structure from a first medical imaging modality to a second medical imaging modality using a plurality of target models of the target anatomical structure, each extracted from a corresponding first medical image acquired using the first medical imaging modality, and a plurality of anchor models of an anchor anatomical structure, each extracted from a corresponding first medical image, the method comprising:
aligning each of the plurality of target models to a second medical image of a current patient acquired using the second medical imaging modality using a deformation field calculated between a corresponding one of the plurality of anchor models and a model of the anchor anatomical structure segmented in the second medical image, resulting in a plurality of aligned target models;
calculating a respective weight for each of the plurality of aligned target models based on a distance measure between the corresponding one of the plurality of anchor models and the model of the anchor anatomical structure segmented in the second medical image; and
generating a fused model of the target anatomical structure in the second medical image as a weighted average of the plurality of aligned target models using the respective weight calculated for each of the plurality of aligned target models.
9. The method of claim 8 , wherein calculating a respective weight for each of the plurality of aligned target models based on a distance measure between the corresponding one of the plurality of anchor models and the model of the anchor anatomical structure segmented in the second medical image comprises:
for each of the plurality of aligned target models, calculating a weight w i as:
w
i
=
1
-
d
i
-
d
min
d
max
-
d
min
,
where d i , is the distance measure between the corresponding on the plurality of anchor models and model of the anchor anatomical structure segmented in the medical image, d min is a minimum distance measure, and d max is a maximum distance measure.
10. The method of claim 8 , wherein one of the plurality of target models and a corresponding one of the plurality of anchor models are extracted from a first medical image of the current patient.
11. The method of claim 10 , wherein generating a fused model of the target anatomical structure in the second medical image as a weighted average of the plurality of aligned target models using the respective weight calculated for each of the plurality of aligned target models comprises:
generating the fused model α as:
a
=
Σ
i
=
0
n
w
i
a
i
Σ
i
=
0
n
w
i
,
where α 0 is the aligned target model extracted from the first medical image of the current patient, α 1 , . . . , α n are aligned target models extracted from first medical images of patients other than the current patient, and w i is the respective weight calculated for each of the aligned target models.
12. The method of claim 10 , wherein generating a fused model of the target anatomical structure in the second medical image as a weighted average of the plurality of aligned target models using the respective weight calculated for each of the plurality of aligned target models comprises:
generating the fused model α as:
a
=
β
a
0
+
(
1
-
β
)
Σ
i
=
1
n
w
i
a
i
β
+
(
1
-
β
)
Σ
i
=
1
n
w
i
,
where α 0 is the aligned target model extracted from the first medical image of the current patient, α 1 , . . . , α n are aligned target models extracted from first medical images of patients other than the current patient, w i is the respective weight calculated for each of the aligned target models, and β is a parameter to tune the relative weighting between the aligned target model extracted from the first medical image of the current patient and the aligned target models extracted from first medical images of patients other than the current patient.
13. The method of claim 8 , wherein the each first medical image is a computed tomography image, the second medical image is a C-arm computed tomography image, the anchor anatomical structure is the pericardium, and the target anatomical structure is at least one of a heart chamber or an aorta.
14. An apparatus for fusion of a model of a target cardiac structure extracted from a first medical image of a patient acquired using a first imaging modality with a second medical image of the patient acquired using a second imaging modality, comprising:
means for segmenting a first pericardium model in the first medical image;
means for segmenting a second pericardium model in the second medical image;
means for estimating a deformation field between the first pericardium model and the second pericardium model; and
means for fusing the model of the target cardiac structure extracted from the first medical image with the second medical image based on the estimated deformation field between the first pericardium model and the second pericardium model, wherein the target cardiac structure is a cardiac structure other than the pericardium and the pericardium is used as anchor structure to fuse the target cardiac structure to the second medical image.
15. The method of claim 14 , wherein the first medical image is a computed tomography volume and the second medical image is a C-arm computed tomography volume.
16. The apparatus of claim 14 , wherein the means for fusing the model of the target cardiac structure extracted from the first medical image with the second medical image based on the estimated deformation field between the first pericardium model and the second pericardium model comprises:
means for generating a fused model of the target cardiac structure by transforming the model of the target cardiac structure extracted from the first medical image to the second medical image using the estimated deformation field.
17. The apparatus of claim 14 , wherein the means for fusing the model of the target cardiac structure extracted from the first medical image with the second medical image based on the estimated deformation field between the first pericardium model and the second pericardium model comprises:
means for generating a patient-specific aligned target model by transforming the model of the target cardiac structure extracted from the first medical image to the second medical image using the estimated deformation field; and
means for calculating a respective weight for each of a plurality of aligned target models, the plurality of aligned target models including the patient-specific aligned target model and one or more aligned target models generated from models of the target cardiac structure extracted from medical images of other patients acquired using the first medical imaging modality; and
means for generating a fused model of the target cardiac structure in the second medical image as a weighted average of the plurality of aligned target models using the respective weight generated for each of the plurality of aligned target models.
18. The apparatus of claim 17 , wherein the means for calculating a respective weight for each of a plurality of aligned target models comprises:
means for determining the respective weight for each of the plurality of aligned target models based on a distance measure between a corresponding pericardium model segmented in a corresponding medical image acquired using the first medical imaging modality and the second pericardium model segmented in the second medical image.
19. An apparatus for fusing a target anatomical structure from a first medical imaging modality to a second medical imaging modality using a plurality of target models of the target anatomical structure, each extracted from a corresponding first medical image acquired using the first medical imaging modality, and a plurality of anchor models of an anchor anatomical structure, each extracted from a corresponding first medical image, comprising:
means for aligning each of the plurality of target models to a second medical image of a current patient acquired using the second medical imaging modality using a deformation field calculated between a corresponding one of the plurality of anchor models and a model of the anchor anatomical structure segmented in the second medical image, resulting in a plurality of aligned target models;
means for calculating a respective weight for each of the plurality of aligned target models based on a distance measure between the corresponding one of the plurality of anchor models and the model of the anchor anatomical structure segmented in the second medical image; and
means for generating a fused model of the target anatomical structure in the second medical image as a weighted average of the plurality of aligned target models using the respective weight calculated for each of the plurality of aligned target models.
20. The apparatus of claim 19 , wherein one of the plurality of target models and a corresponding one of the plurality of anchor models are extracted from a first medical image of the current patient.
21. The apparatus of claim 19 , wherein the each first medical image is a computed tomography image, the second medical image is a C-arm computed tomography image, the anchor anatomical structure is the pericardium, and the target anatomical structure is at least one of a heart chamber or an aorta.
22. A non-transitory computer readable medium storing computer program instructions for fusion of a model of a target cardiac structure extracted from a first medical image of a patient acquired using a first imaging modality with a second medical image of the patient acquired using a second imaging modality, the computer program instructions when executed on a processor causing the processor to perform operations comprising:
segmenting a first pericardium model in the first medical image;
segmenting a second pericardium model in the second medical image;
estimating a deformation field between the first pericardium model and the second pericardium model; and
fusing the model of the target cardiac structure extracted from the first medical image with the second medical image based on the estimated deformation field between the first pericardium model and the second pericardium model, wherein the target cardiac structure is a cardiac structure other than the pericardium and the pericardium is used as anchor structure to fuse the target cardiac structure to the second medical image.
23. The non-transitory computer readable medium of claim 22 , wherein the first medical image is a computed tomography volume and the second medical image is a C-arm computed tomography volume.
24. The non-transitory computer readable medium of claim 22 , wherein fusing the model of the target cardiac structure extracted from the first medical image with the second medical image based on the estimated deformation field between the first pericardium model and the second pericardium model comprises:
generating a fused model of the target cardiac structure by transforming the model of the target cardiac structure extracted from the first medical image to the second medical image using the estimated deformation field.
25. The non-transitory computer readable medium of claim 22 , wherein fusing the model of the target cardiac structure extracted from the first medical image with the second medical image based on the estimated deformation field between the first pericardium model and the second pericardium model comprises:
generating a patient-specific aligned target model by transforming the model of the target cardiac structure extracted from the first medical image to the second medical image using the estimated deformation field; and
calculating a respective weight for each of a plurality of aligned target models, the plurality of aligned target models including the patient-specific aligned target model and one or more aligned target models generated from models of the target cardiac structure extracted from medical images of other patients acquired using the first medical imaging modality; and
generating a fused model of the target cardiac structure in the second medical image as a weighted average of the plurality of aligned target models using the respective weight generated for each of the plurality of aligned target models.
26. The non-transitory computer readable medium of claim 25 , wherein calculating a respective weight for each of a plurality of aligned target models comprises:
for each of the aligned target models, calculating a distance measure between a corresponding pericardium model segmented in a corresponding medical image acquired using the first medical imaging modality and the second pericardium model segmented in the second medical image; and
determining the respective weight for each of the plurality of aligned target models based on the calculated distance measure between the corresponding pericardium model and the second pericardium model.
27. A non-transitory computer readable medium storing computer program instructions for fusing a target anatomical structure from a first medical imaging modality to a second medical imaging modality using a plurality of target models of the target anatomical structure, each extracted from a corresponding first medical image acquired using the first medical imaging modality, and a plurality of anchor models of an anchor anatomical structure, each extracted from a corresponding first medical image, the computer program instructions when executed on a processor causing the processor to perform operations comprising:
aligning each of the plurality of target models to a second medical image of a current patient acquired using the second medical imaging modality using a deformation field calculated between a corresponding one of the plurality of anchor models and a model of the anchor anatomical structure segmented in the second medical image, resulting in a plurality of aligned target models;
calculating a respective weight for each of the plurality of aligned target models based on a distance measure between the corresponding one of the plurality of anchor models and the model of the anchor anatomical structure segmented in the second medical image; and
generating a fused model of the target anatomical structure in the second medical image as a weighted average of the plurality of aligned target models using the respective weight calculated for each of the plurality of aligned target models.
28. The non-transitory computer readable medium of claim 27 , wherein calculating a respective weight for each of the plurality of aligned target models based on a distance measure between the corresponding one of the plurality of anchor models and the model of the anchor anatomical structure segmented in the second medical image comprises:
for each of the plurality of aligned target models, calculating a weight w i as:
w
i
=
1
-
d
i
-
d
min
d
max
-
d
min
,
where d i , is the distance measure between the corresponding on the plurality of anchor models and model of the anchor anatomical structure segmented in the medical image, d min is a minimum distance measure, and d max is a maximum distance measure.
29. The non-transitory computer readable medium of claim 27 , wherein one of the plurality of target models and a corresponding one of the plurality of anchor models are extracted from a first medical image of the current patient.
30. The non-transitory computer readable medium of claim 29 , wherein generating a fused model of the target anatomical structure in the second medical image as a weighted average of the plurality of aligned target models using the respective weight calculated for each of the plurality of aligned target models comprises:
generating the fused model α as:
a
=
Σ
i
=
0
n
w
i
a
i
Σ
i
=
0
n
w
i
,
where α 0 is the aligned target model extracted from the first medical image of the current patient, α 1 , . . . , α n are aligned target models extracted from first medical images of patients other than the current patient, and w i is the respective weight calculated for each of the aligned target models.
31. The non-transitory computer readable medium of claim 29 , wherein generating a fused model of the target anatomical structure in the second medical image as a weighted average of the plurality of aligned target models using the respective weight calculated for each of the plurality of aligned target models comprises:
generating the fused model α as:
a
=
β
a
0
+
(
1
-
β
)
Σ
i
=
1
n
w
i
a
i
β
+
(
1
-
β
)
Σ
i
=
1
n
w
i
,
where α 0 is the aligned target model extracted from the first medical image of the current patient, α 1 , . . . , α n are aligned target models extracted from first medical images of patients other than the current patient, w i is the respective weight calculated for each of the aligned target models, and β is a parameter to tune the relative weighting between the aligned target model extracted from the first medical image of the current patient and the aligned target models extracted from first medical images of patients other than the current patient.
32. The non-transitory computer readable medium of claim 27 , wherein the each first medical image is a computed tomography image, the second medical image is a C-arm computed tomography image, the anchor anatomical structure is the pericardium, and the target anatomical structure is at least one of a heart chamber or an aorta.