MR-levelcheck-2: method for localization of structures in projection images
An embodiment in accordance with the present invention provides a technique for localizing structures of interest in projection images (e.g., x-ray projection radiographs or fluoroscopy) based on structures defined in a preoperative 3D image (e.g., MR or CT). Applications include, but are not limited to, spinal interventions. The present invention achieves 3D-2D image registration (and particularly allowing use with a preoperative MR image) by segmenting the structures of interest in the preoperative 3D image and generating a simulated projection of the segmented structures to be aligned with the 2D projection image. Other applications include various clinical scenarios involving 3D-2D image registration, such as image-guided cranial neurosurgery, orthopedic surgery, biopsy, and radiation therapy.
1. A method, comprising:
receiving, by a device, 3D magnetic resonance image data in the form of a preoperative 3D magnetic resonance image of a subject;
segmenting, by the device, structures of interest in the preoperative 3D magnetic resonance image of the subject,
the structures of interest being segmented by using an automatic segmentation algorithm, and
the structures of interest being segmented to mitigate non-correspondence of anatomical features between the 3D magnetic resonance image data and 2D image data,
the 3D magnetic resonance image data and the 2D image data being acquired from different imaging modalities;
generating, by the device, a simulated projection of the segmented structures of interest to be aligned with 2D image data of the subject;
providing, by the device and for display, a visual representation of the structures of interest; and
providing, by the device and for display, a visual representation of the 3D magnetic resonance image data registered onto the 2D image data of the subject.
2. The method of claim 1 , further comprising:
registering the 3D magnetic resonance image data onto the 2D image data of the subject for a spine.
3. The method of claim 1 , further comprising:
segmenting the 3D magnetic resonance image data with an objective function defined by:
min
u
(
x
)
∈
{
0
,
1
}
〈
1
-
u
,
D
1
〉
+
〈
u
,
D
2
〉
+
∫
g
(
x
)
∇
u
(
x
)
dx
.
4. The method of claim 1 , further comprising:
segmenting the 3D magnetic resonance image data with one chosen from a group consisting of:
projecting MRI intensities within each segmented vertebrae;
dilating the segmentation to include a region approximating a bony cortex; and
projecting a binary region and projecting a vertebral body dilated to include the bony cortex.
5. The method of claim 1 , further comprising:
adding labels to the visual representation of the 3D magnetic resonance image registered onto the 2D image data.
6. The method of claim 1 , further comprising:
applying annotations within the preoperative 3D magnetic resonance image of the subject to mark locations of the structures of interest.
7. The method of claim 6 , further comprising:
initializing segmentation using the annotations applied within the preoperative 3D magnetic resonance image of the subject.
8. The method of claim 1 , further comprising:
applying forward projection to the segmented structures of interest.
9. The method of claim 1 , where mitigating the non-correspondence of the anatomical features comprises:
mitigating the non-correspondence of the anatomical features using one selected from a group consisting of:
signal intensities, and
image gradients.
10. The method of claim 4 , wherein the MRI intensities are T2-weighted signal values.
11. A system, comprising:
one or more memories; and
one or more processors, communicatively coupled to the one or more memories, to:
receive 3D magnetic resonance image data in form of a preoperative 3D magnetic resonance image of a subject;
segment structures of interest in the preoperative 3D magnetic resonance image of the subject,
the structures of interest being segmented by using an automatic segmentation algorithm, and
the structures of interest being segmented to mitigate non-correspondence of anatomical features between the 3D magnetic resonance image data and 2D image data,
the 3D magnetic resonance image data and the 2D image data being acquired from different imaging modalities;
generate a simulated projection of the segmented structures to be aligned with 2D image data of the subject;
provide, for display, a visual representation of the structures of interest; and
provide, for display, a visual representation of the 3D magnetic resonance image data registered onto the 2D image data of the subject.
12. The system of claim 11 , where the one or more processors are further to:
register 3D magnetic resonance image data to 2D image data for a spine.
13. The system of claim 11 , where the one or more processors are to:
segment the 3D magnetic resonance image data with an objective function defined by:
min
u
(
x
)
∈
{
0
,
1
}
〈
1
-
u
,
D
1
〉
+
〈
u
,
D
2
〉
+
∫
g
(
x
)
∇
u
(
x
)
dx
.
14. The system of claim 11 , where the one or more processors are further to:
segment the 3D magnetic resonance image data with one chosen from a group consisting of:
projecting MRI intensities within each segmented vertebrae;
dilating the segmentation to include a region approximating a bony cortex;
projecting a binary region; and
projecting a vertebral body dilated to include the bony cortex.
15. The system of claim 11 , where the one or more processors are further to:
add labels to the visual representation of the 3D magnetic resonance image registered onto the 2D image data of the subject.
16. The system of claim 11 , where the one or more processors are further to:
apply annotations within the preoperative 3D magnetic resonance image to mark locations of the structures of interest.
17. The system of claim 16 , where the one or more processors are further to:
initialize segmentation using the annotations applied within the preoperative 3D magnetic resonance image.
18. The system of claim 11 , where the one or more processors are further to:
apply forward projection to the segmented structures of interest.
19. A method, comprising:
receiving, by a device, 3D magnetic resonance image data in the form of a preoperative 3D magnetic resonance image of a subject;
segmenting, by the device, structures of interest in the preoperative 3D magnetic resonance image of the subject;
segmenting, by the device, the 3D magnetic resonance image data with an objective function:
min
u
(
x
)
∈
{
0
,
1
}
〈
1
-
u
,
D
1
〉
+
〈
u
,
D
2
〉
+
∫
g
(
x
)
∇
u
(
x
)
dx
;
generating, by the device, a simulated projection of the segmented structures of interest to be aligned with 2D image data of the subject;
providing, by the device and for display, a visual representation of the structures of interest; and
providing, by the device and for display, a visual representation of the 3D magnetic resonance image data registered onto the 2D image data of the subject.
20. The method of claim 19 , further comprising:
adding labels to the visual representation of the 3D magnetic resonance image registered onto the 2D image data.
21. The method of claim 19 , where
the structures of interest are segmented to mitigate non-correspondence of anatomical features between the 3D magnetic resonance image data and the 2D image data, and
the 3D magnetic resonance image data and the 2D image data are acquired from different imaging modalities.
22. A system, comprising:
one or more memories;
one or more processors, communicatively coupled to the one or more memories, to:
receive 3D magnetic resonance image data in the form of a preoperative 3D magnetic resonance image of a subject;
segment structures of interest in the preoperative 3D magnetic resonance image of the subject;
segment the 3D magnetic resonance image data with an objective function:
min
u
(
x
)
∈
{
0
,
1
}
〈
1
-
u
,
D
1
〉
+
〈
u
,
D
2
〉
+
∫
g
(
x
)
∇
u
(
x
)
dx
;
generate a simulated projection of the segmented structures of interest to be aligned with 2D image data of the subject;
provide for display a visual representation of the structures of interest; and
provide for display a visual representation of the 3D magnetic resonance image data registered onto the 2D image data of the subject.