Processing 2-D projection images using a neural network
Techniques are described to infer 2-D segmentations of a region of interest using a neural network algorithm. Techniques are described to train the neural network algorithm. The 2-D segmentations are determined based on multiple 2-D projection images. For example, x-ray images can be used as an input.
1 . A method, comprising:
obtaining multiple 2-D projection images associated with multiple views of a scene;
determining, using at least one neural network algorithm and for the multiple 2-D projection images, multiple 2-D segmentations of a region of interest included in the scene, the multiple 2-D segmentations being associated with the multiple views; and
based on a predefined registration of the multiple views in a reference frame, determining an inter-view consistency between the multiple 2-D segmentations associated with the multiple views, wherein
the determining the inter-view consistency includes determining a distance between a first reference point of a first one of the multiple 2-D segmentations and a projection of a second reference point of a second one of the multiple 2-D segmentations into a view of the multiple views associated with the first one of the multiple 2-D segmentations,
the first reference point and the second reference point denote a corresponding feature of the region of interest, and
the projection of the second reference point is based on the predefined registration of the multiple views in the reference frame.
2 . The method of claim 1 ,
wherein said determining of the inter-view consistency is based on a fundamental matrix defining the predefined registration.
3 . The method of claim 2 , further comprising:
enforcing the inter-view consistency by adjusting the multiple 2-D segmentations.
4 . The method of claim 1 , further comprising:
enforcing the inter-view consistency by adjusting the multiple 2-D segmentations.
5 . The method of claim 1 ,
wherein the projection of the second reference point into the view associated with the first one of the multiple 2-D segmentations comprises an epipolar line defined in the view associated with the first one of the multiple 2-D segmentations.
6 . The method of claim 1 ,
wherein said determining of the multiple 2-D segmentations is based on a projection matrix defining the predefined registration.
7 . The method of claim 1 , further comprising:
determining a classification of an object defining the region of interest based on the multiple 2-D segmentations.
8 . The method of claim 1 , further comprising:
determining the predefined registration of the multiple views in the reference frame based on at least one of prior-knowledge of an appearance of the region of interest or a configuration of an imaging facility used to acquire the multiple 2-D projection images.
9 . The method of claim 1 , wherein the multiple 2-D projection images are x-ray images.
10 . A method, comprising:
obtaining multiple 2-D projection images associated with multiple views of a scene,
determining, using at least one neural network algorithm and based on the multiple 2-D projection images, a 3-D segmentation of a region of interest included in the scene, and
determining, based on a predefined registration of the multiple views in a reference frame and based on the 3-D segmentation, an inter-view consistency between multiple 2-D segmentations of the region of interest associated with the multiple views,
wherein the determining the inter-view consistency includes determining a distance between a first reference point of a first one of the multiple 2-D segmentations and a projection of a second reference point of a second one of the multiple 2-D segmentations into a view of the multiple views associated with the first one of the multiple 2-D segmentations,
the first reference point and the second reference point denote a corresponding feature of the region of interest, and
the protection of the second reference point is based on the predefined registration of the multiple views in the reference frame.
11 . The method of claim 10 ,
wherein said determining of the multiple 2-D segmentations is based on a projection matrix defining the predefined registration.
12 . The method of claim 10 , further comprising:
determining a classification of an object defining the region of interest based on the multiple 2-D segmentations.
13 . The method of claim 12 , further comprising:
determining the predefined registration of the multiple views in the reference frame based on at least one of prior-knowledge of an appearance of the region of interest or a configuration of an imaging facility used to acquire the multiple 2-D projection images.
14 . The method of claim 10 , further comprising:
determining the predefined registration of the multiple views in the reference frame based on at least one of prior-knowledge of an appearance of the region of interest or a configuration of an imaging facility used to acquire the multiple 2-D projection images.
15 . A device, comprising:
a processor configured to,
obtain multiple 2-D projection images associated with multiple views of a scene,
determine, using at least one neural network algorithm and for the multiple 2-D projection images, multiple 2-D segmentations of a region of interest included in the scene, the multiple 2-D segmentations being associated with the multiple views, and
based on a predefined registration of the multiple views in a reference frame, determine an inter-view consistency between the multiple 2-D segmentations associated with the multiple views, wherein
to determine the inter-view consistency, the processor is configured to determine a distance between a first reference point of a first one of the multiple 2-D segmentations and a projection of a second reference point of a second one of the multiple 2-D segmentations into a view of the multiple views associated with the first one of the multiple 2-D segmentations,
the first reference point and the second reference point is based on the predefined registration of the multiple views in the reference frame.
16 . A device, comprising:
a processor configured to,
obtain multiple 2-D projection images associated with multiple views of a scene,
determine, using at least one neural network algorithm and based on the multiple 2-D projection images, a 3-D segmentation of a region of interest included in the scene, and
determine, based on a predefined registration of the multiple views in a reference frame and based on the 3-D segmentation, an inter-view consistency between multiple 2-D segmentations of the region of interest associated with the multiple views, wherein
to determine the inter-view consistency, the processor is configured to determine a distance between a first reference point of a first one of the multiple 2-D segmentations and a projection of a second reference point of a second one of the multiple 2-D segmentations into a view of the multiple views associated with the first one of the multiple 2-D segmentations,
the first reference point and the second reference point denote a corresponding feature of the region of interest, and
the projection of the second reference point is based on the predefined registration of the multiple views in the reference frame.