Method and system for registering images containing anatomical structures
A method for registration between a first three-dimensional image acquired according to a first acquisition mode, and including anatomical structures of a patient, and a second two-dimensional image, acquired according to a second acquisition mode by a rotatable and translatable device, the second image including a portion of the anatomical structures of the patient. The registration implements a rigid spatial transformation defined by rotation and translation parameters. The method includes automatically detecting the anatomical structures in the two-dimensional image by applying a first detection neural network trained on a generic database; estimating, from the anatomical structures automatically detected in the second two-dimensional image, by applying at least one classification neural network trained beforehand on a generic database, the rotation and translation parameters of the rigid spatial transformation; and 3D/2D iconic registration between the first three-dimensional image and the second two-dimensional image starting from an initialization with the rigid spatial transformation.
1. A method for registration between a first three-dimensional image acquired according to a first acquisition mode, and comprising anatomical structures of a patient, and a second two-dimensional image, acquired according to a second acquisition mode by an image acquisition device mounted on a scoping arch movable in rotation and in translation, the second image comprising a portion of the anatomical structures of said patient, the registration implementing a rigid spatial transformation defined by rotation and translation parameters,
the method being implemented by a processor of a programmable electronic device and comprising:
automatic detection of the anatomical structures in the two-dimensional image by application of a first detection neural network trained on a generic database,
estimation, from the anatomical structures automatically detected in said second two-dimensional image, by application of at least one classification neural network trained beforehand on a generic database, of the rotation and translation parameters of said rigid spatial transformation, said estimation including:
a first estimation, from said anatomical structures automatically detected in said second two-dimensional image, by application of said at least one classification neural network, of a first angle of rotation and of a second angle of rotation of said rigid spatial transformation, characterizing the position of the scoping arch, and
a second estimation of translational parameters, of a third angle of rotation and of a zoom factor of said rigid spatial transformation, the second estimation using a result of the first estimation, and
3D/2D iconic registration between the first three-dimensional image and the second two-dimensional image starting from an initialization with said rigid spatial transformation.
2. The method according to claim 1 , wherein the anatomical structures are classified into a plurality of predetermined anatomical structure categories, and wherein the estimation of the parameters of the rigid transformation includes the implementation of a second classification neural network per anatomical structure category to obtain transformation parameters estimated per anatomical structure category.
3. The method according to claim 1 , wherein a classification neural network is applied per anatomical structure category to obtain a first angle of rotation and a second angle of rotation estimated per anatomical structure category, the method further including a stitching of the first angles of rotation estimated per anatomical structure category to obtain said estimated first angle of rotation, and a stitching of the second angles of rotation estimated per anatomical structure category to obtain said estimated second parameter.
4. The method according to claim 3 , wherein each stitch consists of an average value calculation or of a median value calculation.
5. The method according to claim 1 , wherein the second estimation comprises:
a generation of a third two-dimensional image obtained from the first three-dimensional image by applying said first and second angles of rotation,
an automatic detection of anatomical structures on said third image, and
a pairing between the structures of said second and third two-dimensional images to obtain said translation parameters, the third angle of rotation and the zoom factor of said rigid spatial transformation.
6. The method according to claim 5 , wherein the detection of anatomical structures in said third two-dimensional image is performed by application of said first detection neural network.
7. The method according to claim 1 , wherein said anatomical structures are bone structures obtained on a second two-dimensional image acquired by fluoroscopy.
8. The method according to claim 1 , wherein said anatomical structures are vascular structures obtained on a second two-dimensional image acquired by angiography.
9. The method according to claim 1 , wherein the first three-dimensional image is acquired according to a first acquisition mode among tomography, MRI, 3D sonography or cone beam computed tomography.
10. A device for registration between a first three-dimensional image acquired according to a first acquisition mode, and comprising anatomical structures of a patient, and a second two-dimensional image, acquired according to a second acquisition mode by an image acquisition device mounted on a scoping arch movable in rotation and in translation, the second image comprising a portion of the anatomical structures of said patient, the registration implementing a rigid spatial transformation defined by rotation and translation parameters,
the device including a processor configured to implement:
automatic detection of the anatomical structures in the two-dimensional image by application of a first detection neural network trained on a generic database,
estimation, from the anatomical structures automatically detected in said second two-dimensional image, by application of at least one classification neural network trained beforehand on a generic database, of the rotation and translation parameters of said rigid spatial transformation, said estimation including:
a first estimation, from said anatomical structures automatically detected in said second two-dimensional image, by application of said at least one classification neural network, of a first angle of rotation and of a second angle of rotation of said rigid spatial transformation, characterizing the position of the scoping arch, and
a second estimation of translational parameters, of a third angle of rotation and of a zoom factor of said rigid spatial transformation, the second estimation using a result of the first estimation, and
3D/2D iconic registration between the first three-dimensional image and the second two-dimensional image starting from an initialization with said rigid spatial transformation.
11. A system for registration between a first three-dimensional image acquired according to a first acquisition mode, and comprising anatomical structures of a patient, and a second two-dimensional image, acquired according to a second acquisition mode, the system comprising:
an image acquisition device mounted on a scoping arch movable in rotation and in translation, configured to acquire the second two-dimensional image, the second two-dimensional image comprising a portion of the anatomical structures of said patient, and
a device for registration between the first three-dimensional image and the second two-dimensional image, the registration implementing a rigid spatial transformation defined by rotation and translation parameters, the device including a processor configured to implement:
automatic detection of the anatomical structures in the two-dimensional image by application of a first detection neural network trained on a generic database,
estimation, from the anatomical structures automatically detected in said second two-dimensional image, by application of at least one classification neural network trained beforehand on a generic database, of the rotation and translation parameters of said rigid spatial transformation, said estimation including:
a first estimation, from said anatomical structures automatically detected in said second two-dimensional image, by application of said at least one classification neural network, of a first angle of rotation and of a second angle of rotation of said rigid spatial transformation, characterizing the position of the scoping arch, and
a second estimation of translational parameters, of a third angle of rotation and of a zoom factor of said rigid spatial transformation, the second estimation using a result of the first estimation, and
3D/2D iconic registration between the first three-dimensional image and the second two-dimensional image starting from an initialization with said rigid spatial transformation.