Multi-layer image registration
Systems/techniques that facilitate multi-layer image registration are provided. In various embodiments, a system can access a first image and a second image. In various aspects, the system can generate, via execution of a machine learning model on the first image and the second image, a plurality of registration fields and a plurality of weight matrices that respectively correspond to the plurality of registration fields. In various instances, the system can register the first image with the second image based on the plurality of registration fields and the plurality of weight matrices.
1. A system, comprising:
a processor that executes computer-executable components stored in a computer-readable memory, the computer-executable components comprising:
a receiver component that accesses:
a first image of a portion of a patient's body captured using a first energy level and a first exposure time, wherein the first image has a defined number of pixels in a defined arrangement, and
a second image of the portion of the patient's body captured using a second energy level and a second exposure time, wherein the second image has the defined number of pixels in the defined arrangement, and wherein at least one of the second energy level is different from the first energy level, or the second exposure time is different from the first exposure time;
a field component that generates, via execution of a machine learning model on the first image and the second image, a plurality of registration fields and a plurality of weight matrices that respectively correspond to the plurality of registration fields, wherein each registration field of the plurality of registration fields comprises respective pixel-wise shift vectors corresponding to each pixel of the first image, and wherein each weight matrix of the plurality of weight matrices comprises respective weight scalars corresponding to each pixel-wise shift vector of the corresponding registration field; and
a registration component that generates a registered image of the portion of the patient's body from the first image with the second image based on the plurality of registration fields and the plurality of weight matrices.
2. The system of claim 1 , wherein the respective weight scalars indicate respective levels of importance of the pixel-wise shift vectors.
3. The system of claim 1 , wherein the machine learning model is a deep learning neural network.
4. The system of claim 1 , wherein a registration field of the plurality of registration fields is rigid or deformable.
5. The system of claim 1 , wherein the registration component generates the registered image by:
applying the plurality of registration fields to the first image, thereby yielding a plurality of registered image layers; and
determining a weighted sum of the plurality of registered image layers according to the plurality of weight matrices.
6. The system of claim 1 , wherein the registration component generates the registered image by:
applying the plurality of weight matrices to the first image, thereby yielding a plurality of weighted image layers;
applying the plurality of registration fields to the plurality of weighted image layers, thereby yielding a plurality of weighted and registered image layers; and
determining a sum of the plurality of weighted and registered image layers.
7. The system of claim 1 , wherein the computer-executable components further comprise:
a training component that trains the machine learning model based on an error between the second image and the registered image.
8. The system of claim 1 , wherein the computer-executable components further comprise:
a training component that trains the machine learning model based on at least one first error between at least one of the plurality of registration fields and at least one ground-truth registration field, or based on at least one second error between at least one of the plurality of weight matrices and at least one ground-truth weight matrix.
9. A computer-implemented method, comprising:
accessing, by a device operatively coupled to a processor;
a first image of a portion of a patient's body captured using a first energy level and a first exposure time, wherein the first image has a defined number of pixels in a defined arrangement, and
a second image of the portion of the patient's body captured using a second energy level and a second exposure time, wherein the second image has the defined number of pixels in the defined arrangement, and wherein at least one of the second energy level is different from the first energy level, or the second exposure time is different from the first exposure time;
generating, by the device via execution of a machine learning model on the first image and the second image, a plurality of registration fields and a plurality of weight matrices that respectively correspond to the plurality of registration fields, wherein each registration field of the plurality of registration fields comprises respective pixel-wise shift vectors corresponding to each pixel of the first image, and wherein each weight matrix of the plurality of weight matrices comprises respective weight scalars corresponding to each pixel-wise shift vector of the corresponding registration field; and
generating, by the device, a registered image of the portion of the patient's body from the first image with the second image based on the plurality of registration fields and the plurality of weight matrices.
10. The computer-implemented method of claim 9 , wherein the respective weight scalars indicate respective levels of importance of the pixel-wise shift vectors.
11. The computer-implemented method of claim 9 , wherein the machine learning model is a deep learning neural network.
12. The computer-implemented method of claim 9 , wherein a registration field of the plurality of registration fields is rigid or deformable.
13. The computer-implemented method of claim 9 , wherein the generating the registered comprises:
applying the plurality of registration fields to the first image, thereby yielding a plurality of registered image layers; and
determining a weighted sum of the plurality of registered image layers according to the plurality of weight matrices.
14. The computer-implemented method of claim 9 , wherein the generating the registered comprises:
applying, by the device, the plurality of weight matrices to the first image, thereby yielding a plurality of weighted image layers;
applying, by the device, the plurality of registration fields to the plurality of weighted image layers, thereby yielding a plurality of weighted and registered image layers; and
determining, by the device, a sum of the plurality of weighted and registered image layers.
15. The computer-implemented method of claim 9 , further comprising:
training, by the device, the machine learning model based on an error between the second image and the registered image.
16. The computer-implemented method of claim 9 , further comprising:
training, by the device, the machine learning model based on at least one first error between at least one of the plurality of registration fields and at least one ground-truth registration field, or based on at least one second error between at least one of the plurality of weight matrices and at least one ground-truth weight matrix.
17. A computer program product for facilitating multi-layer image registration, the computer program product comprising a computer-readable memory having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
access a first image of a portion of a patient's body captured using a first energy level and a first exposure time, wherein the first image has a defined number of pixels in a defined arrangement;
access a second image of the portion of the patient's body captured using a second energy level and a second exposure time, wherein the second image has the defined number of pixels in the defined arrangement, and wherein at least one of the second energy level is different from the first energy level, or the second exposure time is different from the first exposure time;
generate, via execution of a machine learning model on the first image and the second image, a plurality of registration fields and a plurality of weight matrices that respectively correspond to the plurality of registration fields, wherein each registration field of the plurality of registration fields comprises respective pixel-wise shift vectors corresponding to each pixel of the first image, and wherein each weight matrix of the plurality of weight matrices comprises respective weight scalars corresponding to each pixel-wise shift vector of the corresponding registration field; and
generate a registered image of the portion of the patient's body from the first image with the second image based on the plurality of registration fields and the plurality of weight matrices.
18. The computer program product of claim 17 , wherein the generating the registered comprises:
applying the plurality of registration fields to the first image, thereby yielding a plurality of registered image layers; and
executing another machine learning model on the plurality of registered image layers.
19. The computer program product of claim 17 , wherein the plurality of registration fields comprises at least one rigid registration field and at least one deformable registration field.
20. The computer program product of claim 17 , wherein a registration field of the plurality of registration fields is rigid or deformable.