Medical imaging diffeomorphic registration based on machine learning
For registration of medical images with deep learning, a neural network is designed to include a diffeomorphic layer in the architecture. The network may be trained using supervised or unsupervised approaches. By enforcing the diffeomorphic characteristic in the architecture of the network, the training of the network and application of the learned network may provide for more regularized and realistic registration.
1. A method for registration with a medical imaging system, the method comprising:
acquiring first and second sets of scan data representing a patient;
determining displacements of the registration of the scan data of the first set with the scan data of the second set, the displacements being determined by input of the first and second sets of the scan data to a deep machine-learned network, the deep machine-learned network having a plurality of first layers outputting velocities in response to the input and having a second layer outputting the displacements in response to the velocities from the first layers, the second layer being a diffeomorphic layer such that the displacements output by the deep machine-learned network are diffeomorphic; and
generating an image of the patient from the displacements.
2. The method of claim 1 wherein acquiring comprises scanning the patient with different modalities and/or at different times.
3. The method of claim 1 wherein determining comprises determining with the diffeomorphic layer of the deep machine-learned network comprising an integration layer configured to integrate velocities, the integration layer separate from the first layers.
4. The method of claim 3 wherein determining comprises determining with the integration layer configured to exponentiate the velocities through scaling and squaring.
5. The method of claim 1 wherein determining comprises determining with the deep machine-learned network comprising a fully convolutional encoder-decoder network formed by the first layers, the fully convolutional encoder-decoder network having been trained to output the velocities by location from the scan data of the first and second sets and the diffeomorphic layer following the first layers of the fully convolutional encoder-decoder network configured to receive the velocities and output the displacements from the velocities.
6. The method of claim 1 wherein generating the image comprises generating a warped image from the determined displacements.
7. The method of claim 1 wherein determining comprises determining with the deep machine-learned network comprising a sequence of two or more diffeomorphic networks at different scales.
8. The method of claim 1 wherein determining comprises determining with the deep machine-learned network having been trained with ground truth deformations in supervised learning.
9. The method of claim 1 wherein determining comprises determining with the deep machine-learned network having been trained with a similarity metric in unsupervised learning.
10. The method of claim 9 wherein determining comprises determining with the deep machine-learned network having been trained with the similarity metric and a regularizer.
11. The method of claim 9 wherein determining comprises determining with the deep machine-learned network having been trained with the similarity metric comprising a differentiable metric.
12. A medical imaging system for image registration, the medical imaging system comprising:
a medical scanner configured to generate a first image from a scan a patient;
an image processor configured to apply a machine-learned neural network to the first image and a second image, the machine-learned neural network having been trained to generate, by a network architecture being trained to become the machine-learned neural network, a warped image using velocities estimated from the first and second images with generation of the velocities by the network architecture and trained using a diffeomorphic deformation field determined from the velocities by the network architecture; and
a display configured to display the warped image or an image of the velocities.
13. The medical imaging system of claim 12 wherein the machine-learned neural network generates the warped image in a single forward evaluation.
14. The medical imaging system of claim 12 wherein the machine-learned neural network comprises an encoder-decoder network having been trained to estimate the velocities and an integration layer configured to generate the diffeomorphic deformation field from the velocities.
15. The medical imaging system of claim 14 wherein the machine-learned neural network further comprises a warping layer configured to generate the warped image from the diffeomorphic deformation field.
16. The medical imaging system of claim 12 wherein the machine-learned neural network comprises a multi-scale network.
17. The medical imaging system of claim 12 wherein the machine-learned neural network was trained with unsupervised learning using a pre-training defined similarity metric.