Machine-learned network for Fourier transform in reconstruction for medical imaging
For low-complexity to learned reconstruction and/or learned Fourier transform-based operators for reconstruction, a neural network is used for the transform operators. The network architecture is modeled on the Cooley-Tukey fast Fourier transform (FFT) approach. By splitting input data before recursive calls in the network architecture, the network may be trained to perform the transform with similar complexity as FFT. The learned operators may be used in a trained network for reconstruction, such as with a learned iterative framework and image regularizer.
1. A method for converting data between scan space and image domain in an imaging system, the method comprising:
scanning, by the imaging system, an object, the scanning resulting in input data representing the patient;
performing a fast transform of the input data based on application of the input data to a machine-learned neural network, the machine-learned neural network comprising an architecture of a sequence of a convolutional splitting layer, parallel recursive deep networks, a diagonal linear layer, and a convolutional butterfly layer;
generating image domain data from an output of the machine-learned neural network; and
displaying an image from the image domain data,
wherein performing comprises performing with the recursive deep networks comprising nested radix fast Fourier transform networks at iteratively reducing scales.
2. The method of claim 1 wherein performing comprises performing with the architecture of the machine-learned network being structured as a radix fast Fourier transform.
3. The method of claim 1 wherein performing comprises performing with the convolutional spitting layer have a stride of R, R being two or greater.
4. The method of claim 1 wherein performing comprises performing with the nested radix fast Fourier transform networks having unshared weights.
5. The method of claim 4 wherein performing comprises performing with a number of output channels being equal to a number of a stride of the convolution splitting layer.
6. The method of claim 5 wherein performing comprises performing with the convolution splitting layer being two-dimensional, the number of output channels being four and the stride being four.
7. The method of claim 5 wherein performing comprises performing with a complexity of O(N log N) where N is a total number of samples of the image domain data.
8. The method of claim 1 wherein performing comprises performing with the diagonal layer being a twiddle layer.
9. The method of claim 8 wherein the twiddle layer comprises a neural network layer defined to learn weights in diagonal linear operations.
10. The method of claim 1 wherein performing comprises performing with the convolution butterfly layer being a sum and difference-based convolutional layer.
11. The method of claim 1 wherein generating the image domain data comprises reconstructing with a machine-learned reconstruction network trained with gradient descent and regularization and including the machine-learned neural network.
12. The method of claim 1 wherein displaying comprises displaying the magnetic resonance image as a two-dimensional magnetic resonance image.
13. A system for reconstruction in medical imaging, the system comprising:
a medical scanner configured to scan a patient, the scan providing scan data;
a reconstruction processor configured to perform a fast transform of the scan data based on application of the scan data to a machine-learned neural network, the machine-learned neural network comprising an architecture of a sequence of a convolutional splitting layer, parallel recursive deep networks, a diagonal linear layer, and a convolutional butterfly layer and configured to generate image domain data from an output of the machine-learned neural network; and
a display configured to display the image domain data as reconstructed from the scan data,
wherein perform comprises performing with the recursive deep networks comprising nested radix fast Fourier transform networks at iteratively reducing scales.
14. The system of claim 13 wherein the medical scanner comprises a magnetic resonance scanner and where in the image comprises a magnetic resonance image.
15. The system of claim 13 wherein the machine-learned neural network comprises blocks having a radix fast Fourier transform architecture.
16. The system of claim 13 wherein the machine-learned neural network is configured to perform forward or adjoint measurement operations.
17. A method for Fourier transform-based reconstruction in a medical imaging system, the method comprising:
scanning, by the medical imaging system, a patient, the scanning resulting in scan data representing the patient;
reconstructing an image of the patient from the scan data, the reconstructing including performing a fast transform of the scan data based on application of the scan data to a machine-learned neural network, the machine-learned neural network comprising an architecture of a sequence of a convolutional splitting layer, parallel recursive deep networks, a diagonal linear layer, and a convolutional butterfly layer;
generating the scanned image of the patient from an output of the machine-learned neural network; and
displaying the image of the patient,
wherein performing comprises performing with the recursive deep networks comprising nested radix fast Fourier transform networks at iteratively reducing scales.
18. The method of claim 17 wherein reconstructing comprises reconstructing with the transformation being a fast Fourier transformation and the trained neural network including the layer with a diagonal operator after the layer for recursive calls and the convolution butterfly layer after the layer with the diagonal operator.
19. The method of claim 17 wherein reconstructing further comprises reconstructing with the trained neural network having an architecture with a Cooley-Tukey fast Fourier transform structure, the structure including the convolution layer to split and the layer for the recursive calls.