Deep learning techniques for magnetic resonance image reconstruction
A magnetic resonance imaging (MRI) system, comprising: a magnetics system comprising: a B 0 magnet configured to provide a B 0 field for the MRI system; gradient coils configured to provide gradient fields for the MRI system; and at least one RF coil configured to detect magnetic resonance (MR) signals; and a controller configured to: control the magnetics system to acquire MR spatial frequency data using non-Cartesian sampling; and generate an MR image from the acquired MR spatial frequency data using a neural network model comprising one or more neural network blocks including a first neural network block, wherein the first neural network block is configured to perform data consistency processing using a non-uniform Fourier transformation.
1. A method, comprising:
generating a magnetic resonance (MR) image from input MR spatial frequency data using a neural network model configured to process spatial frequency domain data and image domain data, wherein generating the MR image comprises:
applying, to image domain data, at least a first non-uniform Fourier transformation to obtain a first output;
applying at least a second non-uniform Fourier transformation to the input MR spatial frequency data to obtain a second output; and
providing at least the first output and the second output as inputs to a plurality of convolutional layers.
2. The method of claim 1 , wherein the neural network model comprises one or more neural network blocks including a first neural network block configured to perform data consistency processing using at least one of the first or second non-uniform Fourier transformations.
3. The method of claim 1 , further comprising:
obtaining the input MR spatial frequency data;
generating an initial image from the input MR spatial frequency data using at least one of the first or second non-uniform Fourier transformations; and
applying the neural network model to the initial image at least in part by using a first neural network block of the neural network model to perform data consistency processing using at least one of the first or second non-uniform Fourier transformations.
4. The method of claim 1 , wherein a first neural network block of the neural network model is configured to perform data consistency processing at least in part by performing one or more non-uniform Fourier transformations on data by applying a gridding interpolation transformation, a fast Fourier transformation, and a de-apodization transformation to the data.
5. The method of claim 4 , wherein applying the gridding interpolation transformation to the data is performed using sparse graphical processing unit (GPU) matrix multiplication.
6. The method of claim 1 , wherein a first neural network block of the neural network model comprises:
a data consistency block configured to perform data consistency processing; and
the plurality of convolutional layers or a second plurality of convolutional layers.
7. The method of claim 6 , wherein the data consistency block is configured to:
apply the first non-uniform Fourier transformation to a first image, provided as input to the data consistency block, to obtain first MR spatial frequency data; and
apply the second non-uniform Fourier transformation to a difference between the first MR spatial frequency data and the input MR spatial frequency data.
8. The method of claim 7 , wherein applying the first non-uniform Fourier transformation to the first image domain data comprises:
applying, to the first image domain data, a de-apodization transformation followed by a Fourier transformation, and followed by a gridding interpolation transformation.
9. The method of claim 6 , wherein at least one of the plurality of convolutional layers or the second plurality of convolutional layers include one or more convolutional layers and one or more transposed convolutional layers.
10. The method of claim 6 , wherein at least one of the plurality of convolutional layers or the second plurality of convolutional layers have a U-net structure.
11. The method of claim 1 , wherein the neural network model comprises a first neural network block that comprises:
the plurality of convolutional layers configured to receive as input:
the image domain data; and
the second output obtained by applying the second non-uniform Fourier transformation to the input MR spatial frequency data, wherein the second non-uniform Fourier transformation is an adjoint non-uniform Fourier transformation.
12. The method of claim 11 , wherein the plurality of convolutional layers is further configured to receive as input:
output obtained by applying the first non-uniform Fourier transformation and the second non-uniform Fourier transformation to the image domain data.
13. The method of claim 1 , wherein:
the first output is obtained by applying, to the image domain data, the first non-uniform Fourier transformation followed by the second non-uniform Fourier transformation.
14. The method of claim 13 , wherein the image domain data, the first output, and the second output are provided as inputs to the plurality of convolutional layers.
15. The method of claim 1 , further comprising:
applying a convolutional neural network to a result of applying the first non-uniform Fourier transformation to the image domain data to obtain an intermediate output; and
applying the second non-uniform Fourier transformation to the intermediate output to obtain the first output.
16. The method of claim 1 , wherein points in the input MR spatial frequency data were obtained using a non-Cartesian sampling trajectory.
17. The method of claim 16 , wherein the first non-uniform Fourier transformation is determined at least in part by using the non-Cartesian sampling trajectory.
18. The method of claim 1 , wherein the neural network model comprises a first neural network sub-model configured to process the spatial frequency domain data, and a second neural network sub-model configured to process the image domain data.
19. The method of claim 1 , wherein the second non-uniform Fourier transformation is an adjoint non-uniform Fourier transformation.
20. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform a method comprising:
generating a magnetic resonance (MR) image from input MR spatial frequency data using a neural network model configured to process spatial frequency domain data and image domain data, wherein generating the MR image comprises:
applying, to the image domain data, at least a first non-uniform Fourier transformation to obtain a first output;
applying at least a second non-uniform Fourier transformation to the input MR spatial frequency data to obtain a second output; and
providing at least the first output and the second output as inputs to a plurality of convolutional layers.
21. A magnetic resonance imaging (MRI) system, comprising:
a magnetics system comprising:
a B 0 magnet configured to provide a B 0 field for the MRI system;
gradient coils configured to provide gradient fields for the MRI system; and
at least one RF coil configured to detect magnetic resonance (MR) signals;
a controller configured to:
control the magnetics system to acquire MR spatial frequency data using a non-Cartesian sampling trajectory; and
generate an MR image from the acquired MR spatial frequency data using a neural network model configured to process spatial frequency domain data and image domain data, wherein generating the MR image comprises:
applying, to the image domain data, at least a first non-uniform Fourier transformation to obtain a first output;
applying at least a second non-uniform Fourier transformation to the input MR spatial frequency data to obtain a second output; and
providing at least the first output and the second output as inputs to a plurality of convolutional layers.
22. The MRI system of claim 21 , wherein the B 0 magnet is a permanent magnet.