Deep learning techniques for magnetic resonance image reconstruction
A magnetic resonance imaging (MRI) system, comprising: a magnetics system comprising: a B 0 magnet configured to provide a B 0 field for the MRI system; gradient coils configured to provide gradient fields for the MRI system; and at least one RF coil configured to detect magnetic resonance (MR) signals; and a controller configured to: control the magnetics system to acquire MR spatial frequency data using non-Cartesian sampling; and generate an MR image from the acquired MR spatial frequency data using a neural network model comprising one or more neural network blocks including a first neural network block, wherein the first neural network block is configured to perform data consistency processing using a non-uniform Fourier transformation.
1. A method, comprising:
generating a magnetic resonance (MR) image from input MR spatial frequency data using a neural network model that comprises:
a first neural network sub-model configured to process spatial frequency domain data; and
a second neural network sub-model configured to process image domain data;
wherein the generating comprises:
processing the input MR spatial frequency data using the first neural network sub-model to obtain output MR spatial frequency data;
transforming the output MR spatial frequency data to the image domain to obtain input image-domain data; and
processing the input image-domain data using the second neural network sub-model to obtain the MR image.
2. The method of claim 1 ,
wherein the first neural network sub-model includes at least one convolutional layer, a locally-connected layer, and at least one transposed convolutional layer, and
wherein processing the input MR spatial frequency data using the first neural network sub-model comprises:
applying the at least one convolutional layer to the input MR spatial frequency data;
applying the locally-connected layer to data obtained using output of the at least one convolutional layer; and
applying the at least one transposed convolutional layer to data obtained using output of the locally-connected layer.
3. The method of claim 2 , wherein the first neural network sub-model includes a complex-conjugate symmetry layer, and wherein processing the input MR spatial frequency data using the first neural network sub-model comprises:
applying the complex-conjugate symmetry layer to data obtained using output of the at least one transposed convolutional layer.
4. The method of claim 2 , wherein the first neural network sub-model includes a data consistency layer, and wherein processing the input MR spatial frequency data using the first neural network sub-model comprises:
applying the data consistency layer to data obtained using output of the complex-conjugate symmetry layer.
5. The method of claim 1 , wherein the first neural network sub-model includes at least one convolutional layer and at least one transposed convolutional layer.
6. The method of claim 1 , wherein the first neural network sub-model includes at least one locally-connected layer.
7. The method of claim 1 , wherein the first neural network sub-model includes at least one data consistency layer.
8. The method of claim 1 , wherein the first neural network sub-model comprises a data consistency block implemented at least in part using a non-uniform fast Fourier transform and the second neural network sub-model comprises a convolutional neural network block.
9. The method of claim 1 , wherein the first neural network sub-model includes at least one fully-connected layer.
10. The method of claim 1 , wherein the first neural network sub-model includes a fully-connected layer, the method further comprising:
applying the fully-connected layer to a real part of the spatial frequency domain data; and
applying the fully-connected layer to an imaginary part of the spatial frequency domain data.
11. The method of claim 1 , wherein the first neural network sub-model includes a first fully-connected and a second fully connected layer, the method further comprising:
applying the first fully-connected layer to a real part of the spatial frequency domain data;
applying the second fully-connected layer to an imaginary part of the spatial frequency domain data.
12. The method of claim 11 , wherein the first and second fully-connected layers share at least some weights.
13. The method of claim 11 , further comprising:
transforming output of the first and second fully-connected layers using a Fourier transformation to obtain image-domain data; and
providing the image-domain data as input to the second neural network sub-model.
14. The method of claim 1 , wherein the second neural network sub-model comprises a series of blocks comprising respective sets of neural network layers, each of the plurality of blocks comprising at least one convolutional layer and at least one transposed convolutional layer.
15. The method of claim 14 , wherein each of the plurality of blocks further comprises: a Fourier transformation layer, a data consistency layer, and an inverse Fourier transformation layer.
16. The method of claim 1 , further comprising:
training the neural network model using a set of high-field images to obtain a first trained neural network model; and
adapting the first neural network model by using a set of low-field images.
17. The method of claim 1 , wherein the spatial frequency domain data is under-sampled relative to a Nyquist criterion.
18. A system, comprising:
at least one computer hardware processor; and
at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform:
generating a magnetic resonance (MR) image from MR spatial frequency data using a neural network model that comprises:
a first neural network portion configured to process data in a spatial frequency domain; and
a second neural network portion configured to process data in an image domain;
wherein the generating comprises:
processing the MR spatial frequency data using the first neural network portion to obtain output MR spatial frequency data;
transforming the output MR spatial frequency data to the image domain to obtain input image-domain data; and
processing the input image-domain data using the second neural network portion to obtain the MR image.
19. A magnetic resonance imaging (MRI) system, comprising:
a magnetics system comprising:
a B 0 magnet configured to provide a B 0 field for the MRI system;
gradient coils configured to provide gradient fields for the MRI system; and
at least one RF coil configured to detect magnetic resonance (MR) signals;
a controller configured to:
control the magnetics system to acquire MR spatial frequency data;
generate an MR image from MR spatial frequency data using a neural network model that comprises:
a first neural network portion configured to process data in a spatial frequency domain; and
a second neural network portion configured to process data in an image domain;
wherein the generating comprises:
processing the MR spatial frequency data using the first neural network portion to obtain output MR spatial frequency data;
transforming the output MR spatial frequency data to the image domain to obtain input image-domain data; and
processing the input image-domain data using the second neural network portion to obtain the MR image.