IP Library › Granted Patent US 12,265,145
Granted Patent B2
US 12,265,145 · App. 17/642,016 · Granted Apr 1, 2025

Systems and methods of deep learning for large-scale dynamic magnetic resonance image reconstruction

Inventors: Anthony Christodoulou (Los Angeles, CA); Debiao Li (Los Angeles, CA); Yuhua Chen (Los Angeles, CA)
Assignee: CEDARS-SINAI MEDICAL CENTER
G01R33/5608G01R33/483G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,265,145
App. No.
17/642,016
Granted
Apr 1, 2025
Kind
B2
Abstract

A method for performing magnetic resonance imaging on a subject comprises obtaining undersampled imaging data, extracting one or more temporal basis functions from the imaging data, extracting one or more preliminary spatial weighting functions from the imaging data, inputting the one or more preliminary spatial weighting functions into a neural network to produce one or more final spatial weighting functions, and multiplying the one or more final spatial weighting functions by the one or more temporal basis functions to generate an image sequence. Each of the temporal basis functions corresponds to at least one time-varying dimension of the subject. Each of the preliminary spatial weighting functions corresponds to a spatially-varying dimension of the subject. Each of the final spatial weighting functions is an artifact-free estimation of the one of the one or more preliminary spatial weighting functions.

Claims (45)

1. A method for performing magnetic resonance (MR) imaging on a subject, the method comprising:

obtaining undersampled imaging data from a region of interest of the subject, the undersampled imaging data corresponding to an image sequence having a plurality of image frames;

extracting one or more temporal basis functions from the undersampled imaging data, each of the one or more temporal basis functions corresponding to at least one time-varying dimension of the subject;

extracting one or more preliminary spatial weighting functions from the undersampled imaging data, each of the one or more preliminary spatial weighting functions corresponding to a spatially-varying dimension of the subject;

inputting the one or more preliminary spatial weighting functions into a neural network to produce one or more final spatial weighting functions, each of the final spatial weighting functions corresponding to a respective one of the one or more preliminary spatial weighting functions; and

multiplying the one or more final spatial weighting functions by the one or more temporal basis functions to generate the image sequence.

2. The method of claim 1 , wherein at least one of the one or more preliminary spatial weighting functions includes imaging artifacts, and wherein the corresponding one of the one or more final spatial weighting functions is an artifact-free estimation of the one of the one or more preliminary spatial weighting functions.

3. The method of claim 1 , wherein the neural network is a multi-channel neural network.

4. The method of claim 1 , wherein the neural network is a dilated multi-level densely connected network.

5. The method of claim 1 , wherein the neural network includes one or more dense blocks, at least one of the one or more dense blocks including one or more convolution layers.

6. The method of claim 1 , wherein the neural network includes one or more dense blocks, at least one of the one or more dense blocks including one or more activation functions.

7. The method of claim 6 , wherein at least one of the one or more activation functions is an exponential linear unit.

8. The method of claim 1 , wherein the neural network includes one or more dense blocks.

9. The method of claim 1 , wherein the undersampled imaging data is acquired using a plurality of spatial encodings.

10. The method of claim 9 , wherein the undersampled imaging data includes training data acquired at a subset of the plurality of spatial encodings, and wherein the one or more temporal basis functions is extracted from the training data.

11. The method of claim 1 , wherein the subject is a human.

12. The method of claim 1 , wherein the neural network is trained using one or more training input data sets and one or more reference output data sets, each of the one or more reference output data sets being a previously-obtained artifact-free version of a respective one of the one or more training input data sets, each of the one or more reference output data sets being obtained.

13. The method of claim 12 , wherein the one or more sample input data sets includes at least a set of MR data and a set of non-MR data, and wherein the neural network is trained such that neural network is robust to unseen data and is configured to produce the one or more final spatial weighting functions from MR data and non-MR data.

14. The method of claim 12 , wherein the one or more training input data sets includes one or more training preliminary spatial weighting functions, and wherein the one or more reference output data sets includes one or more training final spatial weighting functions, each of the one or more training final spatial weighting functions being a previously-obtained artifact-free version of a respective one of the one or more training preliminary spatial weighting functions.

15. The method of claim 14 , wherein the neural network is configured to process the one or more training preliminary spatial weighting functions to produce one or more estimated final spatial weighting functions, and wherein the neural network is trained using a cost function configured to minimize differences between the one or more training final spatial weighting functions and the one or more estimated final spatial weighting functions.

16. The method of claim 14 , wherein the neural network is configured to process the one or more training preliminary spatial weighting functions to produce one or more estimated final spatial weighting functions, and wherein the neural network is trained using a cost function configured to minimize differences between a training image sequence based on the one or more training final spatial weighting functions and an estimated image sequence based on the one or more estimated final spatial weighting functions.

17. The method of claim 1 , wherein the multiplying of the one or more final spatial weighting functions by the one or more temporal basis functions is performed by the neural network.

18. The method of claim 1 , further comprising training the neural network prior to inputting the one or more preliminary spatial weighting functions into the neural network to produce the one or more final spatial weighting functions, wherein training the neural network includes:

receiving one or more training input data sets to the neural network, the one or more training input data sets including one or more sets of training preliminary spatial weighting functions and one or more sets of training final spatial weighting functions, each of the one or more sets of training final spatial weighting functions being a previously-obtained artifact-free version of a respective one of the one or more sets of training preliminary spatial weighting functions, each of the one or more sets of training preliminary spatial weighting functions and training final spatial weighting functions corresponding to at least one time-varying dimension of the subject;

processing the one or more sets of training preliminary spatial weighting functions according to an architecture of the neural network to produce one or more sets of estimated final spatial weighting functions;

comparing each of the one or more sets of estimated final spatial weighting functions to a corresponding one of the one or more sets of training final spatial weighting functions;

determining if a cost function of the neural network is satisfied by the one or more sets of estimated final spatial weighting functions, the cost function configured to minimize differences between the one or more sets of training final spatial weighting functions and the one or more sets of estimated final spatial weighting functions;

adjusting one or more network weights of the neural network responsive to a determination that the cost function of the neural network is not satisfied; and

repeatedly processing the one or more training preliminary spatial weighting functions with the adjusted one or more network weights until the cost function is satisfied.

19. A system for performing magnetic resonance (MR) imaging on a subject, comprising:

a magnet operable to provide a magnetic field;

a transmitter operable to transmit to a region within the magnetic field;

a receiver operable to receive a magnetic resonance signal from the region with the magnetic field; and

one or more processors operable to control the transmitter and the receiver, the one or more processors being configured to cause the following method to be performed:

obtaining undersampled imaging data from a region of interest of the subject, the undersampled imaging data corresponding to an image sequence having a plurality of image frames;

extracting one or more temporal basis functions from the undersampled imaging data, each of the one or more temporal basis functions corresponding to at least one time-varying dimension of the subject;

extracting one or more preliminary spatial weighting functions from the undersampled imaging data, each of the one or more preliminary spatial weighting functions corresponding to a spatially-varying dimension of the subject;

inputting the one or more preliminary spatial weighting functions into a neural network to produce one or more final spatial weighting functions, each of the final spatial weighting functions corresponding to a respective one of the one or more preliminary spatial weighting functions; and

multiplying the one or more final spatial weighting functions by the one or more temporal basis functions to generate the image sequence.

20. A non-transitory machine-readable medium having stored thereon instructions for performing magnetic resonance (MR) imaging on a subject, which when executed by at least one processor, cause the following method to be performed:

obtaining undersampled imaging data from a region of interest of the subject, the undersampled imaging data corresponding to an image sequence having a plurality of image frames;

extracting one or more temporal basis functions from the undersampled imaging data, each of the one or more temporal basis functions corresponding to at least one time-varying dimension of the subject;

extracting one or more preliminary spatial weighting functions from the undersampled imaging data, each of the one or more preliminary spatial weighting functions corresponding to a spatially-varying dimension of the subject;

inputting the one or more preliminary spatial weighting functions into a neural network to produce one or more final spatial weighting functions, each of the final spatial weighting functions corresponding to a respective one of the one or more preliminary spatial weighting functions; and

multiplying the one or more final spatial weighting functions by the one or more temporal basis functions to generate the image sequence.

Assignments (2)
CONFIRMATORY LICENSE Recorded Jan 25, 2024
From: CEDARS-SINAI MEDICAL CENTER
To: NATIONAL INSTITUTES OF HEALTH (NIH), U.S. DEPT. OF HEALTH AND HUMAN SERVICES (DHHS), U.S. GOVERNMENT
Reel/Frame 066370/0985 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 10, 2022
From: CHRISTODOULOU, ANTHONY; LI, DEBIAO; CHEN, YUHUA
To: CEDARS-SINAI MEDICAL CENTER
Reel/Frame 059225/0344 →
Continuity (2)
Provisional Application 62900279 · Sep 13, 2019
Related Publication 20230194640A1 · Jun 22, 2023
References Cited (41)
US 5778152A · Oki · 1998 [cited by applicant]
US 7194365B1 · Sonntag · 2007 [cited by applicant]
US 11782111B2 · Qu · 2023 [cited by applicant]
US 20050020897A1 · Fuderer · 2005 [cited by applicant]
US 20060186941A1 · Pruessmann · 2006 [cited by applicant]
US 20080100292A1 · Hancu · 2008 [cited by applicant]
US 20080125643A1 · Huisman · 2008 [cited by applicant]
US 20110044524A1 · Wang · 2011 [cited by applicant]
US 20140219531A1 · Epstein · 2014 [cited by applicant]
US 20160202336A1 · Liang · 2016 [cited by applicant]
US 20190128989A1 · Braun · 2019 [cited by examiner]
US 20200103483A1 · Hardy · 2020 [cited by examiner]
US 20200294287A1 · Schlemper · 2020 [cited by examiner]
US 20200319285A1 · Koerzdoerfer · 2020 [cited by examiner]
CN 103837848A · 2019 [cited by applicant]
CN 109903259A · 2023 [cited by applicant]
Sandino, Christopher M., Frank Ong, and Shreyas S. Vasanawala. “Deep subspace learning: Enhancing speed and scalability of deep learning-based reconstruction of dynamic imaging data.” Proc Int Soc Magn Reson Med. Jul. 2… [cited by examiner]
Nakarmi, U., Wang, Y., Lyu, J., Liang, D., & Ying, L. (2017). A kernel-based low-rank (KLR) model for low-dimensional manifold recovery in highly accelerated dynamic MRI. IEEE transactions on medical imaging, 36(11), 22… [cited by examiner]
Chen et al. “Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction”, IEEE Transactions on Medical Imaging, vol. 38, No. 1, (Jan. 1, 2019), pp. 280-290. [cited by applicant]
Oksuz et al. “Detection and Correction of Cardiac MR Motion Artefacts during Reconstruction from k-space”, arxiv.org, Cornell University Library, 201 Olin Library Cornell University Ithaca, NY 14853, Jun. 12, 2019. [cited by applicant]
Chen et al. “Deep Learning Within Temporal Feature Spaces for Large-Scale Dynamic MR Image Reconstruction: Application to 5-D Cardiac MR Multitasking”, Oct. 10, 2019, 16th European Conference—Computer Vision—ECCV 2020, … [cited by applicant]
Schlemper et al. “A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction”, IEEE Trans Med Imaging 37, 491-503 (2018). [cited by applicant]
Qin et al. “Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction”, IEEE Trans Med Imaging 38, 280-290 (2019). [cited by applicant]
Biswas et al. “Dynamic MRI using model-based deep learning and SToRM priors: MoDL-SToRM”, Magn Reson Med (2019). [cited by applicant]
Feng et al. “XD-GRASP: Golden-angle radial MRI with reconstruction of extra motion-state dimensions using compressed sensing”, Magn Reson Med 75, 775-788 (2016). [cited by applicant]
Cheng et al. “Comprehensive multi-dimensional MRI for the simultaneous assessment of cardiopulmonary anatomy and physiology”, Sci Rep 7, 5330 (2017). [cited by applicant]
Christodoulou et al. “Magnetic resonance multitasking for motion-resolved quantitative cardiovascular imaging”, Nature Biomed Eng 2, 215-226 (2018). [cited by applicant]
Liang “Spatiotemporal imaging with partially separable functions”, Proc IEEE Int Symp Biomed Imaging, pp. 988-991 (2007). [cited by applicant]
Lingala et al. “Accelerated dynamic MRI exploiting sparsity and low-rank structure: k-t SLR”, IEEE Trans Med Imaging 30, 1042-1054 (2011). [cited by applicant]
Zhao et al. “Image reconstruction from highly undersampled (k,t)-space data with joint partial separability and sparsity constraints”, IEEE Trans Med Imaging 31, 1809-1820 (2012). [cited by applicant]
Poddar et al. “Dynamic MRI Using SmooThness Regularization on Manifolds (SToRM)”, IEEE Trans Med Imaging 35, 1106-1115 (2016). [cited by applicant]
Bengio et al. “Representation learning: A review and new perspectives”, IEEE Trans Pattern Anal Mach Intell 35, 1798-1828 (2013). [cited by applicant]
Shaw et al. “Free-breathing, non-ECG, continuous myocardial T1 mapping with cardiovascular magnetic resonance Multitasking”, Magn Reson Med 81, 2450-2463 (2019). [cited by applicant]
Chen et al. “Efficient and accurate MRI super-resolution using a generative adversarial network and 3D multi-Level densely connected network”, In: MICCAI, pp. 91-99. Springer, (2017). [cited by applicant]
Jégou et al. “The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 11-19. (2017). [cited by applicant]
Clevert et al. “Fast and accurate deep network learning by exponential linear units (Elus)”, arXiv preprint arXiv:1511.07289 (2015). [cited by applicant]
Kali et al. “Native T1 mapping by 3-T CMR imaging for characterization of chronic myocardial infarctions.” JACC: Cardiovascular Imaging 8, No. 9 1019-1030 (2015). [cited by applicant]
Extended Europan Search Report in European Patent Application No. 20863427.9, mailed Aug. 21, 2023 (9 pages). [cited by applicant]
Muckley, M. J. et al., “Training a Neural Network for Gibbs and Noise Removal in Diffusion MRI,” Magnetic Resonance in Medicine, 85(1): 413-428, Jul. 2020. https://arxiv.org/abs/1905.04176. [cited by applicant]
International Search Report and Written Opinion in International Patent Application No. PCT/US2020/050247, mailed Jan. 7, 2021 (14 pages). [cited by applicant]
Weng Zhuo et al., “Based on k-space acceleration magnetic resonance imaging technology”, A Journal of Chinese Biomedical Engineering, Issue 5, Oct. 20, 2010, pp. 785-792 (8 pp.). [cited by applicant]