IP Library › Granted Patent US 12,299,849
Granted Patent B2
US 12,299,849 · App. 17/807,779 · Granted May 13, 2025

Noise preserving models and methods for resolution recovery of x-ray computed tomography images

Inventors: Roman Melnyk (New Berlin, WI); Madhuri Mahendra Nagare (Karmala, IN); Jie Tang (Merion Station, PA); Obaidullah Rahman (South Bend, IN); Brian E Nett (Wauwatosa, WI); Ken Sauer (South Bend, IN); Charles Addison Bouman, Jr. (West Lafayette, IN)
Assignees: GE Precision Healthcare LLC; Purdue Research Foundation; University of Notre Dame du Lac
G06T5/70A61B6/032G06T7/0012G06T2207/10081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,299,849
App. No.
17/807,779
Granted
May 13, 2025
Kind
B2
Abstract

Noise preserving models and methods for resolution recovery of x-ray computed tomography (e.g., using a computerized tool) are enabled. For example, a system can comprise: a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise: a pair generation component that generates a pair of images, the pair of images comprising an input image and a ground truth image, a training component that trains a machine learning based sharpening algorithm by approximately minimizing a loss function that determines an error between a sharpened image and the ground truth image, and a sharpening component that, using the sharpening algorithm, sharpens the input image to generate the sharpened image, wherein the sharpened image comprises a second noise that is similar in intensity to a first noise of the input image.

Claims (47)

1. A system, comprising:

a memory that stores computer executable components; and

a processor that executes at least one of the computer executable components that:

generates a first set of pairs of training images from a second set of pairs of images, wherein each pair of images of the second set comprises an input image and a noise free ground truth image, wherein the input image is a blurred version of the ground truth image, and wherein the generating comprises, for each pair of images of the second set:

generating a pair of training images of the first set, comprising:

generating a training input image by adding a first amount of noise from a noise sample to the input image based on an input image scaling factor, and

generating a training ground truth image by adding a second amount of noise from the noise sample to the input image based on a ground truth scaling factor, wherein the second amount of noise is scaled to be substantially similar in intensity to the first amount of noise according to a defined threshold;

trains a machine learning based sharpening algorithm to generate sharpened images, using the first set of pairs of training images, by approximately minimizing a loss function associated with reducing image blur that determines respective errors between the training input images and the training ground truth images of the pairs of training images, and maintains a substantially similar intensity of noise between the training input image and the training ground truth image according to the defined threshold; and

sharpens, using the sharpening algorithm, a new input image to generate a sharpened image that has reduced image blur as compared to the new input image, wherein the new input image and the sharpened image comprises a substantially similar intensity of noise according to the defined threshold.

2. The system of claim 1 , wherein the at least one of the computer executable components further:

determines, using the sharpening algorithm, an extent of the sharpening of the input image based on an adjustable sharpening parameter.

3. The system of claim 1 , wherein the at least one of computer executable components further:

reduces artifacts in the sharpened image based on a local edge magnitude of the sharpened image.

4. The system of claim 1 , wherein the at least one of computer executable components further:

reduces artifacts in the sharpened image based on Hounsfield Unit values of the sharpened image.

5. The system of claim 1 , wherein the input image scaling factor is based on a first adjustable parameter.

6. The system of claim 5 , wherein a ground truth scaling factor is based on a second adjustable parameter.

7. The system of claim 1 , wherein the noise sample is generated using scans of a water phantom.

8. The system of claim 1 , wherein the noise sample is generated by subtracting averaged, high-resolution noise from individual, high-resolution noise.

9. A method, comprising:

generating, by a system comprising a processor, a first set of pairs of training images from a second set of pairs of images, wherein each pair of images of the second set comprises an input image and a noise free ground truth image, wherein the input image is a blurred version of the ground truth image, and wherein the generating comprises, for each pair of images of the second set:

generating a pair of training images of the first set, comprising:

generating a training input image by adding a first amount of noise from a noise sample to the input image based on an input image scaling factor, and

generating a training ground truth image by adding a second amount of noise from the noise sample to the input image based on a ground truth scaling factor, wherein the second amount of noise is scaled to be substantially similar in intensity to the first amount of noise according to a defined threshold;

training, by the system, a machine learning based sharpening algorithm to generate sharpened images, using the first set of pairs of training images, by approximately minimizing a loss function associated with reducing image blur that determines respective errors between the training input images and the training ground truth images of the pairs of training images, and maintains a substantially similar intensity of noise between the training input image and the training ground truth image according to the defined threshold; and

sharpening, by the system, using the sharpening algorithm, a new input image to generate a sharpened image that has reduced image blur as compared to the new input image, wherein the new input image and the sharpened image comprises a substantially similar intensity of noise according to the defined threshold.

10. The method of claim 9 , wherein an extent of the sharpening of the input image is determined, using the sharpening algorithm, based on an adjustable sharpening parameter.

11. The method of claim 9 , wherein the input image scaling factor is based on a first adjustable parameter, and wherein the ground truth scaling factor is based on a second adjustable parameter.

12. The method of claim 9 , wherein the noise sample is generated using scans of a water phantom.

13. The method of claim 9 , further comprising:

reducing, by the system, one or more artifacts in the sharpened image based on a local edge magnitude of the sharpened image.

14. The method of claim 9 , further comprising:

reducing, by the system, one or more artifacts in the sharpened image based on Hounsfield Unit values of the sharpened image.

15. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:

generating a first set of pairs of training images from a second set of pairs of images, wherein each pair of images of the second set comprises an input image and a noise free ground truth image, wherein the input image is a blurred version of the ground truth image, and wherein the generating comprises, for each pair of images of the second set:

generating a pair of training images of the first set, comprising:

generating a training input image by adding a first amount of noise from a noise sample to the input image based on an input image scaling factor, and

generating a training ground truth image by adding a second amount of noise from the noise sample to the input image based on a ground truth scaling factor, wherein the second amount of noise is scaled to be substantially similar in intensity to the first amount of noise according to a defined threshold;

training a machine learning based sharpening algorithm to generate sharpened images, using the first set of pairs of training images, by approximately minimizing a loss function associated with reducing image blur that determines respective errors between the training input images and the training ground truth images of the pairs of training images, and maintains a substantially similar intensity of noise between the training input image and the training ground truth image according to the defined threshold; and

using the sharpening algorithm, a new input image to generate a sharpened image that has reduced image blur as compared to the new input image, wherein the new input image and the sharpened image comprises a substantially similar intensity of noise according to the defined threshold.

16. The non-transitory machine-readable storage medium of claim 15 , wherein the input image scaling is based on a first adjustable parameter, and wherein the ground truth scaling factor is based on a second adjustable parameter.

17. The non-transitory machine-readable storage medium of claim 15 , wherein the noise sample is generated using scans of a water phantom.

18. The non-transitory machine-readable storage medium of claim 15 , wherein the noise sample is generated by subtracting averaged, high-resolution noise from individual, high-resolution noise.

19. The non-transitory machine-readable storage medium of claim 15 , wherein the operations further comprise:

reducing one or more artifacts in the sharpened image based on a local edge magnitude of the sharpened image.

20. The non-transitory machine-readable storage medium of claim 15 , wherein the operations further comprise:

reducing one or more artifacts in the sharpened image based on Hounsfield Unit values of the sharpened image.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 25, 2024
From: BOUMAN, CHARLES ADDISON, JR.; NAGARE, MADHURI
To: PURDUE RESEARCH FOUNDATION
Reel/Frame 068687/0568 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2023
From: SAUER, KEN; RAHMAN, OBAIDULLAH
To: UNIVERSITY OF NOTRE DAME DU LAC
Reel/Frame 065295/0355 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2022
From: MELNYK, ROMAN; TANG, JIE; NETT, BRIAN E
To: GE PRECISION HEALTHCARE LLC
Reel/Frame 060250/0432 →
Continuity (1)
Related Publication 20230410259A1 · Dec 21, 2023
References Cited (59)
US 20110268328A1 · Bar-Aviv · 2011 [cited by examiner]
US 20120219215A1 · Lukac · 2012 [cited by examiner]
US 20130202177A1 · Bar-Aviv · 2013 [cited by examiner]
US 20200065940A1 · Tang · 2020 [cited by examiner]
US 20200388014A1 · Hiasa · 2020 [cited by examiner]
US 20210118099A1 · Kearney · 2021 [cited by examiner]
US 20220375038A1 · Nagare et al. · 2022 [cited by applicant]
US 20240062342A1 · Jangid · 2024 [cited by examiner]
Tyler et al., “Beyond Fourth-order Texture Discrimination: Generation of Extreme-order and Statistically-balanced Textures”, Vision Research, vol. 44, 18, 2004, pp. 2187-2199. [cited by applicant]
Solomon et al., “Quantitative Comparison of Noise Texture Across Ct Scanners From Different Manufacturers”, Medical Physics, vol. 39, No. 10, Oct. 2012, pp. 6048-6055. [cited by applicant]
Nagare et al., “Image Enhancement Using Texture Matching Generative Adversarial Networks”, U.S. Appl. No. 18/475,609, filed Sep. 27, 2023, 71 pages. [cited by applicant]
Nagare et al., “A Bias-reducing Loss Function for CT Image Denoising”, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 1175-1179. [cited by applicant]
Nagare et al., “A Noise Preserving Sharpening Filter for CT Image Enhancement”, IEEE International Conference on Image Processing (ICIP), 2022, 4 pages. [cited by applicant]
Gibson, James J., “The Perception of Visual Surfaces”, The American Journal of Psychology, vol. 63, No. 3, 367-384, Jul. 1950, 4 pages. [cited by applicant]
Jagadeesh et al., “Texture-like Representation of Objects In Human Visual Cortex”, Proceedings of the National Academy of Sciences, vol. 119, No. 17, e2115302119, Apr. 19, 2022, 13 pages. [cited by applicant]
Waite et al., “Analysis of Perceptual Expertise in Radiology-current Knowledge and A New Perspective”, Frontiers in Human Neuroscience, vol. 13, No. 213, Jun. 25, 2019, 21 pages. [cited by applicant]
Kulathilake et al., “A Review on Self-adaptation Approaches and Techniques in Medical Image Denoising Algorithms”, Multimedia Tools and Applications, vol. 81, Issue 26, Aug. 4, 2022, 38 pages. [cited by applicant]
Buades et al., “A Non-local Algorithm for Image Denoising”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, 2005, 6 pages. [cited by applicant]
Li et al., “Adaptive Nonlocal Means Filtering Based on Local Noise Level for CT Denoising”, Medical Physics, vol. 41, No. 1, Dec. 31, 2013, 16 pages. [cited by applicant]
Dabov et al., “Image Denoising by Sparse 3dD Transform-domain Collaborative Filtering”, IEEE Transactions on Image Processing, vol. 16, No. 8, Aug. 2007, 16 pages. [cited by applicant]
Beister et al., “Iterative Reconstruction Methods in X-ray CT”, Physica Medica, vol. 28, Issue 2, 2012, pp. 94-108. [cited by applicant]
Kang et al., “A Deep Convolutional Neural Network Using Directional Wavelets for Lowdose X-ray CT Reconstruction”, Medical Physics, vol. 44, No. 10, Oct. 2017, 16 pages. [cited by applicant]
Chen et al., “Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)”, IEEE Transactions on Medical Imaging, vol. 36, No. 12, Dec. 2017, 38 pages. [cited by applicant]
Yang et al., “Low-Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss”, IEEE Transactions on Medical Imaging, vol. 37, No. 6, Jun. 2018, 10 pages. [cited by applicant]
Li et al., “SACNN: Self-Attention Convolutional Neural Network for Low-Dose CT Denoising with Self-Supervised Perceptual Loss Network”, In IEEE Transactions on Medical Imaging, vol. 39, No. 7, Jul. 2020, 13 pages. [cited by applicant]
You et al., “CT Super-resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE)”, IEEE Transactions on Medical Imaging, vol. 39, No. 1, Jan. 2020, 14 pages. [cited by applicant]
Xian et al., “TextureGAN: Controlling Deep Image Synthesis with Texture Patches”, In Proceedings of the IEEE Conference On Computer Vision And Pattern Recognition, 2018, pp. 8456-8465. [cited by applicant]
Starck et al., “Image Decomposition via the Combination of Sparse Representations and a Variational Approach”, IEEE Transactions on Image Processing, vol. 14, No. 10, Oct. 2005, pp. 1570-1582. [cited by applicant]
Zeng et al., “Magnetic Resonance Image Denoising Algorithm Based on Cartoon, Texture, and Residual Parts”, Computational and Mathematical Methods in Medicine, Apr. 1, 2020, vol. 2020, Article ID 1405647, 10 pages. [cited by applicant]
Chen et al., “Image Denoising Algorithm Based on Structure and Texture Part”, International Conference on Computational Intelligence and Security (CIS), 2016, pp. 147-151. [cited by applicant]
Goodfellow et al., “Generative Adversarial Nets”, In Advances in Neural Information Processing Systems, vol. 27, 2014, URL https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf, 9 pag… [cited by applicant]
Bromley et al., “Signature Verification Using a “Siamese” Time Delay Neural Network”, International Journal of Pattern Recognition and Artificial Intelligence, vol. 7 (4), 1993, 8 pages. [cited by applicant]
Goodfellow et al., “Deep Learning”, MIT Press, http://www.deeplearningbook.org., 2016, 9 pages. [cited by applicant]
Isola et al., “Image-to-image Translation with Conditional Adversarial Networks”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1125-1134. [cited by applicant]
Hsieh et al., “A New Era of Image Reconstruction: Truefidelity”, Technical White Paper on Deep Learning Image Reconstruction, GE Healthcare, 2019, 15 pages. [cited by applicant]
Hsieh, Jiang, “Computed Tomography: Principles, Design, Artifacts, and Recent Advances”, Spie Press, 3rd Edition, 2015, 17 pages. [cited by applicant]
Ghalati et al., “Texture Analysis and Its Applications in Biomedical Imaging: A Survey”, IEEE Reviews in Biomedical Engineering, vol. 15, Dec. 7, 2022, pp. 222-246. [cited by applicant]
Melnyk et al., U.S. Appl. No. 17/807,779 titled “Noise Preserving Models and Methods for Resolution Recovery of X-ray Computed Tomography Images”, filed on Jun. 20, 2022, 30 pages. [cited by applicant]
Cohen et al., “Distribution Matching Losses Can Hallucinate Features in Medical Image Translation”, arXiv:1805.08841v3 [cs.CV] Oct. 3, 2018, 11 pages. [cited by applicant]
Arjovsky et al., “Wasserstein Generative Adversarial Networks”, Proceedings of the 34th International Conference on Machine Learning, vol. 70, 2017, 10 pages. [cited by applicant]
Wang et al., “Image Quality Assessment: From Error Visibility to Structural Similarity”, IEEE Transactions on Image Processing, vol. 13, No. 4, Apr. 2004, 14 pages. [cited by applicant]
Szczykutowicz et al., “A Review of Deep Learning CT Reconstruction: Concepts, Limitations, and Promise in Clinical Practice”, Current Radiology Reports, vol. 10, No. 9, 2022, pp. 101-115. [cited by applicant]
Li, B. et al | “Resolution and noise trade-off analysis for volumetric CT”. Medical physics, 34(10), Oct. 2007, pp. 3732-3738, 7 pages. [cited by applicant]
Jiang, M. et al. | “Blind deblurring of spiral CT images”. IEEE Transactions on Medical Imaging, vol. 22, No. 7, Jul. 2003, pp. 837-845, 9 pages. [cited by applicant]
Thibault, J.B., et al. | A three-dimensional statistical approach to improved image quality for multislice helical CT. Medical physics, 34(11), 2007, pp. 4526-4544, 20 pages. [cited by applicant]
Zhang, R., Tet al. | “Model-based iterative reconstruction for dual-energy X-ray CT using a joint quadratic likelihood model”. IEEE transactions on medical imaging, 33(1), 203, pp. 117-134, 18 pages. [cited by applicant]
Kuntz, J. et al. | “Focal spot deconvolution using convolutional neural networks”. Proc. SPIE 10948, Medical Imaging 2019: Physics of Medical Imaging, 109480Q (Mar. 1, 2019); doi: 10.1117/12.2513400, 7 pages. [cited by applicant]
Yim, D. et al. | “A deep convolutional neural network for simultaneous denoising and deblurring in computed tomography”. Journal of Instrumentation, 15(12), p. P12001, 12 pages. [cited by applicant]
Dong, C., et al. | “Image super-resolution using deep convolutional networks”. IEEE transactions on pattern analysis and machine intelligence, arXiv:1501.00092v3 [cs.CV] Jul. 31, 2015, 14 pages. [cited by applicant]
Chen, M., Cet sil | “Learning Blind Denoising Network for Noisy Image Deblurring”. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing, (ICASSP), May 2020, 5 pages. [cited by applicant]
Zhang, K., et al. | “Learning a single convolutional super-resolution network for multiple degradations”. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, arXiv:1712.06116v2 [cs.CV] May … [cited by applicant]
Chi, J., Zet al. | Computed tomography (CT) image quality enhancement via a uniform framework integrating noise estimation and super-resolution networks. Sensors 2019, 19, 3348; doi:10.3390/s19153348, Published: Jul. 30… [cited by applicant]
Wolterink, J.M. et al. | “Generative adversarial networks for noise reduction in low-dose Ct”. IEEE transactions on medical imaging, 36(12), Dec. 2017, pp. 2536-2545, 10 pages. [cited by applicant]
Dong, W., et al. | Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Transactions on image processing, 20(7), 2011, 35 pages. [cited by applicant]
Zhang, K., et al. | Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing, arXiv:1608.03981v1 [cs.CV] Aug. 13, 2016, 13 pages. [cited by applicant]
Phantom Laboratory, The | “Catphan 700 manual,” Product guide, The Phantom Laboratory, 2021, 52 pages. [cited by applicant]
You, Ch. et al. | “Ct super-resolution gan constrained by the identical, residual, and cycle learning ensemble (gancircle)”. asarXiv:1808.04256v3 [eess.IV] Sep. 6, 2018, 16 pages. [cited by applicant]
Kingma, P. et al. | “Adam: A method for stochastic optimization”. arXiv:1412.6980v9 [cs.LG] Jan. 30, 2017, 15 pages. [cited by applicant]
Chollet, K. et al. | “Keras”. Webpage https://keras.io, 2015, last accessed May 24, 2022, 5 pages. [cited by applicant]