IP Library › Granted Patent US 12,402,853
Granted Patent B2
US 12,402,853 · App. 17/880,831 · Granted Sep 2, 2025

Systems and methods for real-time video enhancement

Inventors: David Van Veen (San Francisco, CA); Long Wang (Sunnyvale, CA); Ben Andrew Duffy (Mountain View, CA); Enhao Gong (Sunnyvale, CA); Tao Zhang (Menlo Park, CA)
Assignee: SUBTLE MEDICAL, INC.
A61B6/542A61B6/5258G06T5/50G06T5/70G06V10/82G06T2207/10016G06T2207/10121G06T2207/20081G06T2207/20084G06T2207/20216G06T2207/30004G06T2207/30168
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,402,853
App. No.
17/880,831
Granted
Sep 2, 2025
Kind
B2
Abstract

A computer-implemented method is provided for improving live video quality. The method comprises: acquiring, using a medical imaging apparatus, a stream of consecutive image frames of a subject, and the stream of consecutive image frames are acquired with reduced amount of radiation dose; applying a deep learning network model to the stream of consecutive image frames to generate an image frame with improved quality; and displaying the image frame with improved quality in real-time on a display.

Claims (24)

1. A computer-implemented method for improving live video quality comprising:

(a) acquiring, using a medical imaging apparatus, a stream of consecutive image frames of a subject, wherein the stream of consecutive image frames is acquired with a reduced amount of radiation dose;

(b) applying a deep learning network model to the stream of consecutive image frames to generate an output image frame with improved quality in both temporal domain and spatial domain, wherein the deep learning network model is trained using training datasets comprising a pair of a simulated low-quality video and a simulated high-quality video; and

(c) displaying the output image frame with improved quality in real-time on a display.

2. The computer-implemented method of claim 1 , wherein the simulated high-quality video is generated by applying a temporal averaging algorithm or a denoising algorithm to a video acquired with a normal radiation dose.

3. The computer-implemented method of claim 2 , further comprising computing a noise based on a difference between the video and the simulated high-quality video.

4. The computer-implemented method of claim 2 , further comprising applying a factor to the noise to simulate a level of noise corresponding to the factor.

5. The computer-implemented method of claim 4 , wherein the simulated low-quality video is generated based at least in part on the level of noise and the simulated high-quality video.

6. The computer-implemented method of claim 1 , wherein the deep learning network model comprises a plurality of denoising components.

7. The computer-implemented method of claim 6 , wherein the plurality of denoising components are assembled in a two-layer architecture.

8. The computer-implemented method of claim 7 , wherein each denoising component in a first layer of the two-layer architecture processes a subset of the stream of consecutive frames to output a series of intermediate image frames with an enhanced image quality.

9. The computer-implemented method of claim 8 , wherein a denoising component in the second layer of the two-layer architecture processes the intermediate image frames with the enhanced image quality and generates the output image frame.

10. The computer-implemented method of claim 7 , wherein each denoising component includes a modified U-net model.

11. The computer-implemented method of claim 10 , wherein a denoising component in a second layer of the two-layer architecture has weights different from the weights of a denoising component in the first layer.

12. The computer-implemented method of claim 1 , wherein the medical imaging apparatus is performing fluoroscopic imaging.

13. A non-transitory computer-readable storage medium including instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

(a) acquiring, using a medical imaging apparatus, a stream of consecutive image frames of a subject, wherein the stream of consecutive image frames is acquired with a reduced amount of radiation dose;

(b) applying a deep learning network model to the stream of consecutive image frames to generate an output image frame with an improved quality in both temporal domain and spatial domain, wherein the deep learning network model is trained using training datasets comprising a pair of a simulated low-quality video and a simulated high-quality video; and

(c) displaying the output image frame with the improved quality in real-time on a display.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the simulated high-quality video is generated by applying a temporal averaging algorithm or a denoising algorithm to a video acquired with a normal radiation dose.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more operations further comprise computing a noise based on a difference between the video and the simulated high-quality video.

16. The non-transitory computer-readable storage medium of claim 14 , wherein the one or more operations further comprise applying a factor to the noise to simulate a level of noise corresponding to the factor.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the simulated low-quality video is generated based at least in part on the level of noise and the simulated high-quality video.

18. The non-transitory computer-readable storage medium of claim 13 , wherein the deep learning network model comprises a plurality of denoising components.

Assignments (2)
GRANT OF SECURITY INTEREST IN PATENTS Recorded May 29, 2026
From: SUBTLE MEDICAL, INC.
To: MS PRIVATE CREDIT ADMINISTRATIVE SERVICES LLC
Reel/Frame 075648/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 17, 2022
From: VEEN, DAVID VAN; WANG, LONG; ZHANG, TAO; GONG, ENHAO; DUFFY, BEN ANDREW
To: SUBTLE MEDICAL, INC.
Reel/Frame 061441/0224 →
Continuity (3)
Continuation PCTUS2021017189 · Feb 9, 2021
Provisional Application 62972999 · Feb 11, 2020
Related Publication 20230038871A1 · Feb 9, 2023
References Cited (41)
US 6084981A · Horiba et al. · 2000 [cited by applicant]
US 6314160B1 · Dhawale et al. · 2001 [cited by applicant]
US 20090225940A1 · Aoyama · 2009 [cited by examiner]
US 20160335754A1 · Aaron · 2016 [cited by examiner]
US 20170014094A1 · Hiroshige · 2017 [cited by examiner]
US 20180235557A1 · Rousso · 2018 [cited by examiner]
US 20190035118A1 · Zhao et al. · 2019 [cited by applicant]
US 20190139205A1 · El-Khamy · 2019 [cited by examiner]
US 20190347772A1 · Zhang et al. · 2019 [cited by applicant]
US 20190371450A1 · Lou · 2019 [cited by examiner]
US 20210160556A1 · Jang · 2021 [cited by examiner]
US 20220156884A1 · Kemp · 2022 [cited by examiner]
CN 107610195B · 2021 [cited by applicant]
EP 2356940A1 · 2011 [cited by applicant]
WO WO2002056770 · 2005 [cited by applicant]
WO WO2019019199A1 · 2019 [cited by applicant]
WO WO2021163022A1 · 2021 [cited by applicant]
Zhang et al.: Hybrid 3D/2D-Based Deep Convolutional Neural Network for Spatio-Temporal Denoising of Angiography. ISICDM 2019: Proceedings of the Third International Symposium on Image Computing and Digital Medicine pp. … [cited by applicant]
Balter, et al., “Fluoroscopically guided interventional procedures: a review of radiation effects on patients' skin and hair,” Radiology, vol. 254, No. 2, pp. 326-341, 2010. [cited by applicant]
Davy et al. Non-local video denoising by CNN. arXiv preprint arXiv:1811.12758. pp. 1-14 (2018). [cited by applicant]
Dosovitskiy, et al., “Flownet: Learning optical flow with convolutional networks,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 2758-2766. [cited by applicant]
Grace et al. Retrospective Analysis: Collateral nerve damage and local tissue trauma associated with endovenous laser ablation therapy. International Journal of Scientific and Research Publications Submission. (2018) Av… [cited by applicant]
Hoffman, et al., “Breast cancer in women with scoliosis exposed to multiple diagnostic x rays,” JNCI: Journal of the National Cancer Institute, vol. 81, No. 17, pp. 1307-1312, 1989. [cited by applicant]
Huda, Walter, “Kerma-area product in diagnostic radiology,” American Journal of Roentgenology, vol. 203, No. 6, pp. W565-W569, 2014. [cited by applicant]
Kingma, et al. Adam: A Method for Stochastic Optimization. Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015. [cited by applicant]
Krizhevsky. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105) 2012. [cited by applicant]
Maggioni et al. Video denoising, deblocking, and enhancement through separable 4-D nonlocal spatiotemporal transforms. IEEE Transactions on image processing, 21(9), 3952-3966 (2012). [cited by applicant]
Mastrangelo, et al. “Increased cancer risk among surgeons in an orthopaedic hospital,” Occupational Medicine, vol. 55, No. 6, pp. 498-500, 2005. [cited by applicant]
Pazke et al. Automatic differentiation in pytorch. 31st Conference on Neural Information Processing Systems (NIPS 2017); pp. 1-4 (2017). [cited by applicant]
PCT/US20/17189 International Search Report and Written Opinion dated Apr. 23, 2021. [cited by applicant]
Primak, et al., “Relationship between noise, dose, and pitch in cardiac multi-detector row ct,” Radiographics, vol. 26, No. 6, pp. 1785-1794, 2006. [cited by applicant]
Radiation Term: ALARA; Acronym Index. Health Physics Society, https://hps.org/publicinformation/radterms/radfact1.html. Retrieved from website on Mar. 4, 2020. [cited by applicant]
Rampersaud, et al., “Radiation exposure to the spine surgeon during fluoroscopically assisted pedicle screw insertion,” Spine, vol. 25, No. 20,pp. 2637-2645, 2000. [cited by applicant]
Ronneberger et al. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pp. 234-241(2015). [cited by applicant]
Sampat, et al., “Complex wavelet structural similarity: A new image similarity index,” IEEE transactions on image processing, vol. 18, No. 11, pp. 2385-2401, 2009. [cited by applicant]
Slovis, Thomas, “Children, computed tomography radiation dose, and the as low as reasonably achievable (alara) concept,” Pediatrics, vol. 112, No. 4, pp. 971-972, 2003. [cited by applicant]
“Surgery Treatment.” Dedicated Computing, https://www.dedicatedcomputing.com/markets/healthcare/surgery-treatment/ (2020). [cited by applicant]
Tassano et al. Dvdnet: A Fast Network for Deep Video Denoising. 2019 IEEE International Conference on Image Processing (ICIP) (pp. 1805-1809). IEEE (2019). [cited by applicant]
Tassano et al. FastDVDnet: Towards Real-Time Video Denoising Without Explicit Motion Estimation. arXiv preprint arXiv: 1907.01361. pp. 1-10 (2019). [cited by applicant]
Wu, et al., “Deep high dynamic range imaging with large foreground motions,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 117-132. [cited by applicant]
Zhou, et al., “Minimally invasive surgery under fluoro-navigation for anterior pelvic ring fractures,” Indian journal of orthopaedics, vol. 50, No. 3,pp. 250, 2016. [cited by applicant]