Systems and methods for real-time video denoising

1. A computer-implemented method for improving live video quality comprising:

(a) receiving a stream of consecutive image frames of a subject, wherein the stream of consecutive image frames are acquired using a medical imaging apparatus; and

(b) processing the stream of consecutive image frames using a two-stage deep learning framework by:

i) feeding the stream of consecutive image frames to a first set of denoising components in a first stage of the two-stage deep learning framework, wherein at least one of the first set of denoising components comprises a first deep learning network trained to denoise an image frame in a spatial domain by taking as input an image frame from the stream of consecutive image frames and predicting an intermediate image frame;

ii) feeding a plurality of the intermediate image frames to a second denoising component in a second stage of the two-stage deep learning framework, wherein the second denoising component comprises a second deep learning network trained to predict a final image frame with improved quality in both temporal domain and spatial domain based at least in part on a predicted motion map which motion map indicating a presence of motion in the plurality of the intermediate image frames.

2. The computer-implemented method of claim 1 , wherein the second deep learning network is an integrated multi-task network trained to predict the motion map and perform temporal or spatiotemporal denoising.

3. The computer-implemented method of claim 1 , wherein the second denoising component comprises a network separate from the second deep learning network and wherein the network is trained to predict the motion map.

4. The computer-implemented method of claim 1 , further comprising combining of the plurality of intermediate image frames and a denoised image frame generated by the second denoising component using the motion map to output the final image frame.

5. The computer-implemented method of claim 1 , wherein the motion map is a binary map and has a spatial resolution same as the intermediate image frame.

6. The computer-implemented method of claim 5 , wherein a value of a pixel in the motion map indicates whether to perform temporal or spatiotemporal denoising to the pixel.

7. The computer-implemented method of claim 1 , wherein a number of the stream of consecutive image frames are adjustable.

8. The computer-implemented method of claim 1 , wherein the medical imaging apparatus is performing fluoroscopic imaging.

9. The computer-implemented method of claim 1 , wherein the stream of consecutive image frames is acquired with a reduced amount of radiation dose.

10. The computer-implemented method of claim 1 , wherein the first deep learning network or the second deep learning network includes a modified U-net model.

11. The computer-implemented method of claim 1 , wherein the first deep learning network is trained using training datasets comprising a pair of a simulated low-quality video and a simulated high-quality video.

12. The computer-implemented method of claim 11 , wherein the simulated low-quality video comprises a motion of an object.

13. The computer-implemented method of claim 11 , wherein the pair of the simulated low-quality video and the simulated high-quality video are generated from a video acquired at a standard amount of radiation dose.

14. The computer-implemented method of claim 13 , wherein the simulated low-quality video is generated by introducing a selected type of artifact or a simulated noise at a selected level to the video acquired at the standard amount of radiation dose.

15. A system for improving live video quality comprising:

(i) a communication interface communicatively coupled to a medical imaging apparatus, (ii) a memory for storing a set of software instructions, and (iii) one or more processors configured to execute the set of software instructions to:

(a) receive, from the medical imaging apparatus, a stream of consecutive image frames of a subject; and

(b) process the stream of consecutive image frames using a two-stage deep learning framework by:

i) feed the stream of consecutive image frames to a first set of denoising components in a first stage of the two-stage deep learning framework, wherein at least one of the first set of denoising components comprises a first deep learning network trained to denoise an image frame in a spatial domain by taking as input an image frame from the stream of consecutive image frames and predicting an intermediate image frame;

ii) feed a plurality of the intermediate image frames to a second denoising component in a second stage of the two-stage deep learning framework, wherein the second denoising component comprises a second deep learning network trained to predict a final image frame with improved quality in both temporal domain and spatial domain based at least in part on a predicted motion map which motion map indicating a presence of motion in the plurality of the intermediate image frames.

16. The system of claim 15 , wherein the second deep learning network is an integrated multi-task network trained to predict the motion map and perform temporal or spatiotemporal denoising.

17. The system of claim 15 , wherein the second denoising component comprises a network separate from the second deep learning network and wherein the network is trained to predict the motion map.

18. The system of claim 15 , where the one or more processors are configured to further combine of the plurality of intermediate image frames and a denoised image frame generated by the second denoising component using the motion map to output the final image frame.

19. The system of claim 15 , wherein the motion map is a binary map and has a spatial resolution same as the intermediate image frame.

20. The system of claim 19 , wherein a value of a pixel in the motion map indicates whether to perform temporal or spatiotemporal denoising to the pixel.