Systems and methods for reducing noise in video streams
View Patent ↗Disclosed are apparatus and methods for denoising a video stream of a camera. A current frame of the video stream and a temporally adjacent frame of the video stream that has been previously spatially and temporally denoised are obtained. The current frame is first spatially denoised, while preserving edges in such current frame to generate a plurality of spatially denoised pixels for the current frame. A particular pixel of the current frame is then both spatially and temporally denoised based on a weighted averaging of the particular pixel's associated spatially denoised pixel from the current frame and a plurality of pixels from the temporally adjacent frame that have already been spatially and temporally denoised.
1. A method of denoising a video stream of a camera, the method comprising:
obtaining a current frame of the video stream of the camera;
obtaining a temporally adjacent frame of the video stream that has been previously spatially and temporally denoised;
spatially denoising the current frame while preserving edges in such current frame to generate a plurality of spatially denoised pixels for the current frame; and
after the current frame is spatially denoised, temporally denoising a particular pixel of the current frame based on a weighted averaging of the particular pixel's associated already spatially denoised pixel from the current frame and a plurality of pixels from the temporally adjacent frame that have already been spatially and temporally denoised.
2. The method of claim 1 , wherein spatially denoising the current frame includes applying a bilateral filtering technique to each pixel of the current frame based on a plurality of proximate pixels that are within a region surrounding such each pixel.
3. The method of claim 2 , wherein each pixel of the current frame being denoised and its proximate pixels include a 3 by 3 grid of pixels surrounding such pixel being denoised.
4. The method of claim 2 , wherein the bilateral filtering technique includes providing a weight for each proximate pixel that is inversely proportional to both a Euclidean distance and a color difference.
5. The method of claim 4 , wherein the plurality of pixels from the temporally adjacent frame include pixels at locations corresponding to the particular pixel and the proximate pixels.
6. The method of claim 5 , wherein the plurality of pixels from the temporally adjacent frame further include interpolated pixels at locations not corresponding to the particular pixel and the proximate pixels.
7. The method of claim 5 , wherein a weight of the particular pixel's associated spatially denoised pixel has a value of one.
8. A camera comprising:
an image sensor configured to receive light and generate a detected signal;
one or more processors and memory that are operable to perform the following:
obtaining a current frame of a video stream of the camera wherein the video stream is generated based on the detected signal;
obtaining a temporally adjacent frame of the video stream that has been previously spatially and temporally denoised;
spatially denoising the current frame while preserving edges in such current frame to generate a plurality of spatially denoised pixels for the current frame; and
after the current frame is spatially denoised, temporally denoising a particular pixel of the current frame based on a weighted averaging of the particular pixel's associated already spatially denoised pixel from the current frame and a plurality of pixels from the temporally adjacent frame that have already been spatially and temporally denoised.
9. The camera of claim 8 , wherein spatially denoising the current frame includes applying a bilateral filtering technique to each pixel of the current frame based on a plurality of proximate pixels that are within a region surrounding such each pixel.
10. The camera of claim 9 , wherein each pixel of the current frame being denoised and its proximate pixels include a 3 by 3 grid of pixels surrounding such pixel being denoised.
11. The camera of claim 9 , wherein the bilateral filtering technique includes providing a weight for each proximate pixel that is inversely proportional to both a Euclidean distance and a color difference.
12. The camera of claim 11 , wherein the plurality of pixels from the temporally adjacent frame include pixels at locations corresponding to the particular pixel and the proximate pixels.
13. The camera of claim 12 , wherein the plurality of pixels from the temporally adjacent frame further include interpolated pixels at locations not corresponding to the particular pixel and the proximate pixels.
14. The camera of claim 12 , wherein a weight of the particular pixel's associated spatially denoised pixel has a value of one.
15. At least one non-transitory computer readable storage medium having computer program instructions stored thereon that are arranged to perform the following operations:
obtaining a current frame of a video stream of a camera;
obtaining a temporally adjacent frame of the video stream that has been previously spatially and temporally denoised;
spatially denoising the current flame while preserving edges in such current flame to generate a plurality of spatially denoised pixels for the current frame; and
after the current flame is spatially denoised, temporally denoising a particular pixel of the current flame based on a weighted averaging of the particular pixel's associated already spatially denoised pixel from the current flame and a plurality of pixels from the temporally adjacent flame that have already been spatially and temporally denoised.
16. The at least one non-transitory computer readable storage medium of claim 15 , wherein spatially denoising the current frame includes applying a bilateral filtering technique to each pixel of the current frame based on a plurality of proximate pixels that are within a region surrounding such each pixel.
17. The at least one non-transitory computer readable storage medium of claim 16 , wherein the bilateral filtering technique includes providing a weight for each proximate pixel that is inversely proportional to both a Euclidean distance and a color difference.
18. The at least one non-transitory computer readable storage medium of claim 17 , wherein the plurality of pixels from the temporally adjacent frame include pixels at locations corresponding to the particular pixel and the proximate pixels.
19. The at least one non-transitory computer readable storage medium of claim 18 , wherein the plurality of pixels from the temporally adjacent frame further include interpolated pixels at locations not corresponding to the particular pixel and the proximate pixels.
20. The at least one non-transitory computer readable storage medium of claim 18 , wherein a weight of the particular pixel's associated spatially denoised pixel has a value of one.