Temporal and spatial video noise reduction
A system access a reference frame and temporally adjacent frames. For each portion of the reference image frame, the system calculates a pixel distance value between the portion of the reference image frame and a corresponding portion of each temporally adjacent image frame. If the pixel distance value indicates a potential ghosting artifact, the system computes a set of spatial noise reduction values for the image portion. Otherwise, the system computes a set of temporal noise reduction values for the image portion. The system blends the sets of computed spatial noise reduction values and the sets of computed temporal noise reduction values, and generates a modified reference image frame based on the blended set of noise reduction values.
1. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
obtaining a current input image from a sequence of input images, the current input image having a current temporal location in the sequence of input images;
obtaining a temporally adjacent image from the sequence of input images, the temporally adjacent image having a temporal location adjacent to the current temporal location;
generating a denoised image by denoising the current input image based on the temporally adjacent image, wherein denoising the current input image includes:
obtaining a current portion of the current input image;
generating a denoised portion by denoising the current portion, wherein denoising the current portion includes:
obtaining a difference value for the current portion, wherein obtaining the difference value for the current portion includes:
in response to a determination that a signal-to-noise ratio for the current portion exceeds a defined signal-to-noise threshold, obtaining, as the difference value for the current portion, a first candidate difference value indicating a first difference between the current portion and a corresponding portion of the temporally adjacent image; and
in response to a determination that the defined signal-to-noise threshold exceeds the signal-to-noise ratio for the current portion, obtaining, as the difference value for the current portion, a second candidate difference value, wherein obtaining the second candidate difference value includes:
in response to a determination that the first candidate difference value exceeds a defined aliasing threshold, obtaining a defined value as the second candidate difference value;
in response to a determination that the defined aliasing threshold exceeds the first candidate difference value, obtaining a second difference between the current portion and the corresponding portion of the temporally adjacent image;
in response to a determination that the second difference exceeds the defined value, obtaining the defined value as the second candidate difference value; and
in response to a determination that the defined value exceeds the second difference, obtaining the second difference as the second candidate difference value;
in response to a determination that the difference value for the current portion exceeds a defined maximum value, obtaining, as the denoised portion, a spatial average based on portions of the current input image spatially adjacent to the current portion;
in response to a determination that a defined minimum value exceeds the difference value for the current portion, obtaining, as the denoised portion, a temporal average based on a portion of the temporally adjacent image corresponding to the current portion; and
in response to a determination that the difference value for the current portion exceeds the defined minimum value and the defined maximum value exceeds the difference value for the current portion, obtaining, as the denoised portion, a blended average based on the temporal average and the spatial average;
including the denoised portion in the denoised image; and
outputting the denoised image.
2. The non-transitory computer-readable storage medium of claim 1 , wherein the current portion includes a pixel.
3. The non-transitory computer-readable storage medium of claim 1 , wherein:
obtaining the first candidate difference value includes using a first amount of blurring; and
obtaining the second difference includes using a second amount of blurring greater than the first amount of blurring.
4. The non-transitory computer-readable storage medium of claim 1 , wherein obtaining the blended average includes:
obtaining a weight for the spatial average such that the weight for the spatial average is proportional to the difference value for the current portion.
5. A method comprising:
obtaining a current input image from a sequence of input images, the current input image having a current temporal location in the sequence of input images;
generating a denoised image by denoising the current input image, wherein denoising the current input image includes:
obtaining a current portion of the current input image;
in response to a determination that a defined maximum value exceeds a difference value for the current portion, wherein the difference value is a pixel intensity difference value, and wherein the difference value for the current portion exceeds a defined minimum value, obtaining, as a denoised portion, a blended average based on a spatial average for the current portion and a temporal average for the current portion; and
including the denoised portion in the denoised image; and
outputting the denoised image.
6. The method of claim 5 , wherein denoising the current input image includes:
obtaining the spatial average based on portions of the current input image neighboring the current portion.
7. The method of claim 5 , wherein denoising the current input image includes:
in response to a determination that the difference value for the current portion exceeds the defined maximum value, obtaining, as the denoised portion, the spatial average.
8. The method of claim 5 , wherein denoising the current input image includes:
obtaining the temporal average based on a portion of a temporally adjacent image from the sequence of input images, wherein the portion of the temporally adjacent image corresponds to the current portion.
9. The method of claim 8 , wherein denoising the current input image includes obtaining the temporally adjacent image from the sequence of input images.
10. The method of claim 5 , wherein denoising the current input image includes:
in response to a determination that the defined minimum value exceeds the difference value for the current portion, obtaining, as the denoised portion, the temporal average.
11. The method of claim 5 , wherein denoising the current input image includes obtaining the difference value for the current portion.
12. The method of claim 11 , wherein obtaining the difference value for the current portion includes:
in response to a determination that a signal-to-noise ratio for the current portion exceeds a defined signal-to-noise threshold, obtaining, as the difference value for the current portion, a first candidate difference value indicating a first difference between the current portion and a corresponding portion of the temporally adjacent image.
13. The method of claim 12 , wherein obtaining the difference value for the current portion includes:
in response to a determination that the defined signal-to-noise threshold exceeds the signal-to-noise ratio for the current portion, obtaining, as the difference value for the current portion, a second candidate difference value.
14. The method of claim 13 , wherein obtaining the second candidate difference value includes:
in response to a determination that the first candidate difference value exceeds a defined aliasing threshold, obtaining a defined value as the second candidate difference value;
in response to a determination that the defined aliasing threshold exceeds the first candidate difference value, obtaining a second difference between the current portion and the corresponding portion of the temporally adjacent image;
in response to a determination that the second difference exceeds the defined value, obtaining the defined value as the second candidate difference value; and
in response to a determination that the defined value exceeds the second difference, obtaining the second difference as the second candidate difference value.
15. The method of claim 14 , wherein:
obtaining the first candidate difference value includes using a first amount of blurring; and
obtaining the second difference includes using a second amount of blurring greater than the first amount of blurring.
16. The method of claim 5 , wherein obtaining the blended average includes:
obtaining a weight for the spatial average such that the weight for the spatial average is proportional to the difference value for the current portion.
17. The method of claim 5 , wherein the current portion includes a pixel.
18. An apparatus comprising:
an image signal processor, the image signal processor configured to:
obtain a current input image from a sequence of input images, the current input image having a current temporal location in the sequence of input images;
generate a denoised image by denoising the current input image, wherein denoising the current input image includes:
obtaining a current portion of the current input image;
in response to a determination that a defined maximum value exceeds a difference value for the current portion, wherein the difference value is a pixel intensity difference value, and wherein the difference value for the current portion exceeds a defined minimum value, obtaining, as a denoised portion, a blended average based on a spatial average for the current portion and a temporal average for the current portion; and
including the denoised portion in the denoised image; and
outputting the denoised image.
19. The apparatus of claim 18 , wherein denoising the current input image includes:
obtaining the spatial average based on portions of the current input image neighboring the current portion;
in response to a determination that the difference value for the current portion exceeds the defined maximum value, obtaining, as the denoised portion, the spatial average; and
in response to a determination that the defined minimum value exceeds the difference value for the current portion, obtaining, as the denoised portion, the temporal average.
20. The apparatus of claim 18 , wherein denoising the current input image includes:
obtaining the temporal average based on a portion of a temporally adjacent image from the sequence of input images, wherein the portion of the temporally adjacent image corresponds to the current portion.
21. The apparatus of claim 18 , wherein denoising the current input image includes obtaining the difference value for the current portion, and wherein obtaining the difference value for the current portion includes:
in response to a determination that a signal-to-noise ratio for the current portion exceeds a defined signal-to-noise threshold, obtaining, as the difference value for the current portion, a first candidate difference value indicating a first difference between the current portion and a corresponding portion of the temporally adjacent image; and
in response to a determination that the defined signal-to-noise threshold exceeds the signal-to-noise ratio for the current portion, obtaining, as the difference value for the current portion, a second candidate difference value, wherein obtaining the second candidate difference value includes:
in response to a determination that the first candidate difference value exceeds a defined aliasing threshold, obtaining a defined value as the second candidate difference value;
in response to a determination that the defined aliasing threshold exceeds the first candidate difference value, obtaining a second difference between the current portion and the corresponding portion of the temporally adjacent image;
in response to a determination that the second difference exceeds the defined value, obtaining the defined value as the second candidate difference value; and
in response to a determination that the defined value exceeds the second difference, obtaining the second difference as the second candidate difference value.