Denoising dynamically ray-traced scenes using temporal and spatial variances of historical pixel values
In various examples, systems and methods are disclosed relating to historical reset. One method includes determining at least one history buffer for a frame, determining, in a spatial domain, a spatial component of the accumulated pixel value at the pixel location based on a first spatial moment and a second spatial moment, determining, in a temporal domain, a temporal component of the accumulated pixel value at the pixel location based on a first temporal moment and a second temporal moment. The method further includes determining a pixel value range based at least on the spatial component and the temporal component, determining an amount of historical reset to apply, and updating the accumulated pixel value based at least on the amount of historical reset.
1. A method, comprising:
determining at least one history buffer for a frame, the at least one history buffer comprising an accumulated pixel value at a pixel location of the frame;
determining, in a spatial domain, a spatial component of the accumulated pixel value at the pixel location based on a first spatial moment and a second spatial moment, wherein the first spatial moment corresponds to a set of accumulated pixel values within a spatial region comprising the pixel location in the at least one history buffer, and wherein the second spatial moment corresponds to one or more changes in the set of accumulated pixel values within the spatial region in the at least one history buffer;
determining, in a temporal domain, a temporal component of the accumulated pixel value at the pixel location based on a first temporal moment and a second temporal moment, wherein the first temporal moment comprises the accumulated pixel value, and wherein the second temporal moment comprises a temporal variance corresponding to one or more changes in the accumulated pixel value at the pixel location;
determining a pixel value range based at least on the spatial component in the spatial domain and the temporal component in the temporal domain;
determining an amount of historical reset to apply to the at least one history buffer based on at least one of: the accumulated pixel value at the pixel location of the at least one history buffer, a current pixel value of input data at the pixel location of the frame, and the pixel value range; and
updating the accumulated pixel value based at least on the amount of historical reset.
2. The method of claim 1 , wherein determining the amount of historical reset of the at least one history buffer is in response to the current pixel value of the input data at the pixel location of the frame.
3. The method of claim 1 , wherein the determining the pixel value range is further based on a spatial tolerance and a temporal tolerance, wherein the spatial component is scaled by the spatial tolerance and the temporal component is scaled by the temporal tolerance, wherein at least one of the spatial tolerance or the temporal tolerance is based on a light transport technique or implementation.
4. The method of claim 1 , wherein the at least one history buffer is a multi-channel buffer storing a plurality of color channels and a mean of a square of a luminance at the pixel location, and wherein the one or more changes in the accumulated pixel value is stored as the mean of the square of the luminance in the multi-channel buffer.
5. The method of claim 1 , wherein the amount of historical reset is scaled according to a tuning parameter corresponding to a blending rate of input data with the at least one history buffer.
6. The method of claim 1 , wherein updating the accumulated pixel value comprises applying a linear interpolation (LERP) factor to the accumulated pixel value and the current pixel value to determine an updated pixel value, wherein the LERP factor is based on the amount of historical reset.
7. The method of claim 6 , wherein updating the accumulated pixel value comprises resetting the accumulated pixel value based on the LERP factor, wherein the LERP factor effects an amount of reset of the accumulated pixel value towards the current pixel value.
8. The method of claim 1 , wherein the accumulated pixel value corresponds to one or more of a luminance space, a color space, or chrominance space, and wherein the luminance space comprises an intensity component, the color space comprises a plurality of color components, and the chrominance space comprises a color variation component.
9. The method of claim 1 , further comprising:
providing the updated accumulated pixel value to the at least one history buffer, wherein updating the accumulated pixel value occurs during an execution of a light transport simulation operation for the frame, and wherein the accumulated pixel value is stored using the at least one history buffer; and
outputting, to a display device, content comprising an updated pixel value of the updated accumulated pixel value corresponding to the at least one history buffer.
10. A system, comprising:
one or more processing units to implement:
a temporal accumulator system to temporally accumulate pixel data associated with at least one history buffer for a frame and provide the temporally accumulated pixel data, the at least one history buffer comprising an accumulated pixel value at a pixel location of the frame;
a history system to receive the temporally accumulated pixel data corresponding to the accumulated pixel value, determine a pixel value range based at least on a spatial component in a spatial domain and a temporal component in a temporal domain, determine an amount of historical reset of the at least one history buffer based at least on the accumulated pixel value at the pixel location, a current pixel value of input data at the pixel location of the frame, and the pixel value range, and update the accumulated pixel value based at least on the amount of historical reset; and
a spatial filterer system to apply one or more spatial filters to the updated accumulated pixel value and output the updated and spatially filtered accumulated pixel value;
wherein the determining the pixel value range is further based on a spatial tolerance and a temporal tolerance, wherein the spatial component is scaled by the spatial tolerance and the temporal component is scaled by the temporal tolerance, wherein at least one of the spatial tolerance or the temporal tolerance is based on a light transport simulation technique or implementation.
11. The system of claim 10 , wherein the spatial component is based on a first spatial moment and a second spatial moment, and wherein the temporal component is based on a first temporal moment and a second temporal moment.
12. The system of claim 11 , wherein the first spatial moment comprises a mean of a set of accumulated pixel values within a spatial region comprising the pixel location in the at least one history buffer, and wherein the second spatial moment is a spatial variance corresponding to one or more changes in the set of accumulated pixel values within the spatial region in the at least one history buffer.
13. The system of claim 11 , wherein the first temporal moment comprises the accumulated pixel value, and wherein the second temporal moment comprises a temporal variance corresponding to one or more changes in the accumulated pixel value at the pixel location over a plurality of history buffers.
14. The system of claim 10 , wherein the at least one history buffer is a multi-channel buffer storing a plurality of color channels and a mean of a square of a luminance at the pixel location, and wherein one or more changes in the accumulated pixel value is stored as the mean of the square of the luminance in the multi-channel buffer.
15. The system of claim 10 , wherein the amount of historical reset is scaled according to a tuning parameter corresponding to a blending rate of input data with the at least one history buffer.
16. The system of claim 10 , wherein updating the accumulated pixel value comprises applying a linear interpolation (LERP) factor to the accumulated pixel value and the current pixel value to determine an updated pixel value, wherein the LERP factor is based on the amount of historical reset.
17. The system of claim 16 , wherein updating the accumulated pixel value comprises resetting the accumulated pixel value based on the LERP factor, wherein the LERP factor effects an amount of reset of the accumulated pixel value towards the current pixel value.
18. The system of claim 10 , wherein:
the history system is further to provide the updated accumulated pixel value to the at least one history buffer, wherein updating the accumulated pixel value occurs during an execution of a light transport simulation operation for the frame, and wherein the accumulated pixel value is stored using the at least one history buffer; and
the spatial filterer system is further to provide, as output to a display device, content comprising an updated pixel value of updated and spatially filtered accumulated pixel value.
19. A system, comprising:
an application programming interface (API) to interact with one or more applications executed using one or more circuits, the API to cause the one or more circuits to:
determine at least one history buffer for a frame, the at least one history buffer comprising an accumulated pixel value at a pixel location of the frame;
determine a spatial component of the accumulated pixel value at the pixel location based on a first spatial moment and a second spatial moment, wherein the first spatial moment comprises a mean of a set of accumulated pixel values within a spatial region comprising the pixel location in the at least one history buffer, and wherein the second spatial moment is a spatial variance corresponding to one or more changes in the set of accumulated pixel values within the spatial region in the at least one history buffer;
determine a temporal component of the accumulated pixel value at the pixel location based on a first temporal moment and a second temporal moment, wherein the first temporal moment comprises the accumulated pixel value, and wherein the second temporal moment comprises a temporal variance corresponding to one or more changes in the accumulated pixel value at the pixel location over a plurality of history buffers;
determine a pixel value range based at least on the spatial component in a spatial domain and the temporal component in a temporal domain;
determine an amount of historical reset of the at least one history buffer based on at least one of: the accumulated pixel value at the pixel location of the at least one history buffer, a current pixel value of input data at the pixel location of the frame, or the pixel value range; and
update the accumulated pixel value based at least on the amount of historical reset.