Preserving detail in denoised images for content generation systems and applications
Approaches presented herein provide for the maintaining of fine details that might be removed by a denoiser used to reduce an amount of noise in an image. An input image can be provided to a denoiser, and can also can be simultaneously processed to extract pixel data that may correspond to fine detail or high frequency features. Individual pixels of an image can have a value determined for a material property sampled for that pixel location, and that value can be compared against an average material property value determined for neighboring pixels. The ratio of material values can be multiplied by the value of a corresponding pixel of the denoised image, for any or all pixel locations, to obtain final pixel values for an output image that include less noise than the original image but represent fine detail that may otherwise have been lost during the denoising process.
1 . A computer-implemented method, comprising:
providing a generated image to a denoiser to obtain a denoised image;
determining, for a pixel of the generated image, a material reflectance value;
determining, for a set of neighboring pixels of the pixel in the generated image, an average material reflectance value;
generating a ratio of the material reflectance value for the pixel to the average material reflectance value of the set of neighboring pixels; and
generating an output image, wherein at least one value of at least one pixel of the output image is determined by multiplying a value of the ratio by a pixel value at a corresponding pixel location of the denoised image.
2 . The computer-implemented method of claim 1 , wherein the material reflectance value is determined using a bidirectional reflectance distribution function (BDRF).
3 . The computer-implemented method of claim 1 , wherein the set of neighboring pixels includes at least four neighboring pixels adjacent to the pixel in the generated image.
4 . The computer-implemented method of claim 1 , further comprising:
calculating a ratio of the material reflectance value to the average material reflectance value for each pixel in the generated image to be multiplied by respective pixel values at corresponding pixel locations of the denoised image.
5 . The computer-implemented method of claim 1 , further comprising:
excluding, from the set of neighboring pixels, a neighboring pixel determined to have at least a distance or a surface normal direction differing, by more than a threshold amount, from the distance or the surface normal direction of the pixel.
6 . The computer-implemented method of claim 1 , wherein the generated image is one of a sequence of generated images, and wherein individual images of the sequence are generated using offset camera positions to provide for sub-pixel sampling variation.
7 . The computer-implemented method of claim 1 , wherein the generated image is generated using one or more light transport simulation techniques.
8 . The computer-implemented method of claim 1 , further comprising:
providing the output image as input to an upsampling process.
9 . The computer-implemented method of claim 1 , wherein the set of neighboring pixels includes at least two neighboring pixels.
10 . A processor, comprising:
one or more processing units to:
provide a generated image to a denoiser to obtain a denoised image;
determine, for a pixel of the generated image, a material reflectance value;
determine, for a set of neighboring pixels of the pixel in the generated image, an average material reflectance value;
generate a ratio of the material reflectance value for the pixel to the average material reflectance value of the set of neighboring pixels; and
generate an output image, wherein at least one value of at least one pixel of the output image is determined by multiplying a value of the ratio by a pixel value at a corresponding pixel location of the denoised image.
11 . The processor of claim 10 , wherein the one or more processing units are further to:
calculate a ratio of the material reflectance value to the average material reflectance value for each pixel in the generated image to be multiplied by respective pixel values at corresponding pixel locations of the denoised image.
12 . The processor of claim 10 , wherein the one or more processing units are further to:
exclude, from the set of neighboring pixels, a neighboring pixel determined to have at least a distance or a surface normal direction differing, by more than a threshold amount, from the distance or the surface normal direction of the pixel.
13 . The processor of claim 10 , wherein the generated image is one of a sequence of generated images, and wherein individual images of the sequence are generated using offset camera positions to provide for sub-pixel sampling variation.
14 . The processor of claim 10 , wherein the material reflectance value is determined using a bidirectional reflectance distribution function (BDRF).
15 . The processor of claim 10 , wherein the processor is comprised in at least one of:
a system for performing simulation operations;
a system for performing simulation operations to test or validate autonomous machine applications;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for rendering graphical output;
a system for performing deep learning operations;
a system implemented using an edge device;
a system for generating or presenting virtual reality (VR) content;
a system for generating or presenting augmented reality (AR) content;
a system for generating or presenting mixed reality (MR) content;
a system incorporating one or more Virtual Machines (VMs);
a system implemented at least partially in a data center;
a system for performing hardware testing using simulation;
a system for synthetic data generation;
a system for performing generative AI operations using a large language model (LLM),
a collaborative content creation platform for 3D assets; or
a system implemented at least partially using cloud computing resources.
16 . A system, comprising:
one or more processors to generate an output image by, in part, denoising an input image to generate a denoised image and generating, for individual pixels of the input image, a ratio of a material reflectance value for the individual pixel to an average material reflectance value of a set of neighboring pixels, wherein at least one value of at least one pixel of the output image is determined by multiplying a value of the ratio by a pixel value at a corresponding pixel location of the denoised image.
17 . The system of claim 16 , wherein the one or more processors are further to:
calculate a ratio of the material reflectance value to the average material reflectance value for each pixel in the output image to be multiplied by respective pixel values at corresponding pixel locations of the denoised image.
18 . The system of claim 16 , wherein the one or more processors are further to:
exclude, from the set of neighboring pixels, a neighboring pixel determined to have at least a distance or a surface normal direction differing, by more than a threshold amount, from the distance or the surface normal direction of the individual pixel.
19 . The system of claim 16 , wherein the output image is one of a sequence of generated images, and wherein individual images of the sequence are generated using offset camera positions to provide for sub-pixel sampling variation.
20 . The system of claim 16 , wherein the system comprises at least one of:
a system for performing simulation operations;
a system for performing simulation operations to test or validate autonomous machine applications;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for rendering graphical output;
a system for performing deep learning operations;
a system for performing generative AI operations using a large language model (LLM),
a system implemented using an edge device;
a system for generating or presenting virtual reality (VR) content;
a system for generating or presenting augmented reality (AR) content;
a system for generating or presenting mixed reality (MR) content;
a system incorporating one or more Virtual Machines (VMs);
a system implemented at least partially in a data center;
a system for performing hardware testing using simulation;
a system for synthetic data generation;
a collaborative content creation platform for 3D assets; or
a system implemented at least partially using cloud computing resources.