Detection of loss of details in a denoised image
A computer-implemented method for forming a dataset configured for learning a Convolutional Neural Network (CNN) architecture including an image feature extractor. It comprises providing pairs of images, each pair comprising a reference image and a respective denoised image. For each pair of images, the method provides the pair of images to a pre-trained CNN architecture similar to the one the formed dataset will be configured for. The method computes an error map representing a difference between a first normalized feature of the denoised image and a second normalized feature of the reference image, the first and second normalized features being the output of a same layer of the pre-trained CNN architecture and adds the respective denoised image and the error map to the dataset. This constitutes an improved solution with respect to forming a dataset for learning a CNN architecture to identify areas of degradation generated by a denoiser.
1. A computer-implemented method for forming a dataset configured for learning a Convolutional Neural Network (CNN) architecture, the CNN architecture comprising an image feature extractor, the method comprising:
obtaining pairs of images, each pair comprising a reference image and a respective denoised image, the reference image being a Monte-Carlo rendered ray-traced image, the respective denoised image being a result of inputting a respective noisy image of the reference image to a denoiser; and
for each pair of images:
providing the pair of images to a pre-trained CNN architecture similar to the one for which the formed dataset will be configured,
computing a difference between a first normalized feature of the denoised image and a second normalized feature of the reference image, the first and second normalized features being an output of a same layer of the pre-trained CNN architecture,
computing an error map representing the computed difference, computing the error map including:
down-sampling a resolution of the computed difference, and
computing the error map with the down-sampled resolution of the difference by creating an image having pixel values, each pixel of the error map having a color following a color scale that penalizes the computed difference, thereby obtaining a coarse error map, and
adding the respective denoised image and the error map to the dataset.
2. The method of claim 1 , wherein the pre-trained CNN architecture is a Visual Geometric Group (VGG) architecture.
3. The method of claim 2 , wherein the Visual Geometric Group (VGG) architecture is a VGG-19 architecture, and wherein the first normalized feature of the denoised image and the second normalized feature of the reference image are extracted from 7th and/or 9th and/or 12th layers of the VGG-19 architecture.
4. The method of claim 1 , wherein the adding further comprises:
adding to the dataset the respective denoised image, the error map, and a respective noisy image of the reference image, the noisy image being the one from which the denoised image has been obtained.
5. The method of claim 1 , wherein the respective denoised image is a result of inputting to a denoiser a lower sample count image of the reference image denoised by the denoiser.
6. A non-transitory computer-readable storage medium having recorded thereon a data structure representing a dataset formable according to a computer-implemented method for forming a dataset configured for learning a Convolutional Neural Network (CNN) architecture, the CNN architecture comprising an image feature extractor, the method comprising:
obtaining pairs of images, each pair comprising a reference image and a respective denoised image; and
for each pair of images:
providing the pair of images to a pre-trained CNN architecture similar to the one for which the formed dataset will be configured,
computing a difference between a first normalized feature of the denoised image and a second normalized feature of the reference image, the first and second normalized features being an output of a same layer of the pre-trained CNN architecture,
computing an error map representing the computed difference, computing the error map including:
down-sampling a resolution of the computed difference, and
computing the error map with the down-sampled resolution of the difference by creating an image having pixel values, each pixel of the error map having a color following a color scale that penalizes the computed difference, thereby obtaining a coarse error map, and
adding the respective denoised image and the error map to the dataset.
7. The non-transitory computer-readable storage medium of claim 6 , wherein the pre-trained CNN architecture is a Visual Geometric Group (VGG) architecture.
8. The non-transitory computer-readable storage medium of claim 7 , wherein the Visual Geometric Group (VGG) architecture is a VGG-19 architecture, and wherein the first normalized feature of the denoised image and the second normalized feature of the reference image are extracted from 7th and/or 9th and/or 12th layers of the VGG-19 architecture.
9. The non-transitory computer-readable storage medium of claim 6 , wherein the adding further comprises:
adding to the dataset the respective denoised image, the error map, and a respective noisy image of the reference image, the noisy image being the one from which the denoised image has been obtained.
10. A computer-implemented method of applying a formed dataset configured for learning a Convolutional Neural Network (CNN) architecture, the CNN architecture comprising an image feature extractor, the method comprising:
obtaining pairs of images, each pair comprising a reference image and a respective denoised image, the reference image being a Monte-Carlo rendered ray-traced image, the respective denoised image being a result of inputting a respective noisy image of the reference image to a denoiser;
for each pair of images:
providing the pair of images to a pre-trained CNN architecture similar to the one for which the formed dataset will be configured,
computing a difference between a first normalized feature of the denoised image and a second normalized feature of the reference image, the first and second normalized features being an output of a same layer of the pre-trained CNN architecture,
computing an error map representing the computed difference, computing the error map including:
down-sampling a resolution of the computed difference, and
computing the error map with the down-sampled resolution of the difference by creating an image having pixel values, each pixel of the error map having a color following a color scale that penalizes the computed difference, thereby obtaining a coarse error map, and
adding the respective denoised image and the error map to the dataset; and
learning the CNN architecture based on the dataset.
11. The method of claim 10 , wherein the pre-trained CNN architecture is a Visual Geometric Group (VGG) architecture.
12. The method of claim 11 , wherein the Visual Geometric Group (VGG) architecture is a VGG-19 architecture, and wherein the first normalized feature of the denoised image and the second normalized feature of the reference image are extracted from 7th and/or 9th and/or 12th layers of the VGG-19 architecture.
13. A non-transitory computer-readable storage medium having recorded thereon a computer program, the computer program comprising instructions for performing a computer-implemented method for forming a dataset configured for learning a Convolutional Neural Network (CNN) architecture, the CNN architecture comprising an image feature extractor, the method comprising:
obtaining pairs of images, each pair comprising a reference image and a respective denoised image, the reference image being a Monte-Carlo rendered ray-traced image, the respective denoised image being a result of inputting a respective noisy image of the reference image to a denoiser; and
for each pair of images:
providing the pair of images to a pre-trained CNN architecture similar to the one for which the formed dataset will be configured,
computing a difference between a first normalized feature of the denoised image and a second normalized feature of the reference image, the first and second normalized features being an output of a same layer of the pre-trained CNN architecture,
computing an error map representing the computed difference, computing the error map including:
down-sampling a resolution of the computed difference, and
computing the error map with the down-sampled resolution of the difference by creating an image having pixel values, each pixel of the error map having a color following a color scale that penalizes the computed difference, thereby obtaining a coarse error map, and
adding the respective denoised image and the error map to the dataset.
14. The non-transitory computer-readable storage medium of claim 13 , wherein the pre-trained CNN architecture is a Visual Geometric Group (VGG) architecture.
15. The non-transitory computer-readable storage medium of claim 14 , wherein the Visual Geometric Group (VGG) architecture is a VGG-19 architecture, and wherein the first normalized feature of the denoised image and the second normalized feature of the reference image are extracted from 7th and/or 9th and/or 12th layers of the VGG-19 architecture.
16. The non-transitory computer-readable storage medium of claim 13 , wherein the adding further comprises:
adding to the dataset the respective denoised image, the error map, and a respective noisy image of the reference image, the noisy image being the one from which the denoised image has been obtained.