Noise reconstruction for image denoising
The present disclosure provides example apparatuses, methods, and devices for denoising an image. One example apparatus performs operations including receiving an input image. A trained artificial intelligence model is implemented to form an estimate of a noise pattern in the input image and form an output image by subtracting the estimate of the noise pattern from the input image, where the model is configured to form the estimate of the noise pattern, and the estimate of the noise pattern is representative of a noise pattern that is characteristic to a specific image sensor type.
1. A computer-implemented method for training a model to perform noise reduction on images, the method comprising:
receiving a plurality of training images;
receiving a plurality of noise signatures; and
for each of the plurality of training images:
(i) selecting one of the plurality of noise signatures and applying that noise signature to the respective training image to form a noisy input image;
(ii) forming a first noise estimate in the noisy input image by implementing a candidate version of the model on the noisy input image and forming an estimate of the respective training image by subtracting the first noise estimate from the noisy input image;
(iii) forming a second noise estimate by implementing the candidate version of the model on the respective training image and the selected noise signature; and
(iv) adapting the candidate version of the model in dependence on (a) a difference between the respective training image and the estimate of the respective training image and (b) a difference between the second noise estimate and the selected noise signature.
2. The computer-implemented method as claimed in claim 1 , wherein the forming step (ii) is performed in a first pathway and the forming step (iii) is performed in a second pathway.
3. The computer-implemented method as claimed in claim 2 , wherein each of the first and second pathways comprises an encoder-decoder network.
4. The computer-implemented method as claimed in claim 3 , wherein a plurality of weights of a plurality of decoders of the first and second pathways are shared.
5. The computer-implemented method as claimed in claim 2 , wherein the first pathway and the second pathway are each based on a fully convolutional network.
6. The computer-implemented method as claimed in claim 2 , wherein the second pathway implements an unsupervised learning method.
7. The computer-implemented method as claimed in claim 2 , wherein the first pathway comprises one or more skip connections.
8. The computer-implemented method as claimed in claim 1 , wherein each of the plurality of training images is a RAW image or an RGB image.
9. The computer-implemented method as claimed in claim 1 , wherein the model is a convolutional neural network.
10. A device for training a model to perform noise reduction on images, comprising:
one or more processors; and
a non-transitory computer readable medium storing a program to be executed by the one or more processors, wherein the program comprises instructions that cause the device to perform operations comprising:
receiving a plurality of training images;
receiving a plurality of noise signatures; and
for each of the plurality of training images:
(i) selecting one of the plurality of noise signatures and applying that noise signature to the respective training image to form a noisy input image;
(ii) forming a first noise estimate in the noisy input image by implementing a candidate version of the model on the noisy input image and forming an estimate of the respective training image by subtracting the first noise estimate from the noisy input image;
(iii) forming a second noise estimate by implementing the candidate version of the model on the respective training image and the selected noise signature; and
(iv) adapting the candidate version of the model in dependence on (a) a difference between the respective training image and the estimate of the respective training image and (b) a difference between the second noise estimate and the selected noise signature.
11. The device as claimed in claim 10 , wherein the forming step (ii) is performed in a first pathway and the forming step (iii) is performed in a second pathway.
12. The device as claimed in claim 11 , wherein each of the first and second pathways comprises an encoder-decoder network.
13. The device as claimed in claim 12 , wherein a plurality of weights of a plurality of decoders of the first and second pathways are shared.
14. The device as claimed in claim 11 , wherein the first pathway and the second pathway are each based on a fully convolutional network.