Machine learning to process Monte Carlo rendered images
A method of rendering an image includes Monte Carlo rendering a scene to produce a noisy image. The noisy image is processed to render an output image. The processing applies a machine learning model that utilizes colors and/or features from the rendering system for denoising the noisy image and/or to for adaptively placing samples during rendering.
1. A method of producing an output image, the method comprising:
obtaining training images;
using machine learning incorporating a filter on the training images to output final filter parameters, wherein the using machine learning comprises training a neural network, and the training comprises:
extracting, determining and/or computing features from the training images;
computing test filter parameters using a machine learning model including applying the filter using the features to create a denoised image;
applying an error metric to the denoised image;
correcting the machine learning model based on the error metric including updating the testing filter parameters;
repeating the computing, the applying and the correcting to determine final filter parameters;
receiving a Monte Carlo rendered image that has noise;
executing the filter on the noisy image using the final filter parameters to generate an output image.
2. The method of claim 1 , wherein the training images include both ground truth training images and noisy training images.
3. The method of claim 1 wherein the extracting, determining and/or computing features includes:
determining primary features of the training images;
extracting and/or computing secondary features of the training images using the primary features.
4. The method of claim 3 , wherein the primary features include features selected from the group consisting: positions, colors, world positions, visibility, shading normals, texture values.
5. The method of claim 4 , wherein the secondary features include features selected from the group consisting of: variances and noise approximation in local regions, mean of primary features at various block sizes, standard deviation of the primary features at various block sizes, gradients of primary features, mean deviation of the primary features, median absolute deviation (MAD) of primary features, sampling rate.
6. The method of claim 1 , wherein the training images comprise ground truth sample images.
7. The method of claim 1 , wherein the filter comprises a cross-bilateral filter.
8. The method of claim 1 , wherein the filter comprises a cross non-local means filter.
9. The method of claim 1 , wherein the neural network is a one of a support vector machine, a random forest, deep neural network, multi-layer perceptron, convolutional network, deep convolutional network, recurrent neural network, autoencoder neural network, long short-term memory networks, and generative adversarial network.
10. The method of claim 1 , wherein the features comprise color, illumination or texture.