Machine learning to process Monte Carlo rendered images
A method of rendering an image includes Monte Carlo rendering a scene to produce a noisy image. The noisy image is processed to render an output image. The processing applies a machine learning model that utilizes colors and/or features from the rendering system for denoising the noisy image and/or to for adaptively placing samples during rendering.
1. A computer-implemented method of rendering an image, the method comprising: Monte Carlo rendering a scene with a rendering system to produce a noisy image; processing the noisy image to render an output image, wherein said processing comprises applying a machine learning model that utilizes colors and/or features from the rendering system for denoising the noisy image and/or for adaptively placing samples during rendering, wherein the applying machine learning comprises:
applying an error metric to measure the distance between filtered images and ground truth images;
applying backpropagation to minimize an energy function on results of the error metric.
2. The method of claim 1 , wherein the machine learning model had been trained with ground truth sample images prior to the applying.
3. The method of claim 1 wherein the denoising uses sample colors from the rendering system.
4. The method of claim 1 wherein the denoising uses sample features from the rendering system.
5. The method of claim 1 wherein the applying the machine learning comprises applying a machine learning algorithm directly to the noisy image to compute denoised pixel values of the output image.
6. The method of claim 4 , where the machine learning algorithm uses secondary features derived from the colors and/or features from the rendering system to compute the denoised pixel values of the output image.
7. The method of claim 1 wherein the denoising is implemented with an explicit filter and the applying machine learning comprises obtaining optimal parameters for the filter.
8. The method of claim 7 wherein the explicit filter comprises a cross-bilateral filter and the applying machine learning comprises obtaining optimal parameters for the cross-bilateral filter.
9. The method of claim 1 wherein the explicit filter comprises cross non-local means filter and the applying machine learning comprises obtaining optimal parameters for the cross non-local means filter.
10. The method of claim 1 , wherein the neural network is a one of a neural network, a support vector machine, a random forest, deep neural network, multi-layer perceptron, convolutional network, deep convolutional network, recurrent neural network, autoencoder neural network, long short-term memory networks, and generative adversarial network.
11. The method of claim 1 , wherein the features from the rendering system comprise illumination or texture.