Machine learning based image processing techniques
A machine learning based image processing architecture and associated applications are disclosed herein. In some embodiments, a machine learning framework is trained to learn low level image attributes such as object/scene types, geometries, placements, materials and textures, camera characteristics, lighting characteristics, contrast, noise statistics, etc. Thereafter, the machine learning framework may be employed to detect such attributes in other images and process the images at the attribute level.
1. A method, comprising:
using a machine learning framework to identify a set of filters for removing noise from a sparsely ray traced input image, wherein different filters are identified for different portions of the input image and wherein the set of filters predicts pixel values that would result from ray tracing with more samples of rays; and
outputting an output image comprising a filtered version of the input image resulting from filtering the input image with the identified set of filters, wherein a quality of the output image is equivalent to ray tracing with more samples of rays than the sparsely ray traced input image.
2. The method of claim 1 , wherein the output image is a denoised version of the input image.
3. The method of claim 1 , wherein the sparsely ray traced input image violates a noise threshold and the output image satisfies the noise threshold.
4. The method of claim 1 , wherein the sparsely ray traced input image is ray traced with a small number of samples of rays.
5. The method of claim 1 , wherein the quality of the output image is equivalent to ray tracing with a large number of samples of rays.
6. The method of claim 1 , wherein the quality of the output image is equivalent to ray tracing with a number of samples of rays needed for complete convergence.
7. The method of claim 1 , wherein the method facilitates substantially reducing image render time.
8. The method of claim 1 , wherein the filters comprise spatial filters.
9. The method of claim 1 , wherein to identify a set of filters comprises to identify a set of one or more filter parameters.
10. The method of claim 1 , wherein the method facilitates effectively eliminating ray tracing with large numbers of samples.
11. The method of claim 1 , wherein a prescribed filter of the set of filters is associated with a corresponding noise signature.
12. The method of claim 11 , wherein the noise signature is based on one or more attributes.
13. The method of claim 12 , wherein the attributes comprise one or more attributes associated with object/scene types, geometries, placements, materials, textures, camera characteristics, lighting characteristics, numbers of samples, and contrast.
14. The method of claim 1 , wherein the machine learning framework is trained to learn noise signatures and corresponding filters.
15. The method of claim 1 , wherein the machine learning framework is trained on image datasets comprising ray traced snapshots at different sampling intervals.
16. The method of claim 1 , wherein the machine learning framework is trained on image datasets comprising permutations of a constrained set of objects associated with a prescribed scene type to which the input image belongs.
17. The method of claim 1 , wherein the output image comprises a photorealistic rendering.
18. The method of claim 1 , wherein the output image comprises a frame of an animation or a video sequence.
19. A system, comprising:
a processor configured to:
use a machine learning framework to identify a set of filters for removing noise from a sparsely ray traced input image, wherein different filters are identified for different portions of the input image and wherein the set of filters predicts pixel values that would result from ray tracing with more samples of rays; and
output an output image comprising a filtered version of the input image resulting from filtering the input image with the identified set of filters, wherein a quality of the output image is equivalent to ray tracing with more samples of rays than the sparsely ray traced input image; and
a memory coupled to the processor and configured to provide the processor with instructions.
20. A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
using a machine learning framework to identify a set of filters for removing noise from a sparsely ray traced input image, wherein different filters are identified for different portions of the input image and wherein the set of filters predicts pixel values that would result from ray tracing with more samples of rays; and
outputting an output image comprising a filtered version of the input image resulting from filtering the input image with the identified set of filters, wherein a quality of the output image is equivalent to ray tracing with more samples of rays than the sparsely ray traced input image.
21. The system of claim 19 , wherein the output image is a denoised version of the input image.
22. The system of claim 19 , wherein the sparsely ray traced input image violates a noise threshold and the output image satisfies the noise threshold.
23. The system of claim 19 , wherein the sparsely ray traced input image is ray traced with a small number of samples of rays.
24. The system of claim 19 , wherein the quality of the output image is equivalent to ray tracing with a large number of samples of rays.
25. The system of claim 19 , wherein the quality of the output image is equivalent to ray tracing with a number of samples of rays needed for complete convergence.
26. The system of claim 19 , wherein the system facilitates substantially reducing image render time.
27. The system of claim 19 , wherein the filters comprise spatial filters.
28. The system of claim 19 , wherein to identify a set of filters comprises to identify a set of one or more filter parameters.
29. The system of claim 19 , wherein the system facilitates effectively eliminating ray tracing with large numbers of samples.
30. The system of claim 19 , wherein a prescribed filter of the set of filters is associated with a corresponding noise signature.
31. The system of claim 30 , wherein the noise signature is based on one or more attributes.
32. The system of claim 31 , wherein the attributes comprise one or more attributes associated with object/scene types, geometries, placements, materials, textures, camera characteristics, lighting characteristics, numbers of samples, and contrast.
33. The system of claim 19 , wherein the machine learning framework is trained to learn noise signatures and corresponding filters.
34. The system of claim 19 , wherein the machine learning framework is trained on image datasets comprising ray traced snapshots at different sampling intervals.
35. The system of claim 19 , wherein the machine learning framework is trained on image datasets comprising permutations of a constrained set of objects associated with a prescribed scene type to which the input image belongs.
36. The system of claim 19 , wherein the output image comprises a photorealistic rendering.
37. The system of claim 19 , wherein the output image comprises a frame of an animation or a video sequence.
38. The computer program product of claim 20 , wherein the output image is a denoised version of the input image.
39. The computer program product of claim 20 , wherein the sparsely ray traced input image violates a noise threshold and the output image satisfies the noise threshold.
40. The computer program product of claim 20 , wherein the sparsely ray traced input image is ray traced with a small number of samples of rays.
41. The computer program product of claim 20 , wherein the quality of the output image is equivalent to ray tracing with a large number of samples of rays.
42. The computer program product of claim 20 , wherein the quality of the output image is equivalent to ray tracing with a number of samples of rays needed for complete convergence.
43. The computer program product of claim 20 , wherein the computer program product facilitates substantially reducing image render time.
44. The computer program product of claim 20 , wherein the filters comprise spatial filters.
45. The computer program product of claim 20 , wherein to identify a set of filters comprises to identify a set of one or more filter parameters.
46. The computer program product of claim 20 , wherein the computer program product facilitates effectively eliminating ray tracing with large numbers of samples.
47. The computer program product of claim 20 , wherein a prescribed filter of the set of filters is associated with a corresponding noise signature.
48. The computer program product of claim 47 , wherein the noise signature is based on one or more attributes.
49. The computer program product of claim 48 , wherein the attributes comprise one or more attributes associated with object/scene types, geometries, placements, materials, textures, camera characteristics, lighting characteristics, numbers of samples, and contrast.
50. The computer program product of claim 20 , wherein the machine learning framework is trained to learn noise signatures and corresponding filters.
51. The computer program product of claim 20 , wherein the machine learning framework is trained on image datasets comprising ray traced snapshots at different sampling intervals.
52. The computer program product of claim 20 , wherein the machine learning framework is trained on image datasets comprising permutations of a constrained set of objects associated with a prescribed scene type to which the input image belongs.
53. The computer program product of claim 20 , wherein the output image comprises a photorealistic rendering.
54. The computer program product of claim 20 , wherein the output image comprises a frame of an animation or a video sequence.