Interactive path tracing on the web
A method renders photorealistic images in a web browser. The method is performed at a computing device having a general purpose processor and a graphics processing unit (GPU). The method includes obtaining an environment map and images of an input scene. The method also includes computing textures for the input scene including by encoding an acceleration structure of the input scene. The method further includes transmitting the textures to shaders executing on a GPU. The method includes generating samples of the input scene, by performing at least one path tracing algorithm on the GPU, according to the textures. The method also includes lighting or illuminating a sample of the input scene using the environment map, to obtain a lighted scene, and tone mapping the lighted scene. The method includes drawing output on a canvas, in the web browser, based on the tone-mapped scene to render the input scene.
1. A computer system for rendering images using path tracing, comprising:
one or more processors, including a general purpose processor and a graphics processing unit (GPU);
a display; and
memory;
wherein the memory stores one or more programs configured for execution by the one or more processors, and the one or more programs comprising instructions for:
obtaining an input image having a current pose;
rendering a current frame of the input image with a path-traced sample per pixel;
obtaining a prior frame corresponding to a prior pose, wherein the current frame and the prior frame have at least one overlapping pixel and each of the current frame and prior frame image data includes an alpha channel set to luminance contribution, for each pixel;
re-projecting samples from the prior frame into the current frame based on the alpha channel corresponding to each overlapping pixel with the current frame, including (i) blending diffuse light of the current frame with diffuse light of at least the prior frame using a long temporal filter, and (ii) blending specular light of the current frame with specular light of at least the prior frame using a short temporal filter; and
modifying the current frame with the re-projected samples, including storing a number of samples rendered in the alpha channel for each pixel.
2. The computer system of claim 1 , wherein obtaining the prior frame corresponding to the prior pose comprises determining if a first mesh identifier for the at least one overlapping pixel in the prior frame matches a second mesh identifier for the overlapping pixel in the current frame.
3. The computer system of claim 1 , wherein modifying the current frame with the re-projected samples comprises averaging a color channel value for the pixel in the current frame with corresponding values from the re-projected samples.
4. The computer system of claim 3 , wherein averaging a color channel comprises:
summing the color channel value for the pixel of the prior frame to the color channel value of the current frame; and
modifying the summed color channel value by the current frame's alpha channel value for that pixel.
5. The computer system of claim 1 , wherein re-projecting samples from the prior frame into the current frame comprises:
determining if a surface corresponding to a pixel of the current frame is visible in the prior frame; and
in accordance with a determination that the surface is visible in the prior frame, averaging color channels for the pixel with corresponding values from the re-projected samples.
6. The computer system of claim 1 , wherein the one or more programs further comprise instructions for:
detecting if a camera used to obtain the input image has moved; and
in response to detecting that the camera has moved, blending the current frame with the re-projected samples from the prior frame using an exponential average.
7. The computer system of claim 1 , wherein the one or more programs further comprise instructions for:
detecting if a camera used to obtain the input image is still; and
in response to detecting that the camera is still, linearly blending the current frame with the re-projected samples from the prior frame.
8. The computer system of claim 1 , wherein the one or more programs further comprise instructions for:
detecting if a camera used to obtain the input image is moving; and
in response to detecting that the camera is moving, blurring at least a portion of the current frame.
9. The computer system of claim 1 , wherein the one or more programs further comprise instructions for:
repeating obtaining a new input scene, rendering a current frame, and blending the current frame reusing samples.
10. The computer system of claim 1 , wherein (i) blending the diffuse light of the current frame with diffuse light of at least the prior frame using a long temporal filter, and (ii) blending the specular light of the current frame with specular light of at least the prior frame using a short temporal filter, is based on separate buffers for the specular light and the diffuse light.
11. A method of rendering images using path tracing, the method comprising:
obtaining an input image having a current pose;
rendering a current frame of the input image with a path-traced sample per pixel;
obtaining a prior frame corresponding to a prior pose, wherein the current frame and the prior frame have at least one overlapping pixel and each of the current frame and prior frame image data includes an alpha channel set to luminance contribution, for each pixel;
re-projecting samples from the prior frame into the current frame based on the alpha channel corresponding to each overlapping pixel with the current frame, including (i) blending diffuse light of the current frame with diffuse light of at least the prior frame using a long temporal filter, and (ii) blending specular light of the current frame with specular light of at least the prior frame using a short temporal filter; and
modifying the current frame with the re-projected samples, including storing a number of samples rendered in the alpha channel for each pixel.
12. The method claim 11 , wherein obtaining the prior frame corresponding to the prior pose comprises determining if a first mesh identifier for the at least one overlapping pixel in the prior frame matches a second mesh identifier for the overlapping pixel in the current frame.
13. The method of claim 11 , wherein modifying the current frame with the re-projected samples comprises averaging a color channel value for the pixel in the current frame with corresponding values from the re-projected samples.
14. The method of claim 13 , wherein averaging a color channel comprises:
summing the color channel value for the pixel of the prior frame to the color channel value of the current frame; and
modifying the summed color channel value by the current frame's alpha channel value for that pixel.
15. The method of claim 11 , wherein re-projecting samples from the prior frame into the current frame comprises:
determining if a surface corresponding to a pixel of the current frame is visible in the prior frame; and
in accordance with a determination that the surface is visible in the prior frame, averaging color channels for the pixel with corresponding values from the re-projected samples.
16. The method of claim 11 , further comprising:
detecting if a camera used to obtain the input image has moved; and
in response to detecting that the camera has moved, blending the current frame with the re-projected samples from the prior frame using an exponential average.
17. The method of claim 11 , further comprising:
detecting if a camera used to obtain the input image is still; and
in response to detecting that the camera is still, linearly blending the current frame with the re-projected samples from the prior frame.
18. The method of claim 11 , further comprising:
detecting if a camera used to obtain the input image is moving; and
in response to detecting that the camera is moving, blurring at least a portion of the current frame.
19. The method of claim 11 , further comprising:
repeating obtaining a new input scene, rendering a current frame, and blending the current frame reusing samples.
20. The method of claim 11 , wherein (i) blending the diffuse light of the current frame with diffuse light of at least the prior frame using a long temporal filter, and (ii) blending the specular light of the current frame with specular light of at least the prior frame using a short temporal filter, is based on separate buffers for the specular light and the diffuse light.