3D model rendering using importance sampling
Embodiments are configured to render 3D models using an importance sampling method. First, embodiments obtain a 3D model including a plurality of density values corresponding to a plurality of locations in a 3D space, respectively. Embodiments then sample the color information from within a random subset of the plurality of locations using a probability distribution based on the plurality of density values. Embodiments have a higher probability to sample each location within the random subset of locations if the location has a higher density probability. Embodiments then an image depicting a view of the 3D model based on the sampling within the random subset of the plurality of locations.
1 . A method comprising:
obtaining a 3D model including a plurality of density values corresponding to a plurality of locations in a 3D space, respectively;
computing an opacity of a ray, W, based on the plurality of density values;
computing a radiance value by selecting a random number in the interval [0, W] based on computing the opacity of the ray, identifying a location on the ray corresponding to the random number based on the density values, and querying the 3D model at the location on the ray; and
rendering an image depicting a view of the 3D model based on the radiance value.
2 . The method of claim 1 , wherein rendering the image further comprises:
selecting a camera position for the image, wherein the image depicts a view of the 3D model from the camera position.
3 . The method of claim 2 , wherein rendering the image further comprises:
computing a ray through the 3D model from the camera position.
4 . The method of claim 1 , wherein rendering the further image comprises:
identifying color information for the location, wherein the image is rendered based on the color information.
5 . The method of claim 4 , wherein rendering the further image comprises:
combining the color information for the location to obtain a pixel color for a pixel of the image, wherein the image is rendered based on the pixel color.
6 . The method of claim 1 , wherein:
the opacity is computed without storing the plurality of density values in an array.
7 . The method of claim 1 , further comprising:
filtering the plurality of locations based on the plurality of density values to obtain a plurality of filtered locations, wherein the location is identified from the plurality of filtered locations.
8 . The method of claim 1 , further comprising:
performing a ray marching operation based on the location, wherein the image is rendered based on the ray marching operation.
9 . The method of claim 1 , wherein:
the 3D model comprises a Neural Radiance Field (NeRF) model.
10 . The method of claim 1 , wherein:
the plurality of density values are represented as a discretized density grid.
11 . The method of claim 1 , wherein:
the plurality of density values are represented as a plurality of factorized component tensors.
12 . A non-transitory computer-readable medium storing code, the code comprising instructions executable by a processor to:
obtain a 3D model including a plurality of density values corresponding to a plurality of locations in a 3D space, respectively;
computing an opacity of a ray, W, based on the plurality of density values;
compute a radiance value by selecting a random number in the interval [0, W] based on computing the opacity of the ray, identifying a location on the ray corresponding to the random number based on the density values, and querying the 3D model at the location on the ray; and
render an image depicting a view of the 3D model based on the radiance value.
13 . The non-transitory computer-readable medium of claim 12 , the code further comprising instructions executable by the processor to:
identify color information for the location, wherein the image is rendered based on the color information.
14 . The non-transitory computer-readable medium of claim 12 , the code further comprising instructions executable by the processor to:
filter the plurality of locations based on the plurality of density values to obtain a plurality of filtered locations, wherein the random subset is sampled from the plurality of filtered locations.
15 . An apparatus comprising:
at least one processor;
at least one memory storing instructions executable by the at least one processor;
a color sampling component configured to:
compute an opacity along a ray, W, passing through a 3D model based on a plurality of density values, wherein the plurality of density values correspond to a plurality of locations in a 3D space, respectively, and
compute a radiance value by selecting a random number in the interval [0, W] based on computing the opacity of the ray, identifying a location on the ray corresponding to the random number based on the density values, and querying the 3D model at the location on the ray;
and
a rendering component configured to render an image depicting a view of a 3D model based on the radiance value.
16 . The apparatus of claim 15 , wherein the color sampling component is further configured to:
filter the plurality of locations based on the plurality of density values to obtain a plurality of filtered locations, wherein the location is identified from the plurality of filtered locations.
17 . The apparatus of claim 15 , further comprising:
a density sampling component configured to obtain the plurality of density values.
18 . The apparatus of claim 17 , wherein:
the density sampling component is further configured to store the plurality of density values as a discretized density grid.
19 . The apparatus of claim 15 , wherein the rendering component is further configured to:
identify color information for the location, wherein the image is rendered based on the color information.
20 . The apparatus of claim 15 , wherein the rendering component is further configured to:
combining the color information for the location to obtain a pixel color for a pixel of the image, wherein the image is rendered based on the pixel color.