Real-time volumetric rendering
An image rendering system for rendering two-dimensional images in real-time. The image rendering system can receive an implicit representation model of a three-dimensional image. The image rendering system can construct, based on voxel coordinates, a three-dimensional image based on the implicit representation model. The image rendering system can rotate the three-dimensional image to an orientation in a computing space based on a user input. The image rendering system can generate a two-dimensional image based on the rotated three-dimensional image.
1 . A computer-implemented method, comprising:
modeling, by a computer system, a three-dimensional image by a continuous implicit voxel function using an implicit representation model;
training the implicit representation model using the three-dimensional image;
transmitting the implicit representation model to an image rendering system;
reconstructing, by the image rendering system, based on voxel coordinates, the three-dimensional image based on the implicit representation model;
rotating, by the image rendering system, based on a user input, the three-dimensional image to an orientation in a computing space; and
generating, by the image rendering system, a two-dimensional image based on the rotated three-dimensional image,
wherein the implicit representation model is based on a neural network encoded with a neural radiance field, and wherein the neural network comprises a multilayer perceptron,
wherein the neural network comprises at least fifteen neural layers, wherein each neural layer includes a rectified linear unit layer, and wherein the neural network includes at least two dense connections that concatenate an input of the neural network to at least two neural layers, and
wherein each neural layer has at least one of 256 neurons or 512 neurons, wherein the fifth neural layer and the eleventh neural layer of the neural network have 512 neurons, and wherein the at least two dense connections are concatenated to the fifth neural layer and the tenth neural layer.
2 . The computer-implemented method of claim 1 , wherein the rotating of the three-dimensional image to the orientation in the computing space comprises:
generating, based on the user input, one or more rotational matrices to rotate the three-dimensional image; and
rotating, based on the one or more rotational matrices, the three-dimensional image to the orientation in the computing space.
3 . The computer-implemented method of claim 2 , wherein the user input comprises user-defined angles.
4 . The computer-implemented method of claim 1 , wherein the two-dimensional image is generated based on a maximum intensity projection technique.
5 . The computer-implemented method of claim 4 , wherein the maximum intensity projection technique comprises:
determining projection lines associated with the pixels of the two-dimensional image;
determining, along a path of each projection line, a voxel in the three-dimensional image having a maximum intensity value; and
utilizing maximum intensity values of voxels along paths of the projection lines as intensity values for the pixels of the two-dimensional image.
6 . The computer-implemented method of claim 5 , wherein the projection lines are determined based on a viewpoint of the two-dimensional image, and wherein the projection lines intersect voxels of the three-dimensional image.
7 . The computer-implemented method of claim 1 , wherein a size of the implicit representation model is less than a size of the three-dimensional image.
8 . An image rendering system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the image rendering system to perform:
receiving an implicit representation model representing of a three-dimensional image;
constructing, based on voxel coordinates, the three-dimensional image based on the implicit representation model;
rotating, based on a user input, the three-dimensional image to an orientation in a computing space; and
generating a two-dimensional image based on the rotated three-dimensional image,
wherein the implicit representation model is based on a neural network encoded with a neural radiance field, and wherein the neural network comprises a multilayer perceptron,
wherein the neural network comprises at least fifteen neural layers, wherein each neural layer includes a rectified linear unit layer, and wherein the neural network includes at least two dense connections that concatenate an input of the neural network to at least two neural layers, and
wherein each neural layer has at least one of 256 neurons or 512 neurons, wherein the fifth neural layer and the eleventh neural layer of the neural network have 512 neurons, and wherein the at least two dense connections are concatenated to the fifth neural layer and the tenth neural layer.
9 . The image rendering system of claim 8 , wherein the rotating of the three-dimensional image to the orientation in the computing space comprises:
generating, based on the user input, one or more rotational matrices to rotate the three-dimensional image; and
rotating, based on the one or more rotational matrices, the three-dimensional image to the orientation in the computing space.
10 . The image rendering system of claim 9 , wherein the user input comprises user-defined angles.
11 . The image rendering system of claim 8 , wherein the two-dimensional image is generated in near real-time.
12 . The image rendering system of claim 8 , wherein the two-dimensional image is generated based on a maximum intensity projection technique.
13 . The image rendering system of claim 12 , wherein the maximum intensity projection technique comprises:
determining projection lines associated with the pixels of the two-dimensional image;
determining, along a path of each projection line, a voxel in the three-dimensional image having a maximum intensity value; and
utilizing maximum intensity values of voxels along paths of the projection lines as intensity values for the pixels of the two-dimensional image.
14 . The image rendering system of claim 13 , wherein the projection lines are determined based on a viewpoint of the two-dimensional image, and wherein the projection lines intersect voxels of the three-dimensional image.