Multiresolution hash encoding for neural networks
Neural network performance is improved in terms of training speed and/or accuracy by encoding (mapping) inputs to the neural network into a higher dimensional space via a hash function. The input comprises coordinates used to identify a point within a d-dimensional space (e.g., 3D space). The point is quantized and a set of vertex coordinates corresponding to the point are input to a hash function. For example, for d=3, space may be partitioned into axis-aligned voxels of identical size and vertex coordinates of a voxel containing the point are input to the hash function to produce a set of encoded coordinates. The set of encoded coordinates is used to lookup D-dimensional feature vectors in a table of size T that have been learned. The learned feature vectors are filtered (e.g., linearly interpolated, etc.) based on the coordinates of the point to compute a feature vector corresponding to the point.
1 . A computer-implemented method for mapping input coordinates to a feature vector for input to a neural network model, comprising:
processing the input coordinates according to a hash function to produce encoded coordinates;
using the encoded coordinates directly to read a feature vector stored at an entry of a hash table in on-chip shared memory, wherein during training of the neural network model the feature vector is learned by:
backpropagating a loss gradient with respect to the feature vector through the neural network model; and
updating the feature vector stored at the entry based on the loss gradient; and
inputting the feature vector to the neural network model instead of the input coordinates to generate a prediction of a function at the coordinates.
2 . The computer-implemented method of claim 1 , further comprising, prior to processing the coordinates, quantizing the input coordinates to a set of vertices.
3 . The computer-implemented method of claim 2 , further comprising filtering the feature vector based on the input coordinates and the set of vertices before inputting the feature vector to the neural network model.
4 . The computer-implemented method of claim 1 , further comprising generating additional coordinates for at least one additional resolution of the encoded coordinates.
5 . The computer-implemented method of claim 4 , wherein additional feature vectors for the at least one additional resolution are learned during training of the neural network model.
6 . The computer-implemented method of claim 1 , wherein weights applied to the feature vector by the neural network model are learned during training of the neural network model.
7 . The computer-implemented method of claim 1 , wherein at least one additional component corresponding to the input coordinates is included in the encoded coordinates.
8 . The computer-implemented method of claim 1 , wherein the neural network model is trained for a task of predicting signed distance functions, importance sampling, predicting light and radiance fields, or approximating a mathematical function.
9 . The computer-implemented method of claim 1 , wherein the neural network model weights and a plurality of feature vectors including the feature vector are trained continuously over time.
10 . The computer-implemented method of claim 1 , wherein at least one of the steps of processing, using, or inputting is performed on a server or in a data center to generate an image, and the image is streamed to a user device.
11 . The computer-implemented method of claim 1 , wherein at least one of the steps of processing, using, or inputting is performed within a cloud computing environment.
12 . The computer-implemented method of claim 1 , wherein at least one of the steps of processing, using, or inputting is performed for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle.
13 . The computer-implemented method of claim 1 , wherein at least one of the steps of processing, using, or inputting is performed on a virtual machine comprising a portion of a graphics processing unit.
14 . A system, comprising:
a memory that stores input coordinates; and
a processor that is connected to the memory, wherein the processor is configured to map the coordinates to a feature vector for input to a neural network model by:
processing the input coordinates according to a hash function to produce encoded coordinates;
using the encoded coordinates directly to read a feature vector stored at an entry of a hash table in on-chip shared memory, wherein during training of the neural network model the feature vector is learned by:
backpropagating a loss gradient with respect to the feature vector through the neural network model; and
updating the feature vector stored at the entry based on the loss gradient; and
inputting the feature vector to the neural network model instead of the input coordinates to generate a prediction of a function at the coordinates.
15 . The system of claim 14 , further comprising, prior to processing the coordinates, quantizing the input coordinates to a set of vertices of a multiresolution grid.
16 . The system of claim 14 , wherein the feature vector is filtered based on the input coordinates and the set of vertices before the feature vector is input to the neural network model.
17 . The system of claim 14 , further comprising generating additional coordinates for at least one additional resolution of the encoded coordinates.
18 . The system of claim 14 , wherein at least one additional component corresponding to the input coordinates is included in the encoded coordinates.
19 . A non-transitory computer-readable media storing computer instructions for mapping input coordinates to a feature vector for input to a neural network model that, when executed by one or more processors, cause the one or more processors to perform the steps of:
processing the input coordinates according to a hash function to produce encoded coordinates;
using the encoded coordinates directly to read a feature vector stored at an entry of a hash table in on-chip shared memory, wherein during training of the neural network model the feature vector is learned by:
backpropagating a loss gradient with respect to the feature vector through the neural network model; and
updating the feature vector stored at the entry based on the loss gradient; and
inputting the feature vector to the neural network model instead of the input coordinates to generate a prediction of a function at the coordinates.
20 . The non-transitory computer-readable media of claim 19 , further comprising generating additional coordinates for at least one additional resolution of the encoded coordinates.