Systems and methods for sparse convolution of unstructured data
Systems and methods are disclosed for processing sparse tensors using a trained neural network model. An input sparse tensor may represent a sparse input point cloud. The input sparse tensor is processed using an encoder stage having a series of one or more encoder blocks, wherein each encoder block includes a sparse convolution layer, a sparse intra-channel attention module, a sparse inter-channel attention module, and a sparse residual tower module. Output from the encoder stage is processed using a decoder stage having a series of one or more decoder blocks, wherein each decoder block includes a sparse transpose convolution layer, a sparse inter-channel attention module, and a sparse residual tower module. The output of the decoder stage is an output sparse tensor representing a sparse labeled output point cloud.
1 . A system comprising:
a processor device; and
a memory storing instructions that configure the processor device to compute a block of a trained neural network model, the block including a sparse inter-channel attention module, wherein computing the block comprises:
obtaining, as input to the block of the trained neural network model, an input sparse tensor in which each data point in the input sparse tensor has corresponding coordinate data and a feature vector, and wherein the corresponding coordinate data of each data point represent a coordinate location of the data point in a multi-dimensional coordinate space having three or more dimensions;
applying the sparse inter-channel attention module to the input sparse tensor by:
processing the input sparse tensor using a sparse global pooling squeeze layer to obtain a set of global attention weights;
processing the set of global attention weights using a sparse linear excitation layer to obtain a set of channel-wise attention weights; and
applying the set of channel-wise attention weights to the input sparse tensor to obtain a scaled feature representation having inter-channel attention applied; dampening the inter-channel attention including:
applying, to the scaled feature representation, a damping factor to obtain a regularized scaled feature representation; and
summing the regularized scaled feature representation with the input sparse tensor to obtain a summation output, the input sparse tensor having been scaled by a scaling factor complementary to the damping factor; and
outputting the summation output as an output sparse tensor of the block of the trained neural network model, each data point in the output sparse tensor corresponding to a respective data point in the input sparse tensor with corresponding coordinate data and a weighted feature vector.
2 . The system of claim 1 , wherein the block of the trained neural network model is an encoder block of an encoder stage of the trained neural network model.
3 . The system of claim 1 , wherein the block of the trained neural network model is a decoder block of a decoder stage of the trained neural network model.
4 . A method for computing a block of a trained neural network model, the block including a sparse inter-channel attention module, wherein computing the block comprises:
obtaining, as input to the block of the trained neural network model, an input sparse tensor in which each data point in the input sparse tensor has corresponding coordinate data and a feature vector, and wherein the corresponding coordinate data of each data point represent a coordinate location of the data point in a multi-dimensional coordinate space having three or more dimensions;
applying the sparse inter-channel attention module to the input sparse tensor by:
processing the input sparse tensor using a sparse global pooling squeeze layer to obtain a set of global attention weights;
processing the set of global attention weights using a sparse linear excitation layer to obtain a set of channel-wise attention weights; and
applying the set of channel-wise attention weights to the input sparse tensor to obtain a scaled feature representation having inter-channel attention applied; dampening the inter-channel attention including:
applying, to the scaled feature representation, a damping factor to obtain a regularized scaled feature representation; and
summing the regularized scaled feature representation with the input sparse tensor to obtain a summation output, the input sparse tensor having been scaled by a scaling factor complementary to the damping factor; and
outputting the summation output as an output sparse tensor of the block of the trained neural network model, each data point in the output sparse tensor corresponding to a respective data point in the input sparse tensor with corresponding coordinate data and a weighted feature vector.
5 . The method of claim 4 , wherein the block of the trained neural network model is an encoder block of an encoder stage of the trained neural network model.
6 . The method of claim 4 , wherein the block of the trained neural network model is a decoder block of a decoder stage of the trained neural network model.