Decomposing a deconvolution into multiple convolutions

1 . A method of deep learning, the method comprising:

generating an upsampled input tensor by adding new input activations into an input tensor of a deconvolution, wherein the input tensor comprising a plurality of input activations, wherein a dimension of the upsampled input tensor is larger than a dimension of the input tensor;

identifying a plurality of subtensors from the upsampled input tensor;

determining distributions of the plurality of input activations in the plurality of subtensors;

generating a plurality of reduced kernels from a kernel of the deconvolution based on the distributions, wherein the kernel comprises a plurality of weights, and each reduced kernel comprises a subset of the plurality of weights, and wherein generating the plurality of reduced kernels comprises:

classifying the plurality of subtensors into a plurality of groups, wherein a group comprises a subset of the plurality of subtensors, subtensors in the subset have an activation distribution pattern, and the activation distribution pattern indicates one or more positions of one or more input activations in each of the subtensors in the subset, and

generating a reduced kernel of the plurality of reduced kernels based on the activation distribution pattern; and

providing, to a neural network accelerator, the plurality of reduced kernels and the input tensor, wherein the neural network accelerators is to compute an output tensor of the deconvolution by performing a plurality of convolution operations using the plurality of reduced kernels and the input tensor, each convolution operation performed using a different one of the plurality of reduced kernels and the input tensor, wherein a dimension of the output tensor of the deconvolution is larger than the dimension of the input tensor but smaller than the dimension of the upsampled input tensor.

2 . The method of claim 1 , wherein generating the upsampled input tensor comprises:

determining a padding size for the upsampled input tensor based on a size of the kernel and a padding size of the deconvolution; and

adding at least some of the new input activations to edges of the input tensor based on the padding size.

3 . The method of claim 2 , wherein generating the upsampled input tensor further comprises:

inserting some other ones of the new input activations between adjacent input activations of the plurality of input activations based on a stride size of the deconvolution.

4 . The method of claim 1 , wherein a size of the plurality of subtensors is the same as a size of the kernel.

5 . The method of claim 1 , wherein generating the reduced kernel of the plurality of reduced kernels based on the activation distribution pattern comprises:

identifying one or more weights from the kernel based on the one or more positions of one or more input activations in each of the subtensors in the subset, wherein one or more positions of the one or more weights in the kernel are the same as the one or more positions of one or more input activations in each of the subtensors; and

including the one or more weights in the reduced kernel.

6 . The method of claim 1 , wherein a convolution operation of the plurality of convolution operations are performed on the reduced kernel and the input tensor to produce output activations in the output tensor, and positions of the output activations in the output tensor are determined based on positions of the subtensors in the subset in the upsampled input tensor.

7 . The method of claim 1 , wherein the plurality of reduced kernels comprises a first reduced kernel and a second reduced kernel, and the first reduced kernel and the second reduced kernel have different numbers of weights.

8 . The method of claim 1 , wherein the output tensor comprises a plurality of output activations, and a reduced kernel of the plurality of reduced kernels is to be used to produce one or more output activations of the plurality of output activations.

9 . The method of claim 1 , wherein:

the output tensor comprises a first output activation produced by using a first reduced kernel of the plurality of reduced kernels,

the output tensor further comprises second output activations produced by a second reduced kernel of the plurality of reduced kernels, and

the first output activation is arranged between the second output activations in the output tensor.

10 . One or more non-transitory computer-readable media storing instructions executable to perform operations for deep learning, the operations comprising:

generating an upsampled input tensor by adding new input activations into an input tensor of a deconvolution, wherein the input tensor comprising a plurality of input activations, wherein a dimension of the upsampled input tensor is larger than a dimension of the input tensor;

identifying a plurality of subtensors from the upsampled input tensor;

determining distributions of the plurality of input activations in the plurality of subtensors;

generating a plurality of reduced kernels from a kernel of the deconvolution based on the distributions, wherein the kernel comprises a plurality of weights, and each reduced kernel comprises a subset of the plurality of weights, and wherein generating the plurality of reduced kernels comprises:

classifying the plurality of subtensors into a plurality of groups, wherein a group comprises a subset of the plurality of subtensors, subtensors in the subset have an activation distribution pattern, and the activation distribution pattern indicates one or more positions of one or more input activations in each of the subtensors in the subset, and

generating a reduced kernel of the plurality of reduced kernels based on the activation distribution pattern; and

providing, to a neural network accelerator, the plurality of reduced kernels and the input tensor, wherein the neural network accelerators is to compute an output tensor of the deconvolution by performing a plurality of convolution operations using the plurality of reduced kernels and the input tensor, each convolution operation performed using a different one of the plurality of reduced kernels and the input tensor, wherein a dimension of the output tensor of the deconvolution is larger than the dimension of the input tensor but smaller than the dimension of the upsampled input tensor.

11 . The one or more non-transitory computer-readable media of claim 10 , wherein generating the upsampled input tensor comprises:

determining a padding size for the upsampled input tensor based on a size of the kernel and a padding size of the deconvolution; and

adding at least some of the new input activations to edges of the input tensor based on the padding size.

12 . The one or more non-transitory computer-readable media of claim 11 , wherein generating the upsampled input tensor further comprises:

inserting some other ones of the new input activations between adjacent input activations of the plurality of input activations based on a stride size of the deconvolution.

13 . The one or more non-transitory computer-readable media of claim 10 , wherein a size of the plurality of subtensors is the same as a size of the kernel.

14 . The one or more non-transitory computer-readable media of claim 10 , wherein generating the reduced kernel of the plurality of reduced kernels based on the activation distribution pattern comprises:

identifying one or more weights from the kernel based on the one or more positions of one or more input activations in each of the subtensors in the subset, wherein one or more positions of the one or more weights in the kernel are the same as the one or more positions of one or more input activations in each of the subtensors; and

including the one or more weights in the reduced kernel.

15 . The one or more non-transitory computer-readable media of claim 10 , wherein a convolution operation of the plurality of convolution operations are performed on the reduced kernel and the input tensor to produce output activations in the output tensor, and positions of the output activations in the output tensor are determined based on positions of the subtensors in the subset in the upsampled input tensor.

16 . The one or more non-transitory computer-readable media of claim 10 , wherein the plurality of reduced kernels comprises a first reduced kernel and a second reduced kernel, and the first reduced kernel and the second reduced kernel have different numbers of weights.

17 . The one or more non-transitory computer-readable media of claim 10 , wherein the output tensor comprises a plurality of output activations, and a reduced kernel of the plurality of reduced kernels is to be used to produce one or more output activations of the plurality of output activations.

18 . The one or more non-transitory computer-readable media of claim 10 , wherein:

the output tensor comprises a first output activation produced by using a first reduced kernel of the plurality of reduced kernels,

the output tensor further comprises second output activations produced by a second reduced kernel of the plurality of reduced kernels, and

the first output activation is arranged between the second output activations in the output tensor.

19 . An apparatus for training a target neural network, the apparatus comprising:

a computer processor for executing computer program instructions; and

a non-transitory computer-readable memory storing computer program instructions executable by the computer processor to perform operations comprising:

generate an upsampled input tensor by adding new input activations into an input tensor of a deconvolution, wherein the input tensor comprising a plurality of input activations, wherein a dimension of the upsampled input tensor is larger than a dimension of the input tensor,

identify a plurality of subtensors from the upsampled input tensor,

determining distributions of the plurality of input activations in the plurality of subtensors,

generating a plurality of reduced kernels from a kernel of the deconvolution based on the distributions, wherein the kernel comprises a plurality of weights, and each reduced kernel comprises a subset of the plurality of weights, and wherein generating the plurality of reduced kernels comprises:

classifying the plurality of subtensors into a plurality of groups, wherein a group comprises a subset of the plurality of subtensors, subtensors in the subset have an activation distribution pattern, and the activation distribution pattern indicates one or more positions of one or more input activations in each of the subtensors in the subset, and

generating a reduced kernel of the plurality of reduced kernels based on the activation distribution pattern, and

providing, to a neural network accelerator, the plurality of reduced kernels and the input tensor, wherein the neural network accelerators is to compute an output tensor of the deconvolution by performing a plurality of convolution operations using the plurality of reduced kernels and the input tensor, each convolution operation performed using a different one of the plurality of reduced kernels and the input tensor, wherein a dimension of the output tensor of the deconvolution is larger than the dimension of the input tensor but smaller than the dimension of the upsampled input tensor.

20 . The apparatus of claim 19 , wherein generating the upsampled input tensor comprises:

determining a padding size for the upsampled input tensor based on a size of the kernel and a padding size of the deconvolution; and

adding at least some of the new input activations to edges of the input tensor based on the padding size.

21 . The apparatus of claim 19 , wherein a size of the plurality of subtensors is the same as a size of the kernel.

22 . The apparatus of claim 19 , wherein:

the output tensor comprises a first output activation produced by using a first reduced kernel of the plurality of reduced kernels,

the output tensor further comprises second output activations produced by a second reduced kernel of the plurality of reduced kernels, and

the first output activation is arranged between the second output activations in the output tensor.