Fused convolutions for fast deep neural network
Fused channel and/or fused filter convolutions for fast deep neural network execution are provided. In one aspect, a system includes: a processor, connected to a memory, configured to: implement an approximated datapath in a deep neural network having a sequence of adders and multipliers for adding up operands to provide accumulated sums for two or more groups of neurons in the deep neural network, and multiplying the accumulated sums to obtain a product; and make an inference using the deep neural network based on the product from the approximated datapath. A method for approximation in a deep neural network is also provided.
1 . A system comprising:
a processor;
a computer readable storage media; and
program instructions stored on the computer readable storage media for execution by the processor to perform operations comprising:
inputting an input into a deep neural network wherein the deep neural network comprises an intermediate processing element that receives multiple intermediate inputs, respectively, from multiple upstream processing elements disposed in the deep neural network upstream from the intermediate processing element within the deep neural network, wherein the multiple intermediate inputs are based on the input into the deep neural network, and wherein the intermediate processing element processes the multiple intermediate inputs through a sequence comprising:
firstly, adding up a first group of input activations from two or more of the multiple intermediate inputs to provide a first accumulated sum,
adding up a second group of weight tensors for the two or more of the multiple intermediate inputs to provide a second accumulated sum, wherein the weight tensors correspond to the input activations,
subsequently, multiplying the first accumulated sum against the second accumulated sum to obtain a product; and
forwarding an intermediate processing element output to one or more downstream processing elements of the deep neural network, wherein the intermediate processing element output is based on the product; and
receiving, from an output layer of the deep neural network, an inference for the input, wherein the deep neural network determines the inference based on the intermediate processing element output.
2 . A method comprising:
inputting an input into a deep neural network, wherein the deep neural network comprises an intermediate processing element that receives multiple intermediate inputs, respectively, from multiple upstream processing elements disposed in the deep neural network upstream from the intermediate processing element within the deep neural network, wherein the multiple intermediate inputs are based on the input into the deep neural network, and wherein the intermediate processing element processes the multiple intermediate inputs through a sequence comprising:
firstly adding up a first group of input activations from two or more of the multiple intermediate inputs to provide a first accumulated sum, adding up a second group of weight tensors for the two or more of the multiple intermediate inputs to provide a second accumulated sum, wherein the weight tensors correspond to the input activations,
subsequently, multiplying the first accumulated sum against the second accumulated sum to obtain a product, and
forwarding an intermediate processing element output to one or more downstream processing elements of the deep neural network, wherein the intermediate processing element output is based on the product; and
receiving, from an output layer of the deep neural network, an inference for the input, wherein the deep neural network determines the inference based on the intermediate processing element output.
3 . The method of claim 2 , wherein:
the adding up of the first group of input activations comprises adding up more than two input activations to produce the first accumulated sum, and
the adding up of the second group of weight tensors comprises adding up more than two corresponding weight tensors that correspond, respectively, to the more than two input activations to produce the second accumulated sum.
4 . The method of claim 3 , wherein the weight tensors are applied statically during the inference and are determined during a training stage of the deep neural network.
5 . The method of claim 2 , further comprising:
mapping different input channels, different filters, or a combination thereof to the input activations so that the different input channels, the different filters, or the combination thereof are fused.
6 . The method of claim 2 , wherein the intermediate processing element, the multiple upstream processing elements, and the one or more downstream processing elements are respective neurons of the deep neural network.
7 . The method of claim 2 , wherein the intermediate processing element, the multiple upstream processing elements, and the one or more downstream processing elements are respective analog crossbars of the deep neural network, and the deep neural network is a hardware accelerator.
8 . The method of claim 2 , wherein the intermediate processing element, the multiple upstream processing elements, and the one or more downstream processing elements are respective resistive processing units whose resistance is adjusted according to voltage applied.
9 . The method of claim 2 , wherein the product is input into at least one of a normalization function and an activation function whose functional output constitutes the intermediate processing element output.
10 . The method of claim 2 , wherein the intermediate processing element is within a first intermediate layer of the deep neural network and the one or more downstream processing elements are in a downstream layer of the deep neural network, the downstream layer being disposed downstream from the intermediate layer within the deep neural network.
11 . The method of claim 2 , wherein:
the first group of input activations comprises filter dimension input activations for the two or more multiple upstream processing elements in the deep neural network,
the second group of weight tensors comprises corresponding filter kernels, respectively, to the input activations, and
the multiple upstream processing elements are adjacent filters that the sequence fuses together.
12 . The method of claim 2 , wherein the input is an image and the sequence fuses adjacent pixels of an intermediate image produced within the deep neural network.
13 . The method of claim 2 , wherein:
the first group of input activations comprises activation pixels,
the second group of weight tensors comprises filter kernels, and
the filter kernels vary along both channel and filter dimensions.
14 . A computer program product comprising:
a computer readable storage medium; and
program instructions stored on the computer readable storage medium for performing operations comprising:
inputting an input into a deep neural network, wherein the deep neural network comprises an intermediate processing element that receives multiple intermediate inputs, respectively, from multiple upstream processing elements disposed in the deep neural network upstream from the intermediate processing element within the deep neural network, wherein the multiple intermediate inputs are based on the input into the deep neural network, and wherein the intermediate processing element processes the multiple intermediate inputs through a sequence comprising:
firstly, adding up a first group of input activations from two or more of the multiple intermediate inputs to provide a first accumulated sum,
adding up a second group of weight tensors for the two or more of the multiple intermediate inputs to provide a respective second accumulated sum, wherein the weight tensors correspond to the input activations,
multiplying the first accumulated sum against the second accumulated sum to obtain a product, and
forwarding an intermediate processing element output to one or more downstream processing elements of the deep neural network, wherein the intermediate processing element output is based on the product; and
receiving, from an output layer of the deep neural network, an inference for the input, wherein the deep neural network determines the inference based on the intermediate processing element output.
15 . The computer program product of claim 14 , wherein:
the adding up of the first group of input activations comprises adding up more than two input activations to produce the first accumulated sum, and
the adding up of the second group of weight tensors comprises adding up more than two corresponding weight tensors that correspond, respectively, to the more than two input activations to produce the second accumulated sum.
16 . The computer program product of claim 14 , wherein the operations further comprise:
mapping different input channels, different filters, or a combination thereof to the input activations so that the different input channels, the different filters, or the combination thereof are fused.