Neural networks for embedded devices
A neural network architecture is used that reduces the processing load of implementing the neural network. This network architecture may thus be used for reduced-bit processing devices. The architecture may limit the number of bits used for processing and reduce processing to prevent data overflow at individual calculations of the neural network. To implement this architecture, the number of bits used to represent inputs at levels of the network and the related filter masks may also be modified to ensure the number of bits of the output does not overflow the resulting capacity of the reduced-bit processor. To additionally reduce the load for such a network, the network may implement a “starconv” structure that permits the incorporation of nearby nodes in a layer to balance processing requirements and permit the network to learn from context of other nodes.
1. A method of generating a neural network structure including one or more input layers each associated with one or more filters, the method comprising:
determining, for an architecture of a device, a bit length of a set of registers of the device used to perform arithmetic operations;
determining a first integer representation for the one or more input layers and a second integer representation for the one or more filters, the first integer representation associated with a first range of integer values and the second integer representation associated with a second range of integer values;
generating dimensionalities of the one or more input layers and the one or more filters, the dimensionalities determined such that an output value generated by combining elements of an input layer as maximum values of the first integer representation with elements of a corresponding filter as maximum values of the second integer representation does not overflow the bit length of the registers; and
generating the neural network structure with the determined dimensionalities, wherein for each individual layer which forms the neural network structure, activations are quantized using activation parameters for the individual layer and the weights are quantized using layer parameters for the individual layer, wherein the activations and the activation parameters are provided as input to the individual layer, and wherein quantized output associated with the individual layer is dequantized using (1) the input activation parameters and (2) the layer parameters.
2. The method of claim 1 , wherein generating the dimensionalities comprises generating the one or more filters for a corresponding input layer as star-shaped filters.
3. The method of claim 1 , further comprising:
receiving a set of input values corresponding to the elements of an input layer in the one or more input layers which form activations for the input layer, and a set of weights corresponding to the elements of a filter in the one or more filters with the generated dimensionalities;
quantizing the set of input values by assigning each input value to a corresponding integer value in the first integer representation;
quantizing the set of weights by assigning each weight to a corresponding integer value in the second integer representation; and
combining the set of input values and the set of weights to generate a quantized output.
4. The method of claim 3 , wherein the neural network structure includes a shuffle layer placed after the corresponding input layer, the method further comprising:
receiving another set of input values at the shuffle layer, wherein the another set of input values are arranged with respect to a plurality of channels; and
interleaving ordering of the plurality of channels at the shuffle layer.
5. The method of claim 3 , wherein quantizing the set of input values comprises:
obtaining a dataset including a plurality of data instances;
propagating the plurality of data instances through the neural network structure to obtain input values at the input layer;
identifying a lower bound value and an upper bound value from the input values obtained at the input layer; and
dividing a range between the lower bound value and the upper bound value into a plurality of bins each assigned to a corresponding integer value in the first integer representation.
6. The method of claim 3 , wherein quantizing the set of weights comprises: identifying a lower bound value and an upper bound value from the set of weights; and
dividing a range between the lower bound value and the upper bound value into a plurality of bins each assigned to a corresponding integer value in the second integer representation.
7. The method of claim 1 , wherein the bit length of the set of registers are 8 bits, and the arithmetic operations are performed using 8-bit arithmetic.
8. A non-transitory computer-readable medium containing instructions for execution on a processor, the instructions comprising:
determining, for an architecture of a device, a bit length of a set of registers of the device used to perform arithmetic operations;
determining a first integer representation for the one or more input layers and a second integer representation for the one or more filters, the first integer representation associated with a first range of integer values and the second integer representation associated with a second range of integer values;
generating dimensionalities of the one or more input layers and the one or more filters, the dimensionalities determined such that an output value generated by combining elements of an input layer as maximum values of the first integer representation with elements of a corresponding filter as maximum values of the second integer representation does not overflow the bit length of the registers; and
generating the neural network structure with the determined dimensionalities, wherein for each individual layer which forms the neural network structure, activations are quantized using activation parameters for the individual layer and the weights are quantized using layer parameters for the individual layer, wherein the activations and the activation parameters are provided as input to the individual layer, and wherein quantized output associated with the individual layer is dequantized using (1) the input activation parameters and (2) the layer parameters.
9. The non-transitory computer-readable medium of claim 8 , wherein generating the dimensionalities comprises generating the one or more filters for a corresponding input layer as star-shaped filters.
10. The non-transitory computer-readable medium of claim 8 , the instructions further comprising:
receiving a set of input values corresponding to the elements of an input layer in the one or more input layers which form activations for the input layer, and a set of weights corresponding to the elements of a filter in the one or more filters with the generated dimensionalities;
quantizing the set of input values by assigning each input value to a corresponding integer value in the first integer representation;
quantizing the set of weights by assigning each weight to a corresponding integer value in the second integer representation; and
combining the set of input values and the set of weights to generate a quantized output.
11. The non-transitory computer-readable medium of claim 10 , wherein the neural network structure includes a shuffle layer placed after the corresponding input layer, the instructions further comprising:
receiving another set of input values at the shuffle layer, wherein the another set of input values are arranged with respect to a plurality of channels; and
interleaving ordering of the plurality of channels at the shuffle layer.
12. The non-transitory computer-readable medium of claim 10 , wherein quantizing the set of input values comprises:
obtaining a dataset including a plurality of data instances;
propagating the plurality of data instances through the neural network structure to obtain input values at the input layer;
identifying a lower bound value and an upper bound value from the input values obtained at the input layer; and
dividing a range between the lower bound value and the upper bound value into a plurality of bins each assigned to a corresponding integer value in the first integer representation.
13. The non-transitory computer-readable medium of claim 10 , wherein quantizing the set of weights comprises:
identifying a lower bound value and an upper bound value from the set of weights; and
dividing a range between the lower bound value and the upper bound value into a plurality of bins each assigned to a corresponding integer value in the second integer representation.
14. The non-transitory computer-readable medium of claim 8 , wherein the bit length of the set of registers are 8 bits, and the arithmetic operations are performed using 8-bit arithmetic.
15. A system comprising:
a processor configured to execute instructions;
a computer-readable medium containing instructions for execution on the processor, the instructions causing the processor to perform steps of:
determining, for an architecture of a device, a bit length of a set of registers of the device used to perform arithmetic operations;
determining a first integer representation for the one or more input layers and a second integer representation for the one or more filters, the first integer representation associated with a first range of integer values and the second integer representation associated with a second range of integer values;
generating dimensionalities of the one or more input layers and the one or more filters, the dimensionalities determined such that an output value generated by combining elements of an input layer as maximum values of the first integer representation with elements of a corresponding filter as maximum values of the second integer representation does not overflow the bit length of the registers; and
generating the neural network structure with the determined dimensionalities, wherein for each individual layer which forms the neural network structure, activations are quantized using activation parameters for the individual layer and the weights are quantized using layer parameters for the individual layer, wherein the activations and the activation parameters are provided as input to the individual layer, and wherein quantized output associated with the individual layer is dequantized using (1) the input activation parameters and (2) the layer parameters.
16. The system of claim 15 , wherein generating the dimensionalities comprises generating the one or more filters for a corresponding input layer as star-shaped filters.
17. The system of claim 15 , the instructions further comprising:
receiving a set of input values corresponding to the elements of an input layer in the one or more input layers which form activations for the input layer, and a set of weights corresponding to the elements of a filter in the one or more filters with the generated dimensionalities;
quantizing the set of input values by assigning each input value to a corresponding integer value in the first integer representation;
quantizing the set of weights by assigning each weight to a corresponding integer value in the second integer representation; and
combining the set of input values and the set of weights to generate a quantized output.
18. The system of claim 17 , wherein the neural network structure includes a shuffle layer placed after the corresponding input layer, the instructions further comprising:
receiving another set of input values at the shuffle layer, wherein the another set of input values are arranged with respect to a plurality of channels; and
interleaving ordering of the plurality of channels at the shuffle layer.
19. The system of claim 17 , wherein quantizing the set of input values comprises:
obtaining a dataset including a plurality of data instances;
propagating the plurality of data instances through the neural network structure to obtain input values at the input layer;
identifying a lower bound value and an upper bound value from the input values obtained at the input layer; and
dividing a range between the lower bound value and the upper bound value into a plurality of bins each assigned to a corresponding integer value in the first integer representation.
20. The system of claim 15 , wherein the bit length of the set of registers are 8 bits, and the arithmetic operations are performed using 8-bit arithmetic.