Convolutional neural network hardware configuration
A method of configuring a hardware implementation of a Convolutional Neural Network (CNN), the method comprising: determining, for each of a plurality of layers of the CNN, a first number format for representing weight values in the layer based upon a distribution of weight values for the layer, the first number format comprising a first integer of a first predetermined bit-length and a first exponent value that is fixed for the layer; determining, for each of a plurality of layers of the CNN, a second number format for representing data values in the layer based upon a distribution of expected data values for the layer, the second number format comprising a second integer of a second predetermined bit-length and a second exponent value that is fixed for the layer; and storing the determined number formats for use in configuring the hardware implementation of a CNN.
1. A computer-implemented method of configuring a hardware implementation of a Convolutional Neural Network (CNN), the method comprising:
receiving, in a computer system, parameters of a CNN having a plurality of layers, said parameters including weight values and expected data values;
determining, in the computer system, for each of a plurality of layers of the CNN, a first number format for representing weight values in the layer based upon a distribution of weight values for the layer, the first number format comprising a first integer of a first predetermined bit-length and a first exponent value that is fixed for the layer;
determining, in the computer system, for each of a plurality of layers of the CNN, a second number format for representing data values in the layer based upon a distribution of expected data values for the layer, the second number format comprising a second integer of a second predetermined bit-length and a second exponent value that is fixed for the layer;
storing in a computer readable storage medium the determined number formats for use in configuring the hardware implementation of a CNN; and
tuning the weight values, the tuning comprising:
selecting, for each of a plurality of layers, a subset of the weight values in that layer, wherein the subset is less than all of the weight values in that layer,
quantising, for each of a plurality of layers, the selected subset of weight values in that layer in accordance with the first number format for that layer to generate modified weight values in that layer, wherein the modified weight values in that layer are less than all of the weight values in that layer, and
training the CNN using the modified weight values to generate tuned weight values.
2. The method of claim 1 , wherein the first and second predetermined bit-lengths are each fixed across a plurality of layers of the CNN.
3. The method of claim 1 , further comprising converting the weight values for each of the plurality of layers of the CNN into the first number format and converting input data to the CNN into the second number format, wherein the converting comprises performing quantisation based upon the first and second number formats.
4. The method of claim 3 , wherein the quantisation is performed based upon nearest-neighbour rounding.
5. The method of claim 3 , further comprising storing the converted weight values for each of the plurality of layers of the CNN and data values to the CNN in a memory accessible by the hardware implementation of the CNN or in one or more buffers in the hardware implementation of the CNN.
6. The method of claim 1 , further comprising storing the weight values for each of the plurality of layers of the CNN, the storing comprising:
storing, for each weight value, a value representing the first integer of that weight value; and
storing, for each layer, a value representing the first exponent value for that layer.
7. The method of claim 1 , further comprising storing the data values for each of the plurality of layers of the CNN, the storing comprising:
storing, for each data value, a value representing the second integer of that data value; and
storing, for each layer, a value representing the second exponent value for that layer.
8. The method of claim 1 , wherein the weight values in each layer of the CNN are generated by training the CNN using a training data set and wherein the generated weight values are in a floating-point format.
9. The method of claim 8 , wherein the distribution of expected data values is derived from the training data set.
10. The method of claim 1 , further comprising determining the first exponent value for each layer by:
identifying a bit index that represents a range of values based upon the distribution of weight values for that layer;
right-shifting the position of the bit index from the identified bit index a number of bits based upon the first predetermined bit-length to identify a first shifted bit index; and
allocating the value of the first shifted bit index as the first exponent value.
11. The method of claim 1 , further comprising determining the second exponent value for each layer by:
identifying a bit index that represents a range of values based upon the distribution of expected data values for that layer;
right-shifting the position of the bit index from the identified bit index a number of bits based upon the second predetermined bit-length to identify a second shifted bit index; and
allocating the value of the second shifted bit index as the second exponent value.
12. A computer system for configuring a hardware implementation of a Convolutional Neural Network (CNN), the computer system comprising:
hardware logic configured to determine, for each of a plurality of layers of the CNN, a first number format for representing weight values in the layer based upon a distribution of weight values for the layer, the first number format comprising a first integer of a first predetermined bit-length and a first exponent value that is fixed for the layer;
hardware logic configured to determine, for each of a plurality of layers of the CNN, a second number format for representing data values in the layer based upon a distribution of expected data values for the layer, the second number format comprising a second integer of a second predetermined bit-length and a second exponent value that is fixed for the layer;
the computer system further configured to store the determined number formats in a computer readable storage medium for use in configuring the hardware implementation of a CNN; and
tune the weight values, wherein tuning the weight values comprises:
selecting, for each of a plurality of layers, a subset of the weight values in that layer, wherein the subset is less than all of the weight values in that layer,
quantising, for each of a plurality of layers, the selected subset of weight values in that layer in accordance with the first number format for that layer to generate modified weight values in that layer, wherein the modified weight values in that layer are less than all of the weight values in that layer, and
training the CNN using the modified weight values to generate tuned weight values.
13. The computer system of claim 12 , wherein the first and second predetermined bit-lengths are each fixed across a plurality of layers of the CNN.
14. The computer system of claim 12 , wherein the computer system is configured to convert the weight values for each of the plurality of layers of the CNN into the first number format and convert input data to the CNN into the second number format, wherein the conversion comprises performing quantisation based upon the first and second number formats.
15. The computer system of claim 14 , wherein the quantisation is performed based upon nearest-neighbour rounding.
16. The computer system of claim 14 , wherein the computer system is configured to store the converted weight values for each of the plurality of layers of the CNN and data values to the CNN in a memory accessible by the hardware implementation of the CNN or in one or more buffers in the hardware implementation of the CNN.
17. The computer system of claim 12 , wherein the computer system is configured to store the weight values for each of the plurality of layers of the CNN, wherein storing the weight values comprises:
storing, for each weight value, a value representing the first integer of that weight value; and
storing, for each layer, a value representing the first exponent value for that layer.
18. The computer system of claim 12 , wherein the computer system is configured to store the data values for each of the plurality of layers of the CNN, wherein storing the data values comprises:
storing, for each data value, a value representing the second integer of that data value; and
storing, for each layer, a value representing the second exponent value for that layer.
19. The computer system of claim 12 , wherein the weight values in each layer of the CNN are generated by training the CNN using a training data set and wherein the generated weight values are in a floating-point format.
20. A non-transitory computer readable storage medium having stored thereon computer executable instructions that when executed cause at least one processor to configure a hardware implementation of a Convolutional Neural Network (CNN), by:
determining, for each of a plurality of layers of the CNN, a first number format for representing weight values in the layer based upon a distribution of weight values for the layer, the first number format comprising a first integer of a first predetermined bit-length and a first exponent value that is fixed for the layer;
determining, for each of a plurality of layers of the CNN, a second number format for representing data values in the layer based upon a distribution of expected data values for the layer, the second number format comprising a second integer of a second predetermined bit-length and a second exponent value that is fixed for the layer;
storing the determined number formats in a computer readable storage medium for use in configuring the hardware implementation of a CNN; and
tuning the weight values, the tuning comprising:
selecting, for each of a plurality of layers, a subset of the weight values in that layer, wherein the subset is less than all of the weight values in that layer,
quantising, for each of a plurality of layers, the selected weight values in that layer in accordance with the first number format for that layer to generate modified weight values in that layer, wherein the selected weight values in that layer are less than all of the weight values in that layer, and
training the CNN using the modified weight values to generate tuned weight values.