Add-mulitply-add convolution computation for a convolutional neural network
An inference device comprises a weight storage part that stores weights, an input data storage part that stores input data, and a PE (Processing Element) that executes convolution computation in convolutional neural network using the weights and input data. The PE adds up weight elements to be multiplied with elements of the input data for each of variable values of the elements of the input data. The PE multiplies each of the variable values of the elements of the input data with each cumulative sum value of weights corresponding to the variable values of the elements of the input data. The PE adds up a plurality of multiplication results obtained by the multiplications.
1. An inference device, comprising:
a weight storage that stores weights,
an input data storage that stores input data, and
a PE (Processing Element) that executes convolution computation in a convolutional neural network using the weights and the input data, wherein
the PE comprises one or more processors that are configured to implement:
a selector into which the elements of the input data and the weight elements are input,
a plurality of accumulation processors each calculating and storing cumulative sum of the weights for each of the variable values of the input data elements,
a plurality of multipliers respectively associated with the plurality of accumulation processors and assigned to a value to be multiplied with the cumulative sum value of the weights, and
an add-up processor into which the multiplication results by the plurality of multipliers are input and that adds up the multiplication results,
wherein the selector determines an accumulation processor to be an output destination of the input weight elements among the plurality of accumulation processors,
wherein the accumulation processor adds up weight elements to be multiplied with elements of the input data for each of variable values of the elements of the input data,
wherein the multiplier multiplies each of the variable values of the elements of the input data with each cumulative sum value of weights corresponding to the variable values of the elements of the input data, and
wherein the add-up processor adds up a plurality of multiplication results obtained by the multiplication.
2. The inference device according to claim 1 , wherein
the input data storage stores quantized input data,
the one or more processors of the PE are further configured to implement a separation processor that factorizes the input data elements and separates the factorized input data to predetermined first factor group and second factor group, and a shifting processor which shifts the weight element values by the number of bits corresponding to the values of the second factor group, and
the selector determines the output destination of the weight elements according to the value of the first factor group.
3. The inference device according to claim 2 , wherein
the plurality of accumulation processors is arranged for each of variable values of the first factor group,
each of the plurality of multipliers is assigned to a value obtained by fixing the second factor group at zero and varying the first factor group in the input data separated to the first factor group and the second factor group as a value to be multiplied with the cumulative sum value of the weights.
4. The inference device according to claim 1 , comprising:
a weight processor that accesses to the weight storage and provides the weights to the selector, and
an input data processor that accesses to the input data storage and provides the input data to the selector,
wherein the weight processor and the input data processor do not provide to the selector the input data element whose element value is zero in the input data and a corresponding weight element.
5. The inference device according to claim 1 , wherein
the plurality of accumulation processors is respectively configured by comprising an add-up processor and a register.
6. The inference device according to claim 1 , wherein
the plurality of accumulation processors respectively comprises a register,
the PE comprises:
a cumulative sum value selector that selects one output of the accumulation processor among the plurality of accumulation processors based on the variable values of the input data element, and
a shared add-up processor that adds the value output by the cumulative sum value selector to the weight element value and outputs summation result to the selector.
7. The inference device according to claim 1 , wherein
the PE further comprises an input data restriction processor that restricts variable values of the input data obtained from the input data storage.
8. The inference device according to claim 1 , comprising:
a plurality of PEs,
wherein the plurality of PEs executes convolution computation in the convolutional neural network in parallel.
9. A convolution computation method for an inference device that comprises a weight storage that stores weights, an input data storage that stores input data, a plurality of accumulation processors each calculating and storing cumulative sum of the weights for each of the variable values of the input data elements, a plurality of multipliers respectively associated with the plurality of accumulation processors and assigned to a value to be multiplied with the cumulative sum value of the weights, and an add-up processor into which the multiplication results by the plurality of multipliers are input and that adds up the multiplication results, executes convolution computation in a convolutional neural network using the weights and the input data, wherein
the method comprises:
the accumulation processor adding up weight elements to be multiplied with elements of the input data for each of variable values of the elements of the input data,
the multiplier multiplying each of the variable values of the elements of the input data with each cumulative sum value of weights corresponding to the variable values of the elements of the input data, and
the add-up processor adding up a plurality of multiplication results obtained by the multiplication.
10. The convolution computation method according to claim 9 , comprising:
factorizing the input data elements,
separating the factorized input data to predetermined first factor group and second factor group, and
shifting the weight element values by the number of bits corresponding to the values of the second factor group.
11. The convolution computation method according to claim 10 , comprising:
fixing the second factor group at zero, and
varying the first factor group in the input data.
12. The convolution computation method according to claim 9 , comprising:
providing to the accumulation processor no input data element when element value is zero in the input data and a corresponding weight element.
13. A non-transitory computer readable medium storing a program executed by a computer implemented in an inference device that comprises a weight storage that stores weights, an input data storage that stores input data, a plurality of accumulation processors each calculating and storing cumulative sum of the weights for each of the variable values of the input data elements, a plurality of multipliers respectively associated with the plurality of accumulation processors and assigned to a value to be multiplied with the cumulative sum value of the weights, and an add-up processor into which the multiplication results by the plurality of multipliers are input and that adds up the multiplication results, executes convolution computation in a convolutional neural network using the weights and the input data,
wherein the program causes the computer to execute the following processes:
adding up weight elements to be multiplied with elements of the input data for each of variable values of the elements of the input data by the accumulation processor,
multiplying each of the variable values of the elements of the input data with each cumulative sum value of weights corresponding to the variable values of the elements of the input data by the multiplier, and
adding up a plurality of multiplication results obtained by the multiplication by the add-up processor.
14. The non-transitory computer readable medium storing a program executed by a computer implemented in an inference device according to claim 13 , comprising:
factorizing the input data elements,
separating the factorized input data to predetermined first factor group and second factor group, and
shifting the weight element values by the number of bits corresponding to the values of the second factor group.
15. The non-transitory computer readable medium storing a program executed by a computer implemented in an inference device according to claim 14 , comprising:
fixing the second factor group at zero, and
varying the first factor group in the input data.
16. The non-transitory computer readable medium storing a program executed by a computer implemented in an inference device according to claim 15 , comprising:
providing to the accumulation processor no input data element when element value is zero in the input data and a corresponding weight element.
17. The non-transitory computer readable medium storing a program executed by a computer implemented in an inference device according to claim 13 , wherein
the inference device executes convolution computation in the convolutional neural network in parallel.