Neural network processing unit including approximate multiplier and system on chip including the same
A neural network processing unit may be configured to perform an approximate multiplication operation and a system on chip may include the neural network processing unit. The neural network processing unit may include a plurality of neural processing units and may perform a computation based on one or more instances of input data and a plurality of weights. At least one neural processing unit is configured to receive a first value and a second value and perform an approximate multiplication operation based on the first value and the second value and is further configured to perform a stochastic rounding operation based on an output value of the approximate multiplication operation.
1. A neural network processing unit configured to perform a computation based on one or more instances of input data and a plurality of weights, the neural network processing unit comprising:
processing circuitry configured to output at least a first control signal and a second control signal to at least one neural processor (NPU), the first and second control signals respectively configured to enable a selection by the at least one NPU between a training mode and an inference mode; and
a plurality of neural processors (NPUs) configured to implement a neural network, and including the at least one NPU,
wherein the at least one NPU of the plurality of NPUs is configured to
switch to a fixed-point approximate multiplication training mode in response to receiving the first control signal,
receive a first value and a second value while in the fixed-point approximate multiplication training mode,
perform a fixed-point approximate multiplication operation based on the first value and the second value in response to receiving the first value and the second value while in the fixed-point approximate multiplication training mode,
review an output value for a loss of accuracy, the output value including a result of the fixed-point approximate multiplication operation and the review including
performing a stochastic rounding operation on the output value, and
determining, based on a result of the stochastic rounding operation, the loss of accuracy for the output value, and
train the neural network by tuning a parameter of the at least one NPU based on the determined loss, and
wherein the at least one NPU of the plurality of NPUs is configured to
select a general multiplication inference mode in response to receiving the second control signal,
receive an input value while in the general multiplication inference mode, and
perform a general multiplication operation based on the input value and the tuned parameter in response to receiving the input value while in the general multiplication inference mode.
2. The neural network processing unit of claim 1 , wherein the at least one NPU is further configured to
alternatively select one element of the one or more instances of input data and an output value of one NPU of the plurality of NPUs, and
output the selected one element as the first value.
3. The neural network processing unit of claim 1 , wherein the second value includes at least one weight of the plurality of weights.
4. The neural network processing unit of claim 1 , wherein the at least one NPU is further configured to
accumulate one or more output values of the fixed-point approximate multiplication operation; and
perform an addition operation based on the output value of the approximate multiplication operation and an output value of the accumulating.
5. The neural network processing unit of claim 4 , wherein the at least one NPU is configured to perform the stochastic rounding operation on the output value of the accumulating.
6. The neural network processing unit of claim 1 , wherein the at least one NPU includes a fixed-point-type device.
7. A system on chip, comprising:
one or more semiconductor intellectual property cores (IPs);
processing circuitry configured to output at least a first control signal and a second control signal to at least one neural processor (NPU), the first and second control signals respectively configured to enable a selection by the at least one NPU between a training mode and an inference mode; and
a neural network processing unit configured implement a neural network and to receive input data from the one or more IPs, and perform a neural network computation based on the input data and a plurality of weights, the neural network processing unit including a plurality of neural processors (NPUs) and the plurality of NPUs include the at least one NPU,
wherein the at least one NPU of the plurality of NPUs is configured to
switch to a fixed-point approximate multiplication training mode in response to receiving the first control signal,
receive a first value and a second value while in the fixed-point approximate multiplication training mode,
perform a fixed-point approximate multiplication operation on the first value and the second value in response to receiving the first value and the second value while in the fixed-point approximate multiplication training mode,
review an output value for a loss of accuracy, the output value including a result of the approximate multiplication operation and the review including
perform a stochastic rounding operation on the output value to output a post activation regarding the output of the approximate multiplication operation, and
determine, based on the result of the stochastic rounding operation, the loss of accuracy for the output value, and
train the neural network by tuning a parameter of the at least one NPU based on the determined loss, and
wherein the at least one NPU of the plurality of NPUs is configured to
select a general multiplication inference mode in response to receiving the second control signal,
receive an input value while in the general multiplication inference mode, and
perform a general multiplication operation based on the input value and the tuned parameter in response to receiving the input value while in the general multiplication inference mode.
8. The system on chip of claim 7 , wherein the neural network processing unit further includes data random access memory (data RAM) configured to receive training data from the one or more IPs in the fixed-point approximate multiplication training mode and store the training data.
9. The system on chip of claim 8 , wherein the at least one NPU is configured to
receive training data output from the data RAM and an output value of one of the plurality of NPUs,
select one of the training data and the output value, and
output the selected one of the training data and the output value as the first value.
10. The system on chip of claim 7 , wherein the second value includes at least one weight of the plurality of weights.
11. The system on chip of claim 7 , wherein the at least one NPU is configured to
accumulate one or more output values of the fixed-point approximate multiplication operation,
perform an addition operation based on the output value of the fixed-point approximate multiplication operation and an output value of the accumulating, and
perform the stochastic rounding operation on the output value of the accumulating.
12. A neural network processing unit configured to perform a training operation based on one or more instances of training data and a plurality of weights in a training mode or to perform an inference operation based on one or more instances of input data and the plurality of weights in an inference mode, the neural network processing unit comprising:
a controller configured to output at a first control signal and a second control signal to at least one neural processor (NPU), the first and second control signals respectively configured to enable a selection by the at least one NPU between the training mode and the inference mode; and
a plurality of neural processors (NPUs) including the at least one NPU,
wherein the at least one NPU of the plurality of NPUs is configured to,
switch to a fixed-point approximate multiplication training mode in response to receiving the first control signal,
receive a first value and a second value while in the fixed-point approximate multiplication training mode,
perform a fixed-point approximate multiplication operation on the first value and the second value in the training mode in response to receiving the first value and the second value while in the fixed-point approximate multiplication training mode,
perform an addition operation on an output value of the fixed-point approximate multiplication operation and a third value,
accumulate an output value of the fixed-point approximate multiplication operation,
review an output value for a loss of accuracy, the output value including an accumulation value output based on the accumulating and the review including
performing a stochastic rounding operation on an accumulation value output based on the accumulating to output a post activation regarding the accumulation value,
determining, based on a result of the stochastic rounding operation, the loss of accuracy for the output value, and
train the neural network by tuning a parameter of the at least one NPU based on the determined loss, and
wherein the at least one NPU of the plurality of NPUs is configured to
select a general multiplication inference mode in response to receiving the second control signal,
to receive an input value while in the general multiplication inference mode, and
perform a general multiplication operation based on the input value and the tuned parameter in response to receiving the input value while in the general multiplication inference mode.
13. The neural network processing unit of claim 12 , wherein
the first value includes one of the one or more instances of training data and an output value of one of the plurality of NPUs, and
the second value includes at least one weight of the plurality of weights.
14. The neural network processing unit of claim 12 , wherein the third value includes the accumulation value output based on the accumulating.
15. The neural network processing unit of claim 12 , wherein the training operation includes a fixed-point-type training operation including the fixed-point approximate multiplication operation and an approximate addition operation.
16. The neural network processing unit of claim 1 , wherein the at least one NPU includes
an approximate multiplier configured to perform the fixed-point approximate multiplication operation and to be enabled by the first control signal, and
an approximate adder configured to perform an approximate addition operation on an output of the approximate multiplier.
17. The system on chip of claim 7 , wherein the at least one NPU includes
an approximate multiplier configured to perform the fixed-point approximate multiplication operation and to be enabled by the first control signal, and
an approximate adder configured to perform an approximate addition operation on an output of the approximate multiplier.