SYSTEMS AND METHODS FOR ASYMMETRICAL SCALING FACTOR SUPPORT FOR NEGATIVE AND POSITIVE VALUES
Disclosed herein includes a system, a method, and a device for asymmetrical scaling factor support for negative and positive values. A device can include a circuit having a shift circuitry and multiply circuitry. The circuit can be configured to perform computation for a neural network, including multiplying, via the multiply circuitry, a first value and a second value. The circuit can be configured to perform computation for a neural network, including shifting, via the shift circuitry, a result of the multiplying by a determined number of bits. The circuit can be configured to perform computation for a neural network, including outputting the result of the multiplying when a sign bit of the first value is negative, and a result of the shifting when the sign bit of the first value is positive.
1 . A device comprising:
a circuit comprising shift circuitry and multiply circuitry, and configured to perform computation for a neural network, comprising:
multiplying, via the multiply circuitry, a first value and a second value;
shifting, via the shift circuitry, a result of the multiplying by a determined number of bits; and
outputting the result of the multiplying when a sign bit of the first value is negative, and a result of the shifting when the sign bit of the first value is positive.
2 . The device according to claim 1 , wherein:
the circuit further comprises a multiplexer, and
the circuit is configured to output, via the multiplexer according to the sign bit of the first value, the result of the multiplying or the result of the shifting.
3 . The device according to claim 1 , wherein the first value comprises an activation for a first layer of the neural network.
4 . The device according to claim 3 , wherein the determined number of bits corresponds to an exponent with base 2 of a scaling factor of an activation function for the first layer of the neural network.
5 . The device according to claim 4 , wherein the activation function comprises a leaky rectifier linear unit (ReLu) function.
6 . The device according to claim 1 , wherein the determined number of bits is m, and m is an integer greater than 1.
7 . The device according to claim 1 , wherein the determined number of bits is 2.
8 . The device according to claim 1 , wherein the circuit further comprises comparator circuitry configured to determine whether the sign bit of the first value is negative or positive.
9 . The device according to claim 1 , wherein
the circuit includes multiplier and accumulator (MAC) circuitry comprising accumulator circuitry, and
the circuit is further configured to provide a result of the outputting to the accumulator circuitry of the MAC circuitry.
10 . The device according to claim 9 , wherein the computation for the neural network further comprises:
second multiplying, via the multiply circuitry, a third value and a fourth value;
second shifting, via the shift circuitry, a result of the second multiplying by the determined number of bits;
second outputting the result of the second multiplying when a sign bit of the third value is negative, and a result of the second shifting when the sign bit of the third value is positive; and
providing a result of the second outputting to the accumulator circuitry of the MAC circuitry.
11 . A method comprising:
multiplying, by multiply circuitry of a circuit, a first value and a second value for a neural network;
shifting, by shift circuitry of the circuit, a result of the multiplying by a determined number of bits; and
outputting, by the circuit, the result of the multiplying when a sign bit of the first value is negative, and a result of the shifting when the sign bit of the first value is positive.
12 . The method according to claim 11 , comprising outputting, via a multiplexer of the circuit based on the sign bit of the first value, the result of the multiplying or the result of the shifting.
13 . The method according to claim 11 , wherein the first value comprises an activation for a first layer of the neural network.
14 . The method according to claim 13 , wherein the determined number of bits corresponds to an exponent with base 2 of a scaling factor of an activation function for the first layer of the neural network.
15 . The method according to claim 14 , wherein the activation function comprises a leaky rectifier linear unit (ReLu) function.
16 . The method according to claim 11 , wherein the predetermined number of bits is m, and m is an integer greater than 1.
17 . The method according to claim 11 , wherein the determined number of bits is 2.
18 . The method according to claim 11 , further comprising determining, by comparator circuitry of the circuit, whether the sign bit of the first value is negative or positive.
19 . The method according to claim 11 , further comprising providing a result of the outputting to accumulator circuitry of the circuit.
20 . The method according to claim 19 , further comprising:
second multiplying, via the multiply circuitry, a third value and a fourth value;
second shifting, via the shift circuitry, a result of the second multiplying by the predetermined number of bits;
second outputting the result of the second multiplying when a sign bit of the third value is negative, and a result of the second shifting when the sign bit of the third value is positive; and
providing a result of the second outputting to the accumulator circuitry of the circuit.