Method for neural network with weight quantization

1 . A method, comprising:

training a spiking neural network (SNN) in a first device to generate a plurality of first weight values of M bits;

calculating a plurality of second weight values of N bits corresponding to the plurality of first weight values according to a threshold value of the SNN, the number M, and the plurality of first weight values, wherein the number N is smaller than the number M, wherein the threshold value of the SNN is compared with a membrane potential of a neuron of the SNN to determine whether the neuron fires an output spike;

retraining the spiking neural network with the plurality of second weight values to update the plurality of second weight values; and

performing a write operation to save the updated plurality of second weight values in a memory in a second device for performing a spiking neural network operation in the second device,

wherein the calculating the plurality of second weight values comprises:

calculating each of the plurality of second weight values based on an equation below,

W

q

=

⌈

W

r

V

th

⨯

2

N

-

1

⌋

wherein Wq is a corresponding one of the plurality of second weight values, Wr is a corresponding one of the plurality of first weight values, and Vth is the threshold value.

2 . The method of claim 1 , wherein the threshold value is of the N bits.

3 . The method of claim 1 , wherein a maximum value of the plurality of second weight values equals to (2 N-1 ), and

a minimum value of the plurality of second weight values equals to −(2 N-1 ).

4 . The method of claim 1 , wherein the calculating the plurality of second weight values comprises:

calculating the plurality of second weight values according to a ratio between the threshold value and a maximum value of the plurality of first weight values;

wherein the retraining the spiking neural network with the plurality of second weight values comprises:

generating a training factor K in the spiking neural network based on a comparison between output data and desire data corresponding to training data for retraining the spiking neural network, wherein the threshold value is of (N+K) bits;

generating the membrane potential according to the training data and the plurality of second weight values; and

when a (N+K+1)-th bit of the membrane potential is 1, generating the output spike.

5 . The method of claim 1 , wherein the retraining the spiking neural network with the plurality of second weight values comprises:

applying training data on an input of a layer in the spiking neural network;

generating the membrane potential according to the training data and the plurality of second weight values;

comparing the membrane potential with the threshold value to generate output data; and

in response to the comparison between the output data and desire data corresponding to the training data, updating the plurality of second weight values as the optimized plurality of second weight values.

6 . A computer program product comprising a computer readable memory having program codes stored thereon to implement a first spiking neural network, wherein the program codes are executable by a processor to perform operations comprising:

calculating, according to a plurality of first weight values and a threshold value of the first spiking neural network, a plurality of second weight values of N bits and storing the plurality of second weight values in the computer readable memory, wherein the threshold value of the first spiking neural network is compared with a membrane potential of a neuron of the first spiking neural network to determine whether the neuron fires an output spike;

retraining the first spiking neural network with the plurality of second weight values and the threshold value of the first spiking neural network to update the plurality of second weight values and the threshold value of the first spiking neural network; and

outputting the updated plurality of second weight values for a second spiking neural network corresponding to the first spike neural network,

wherein the calculating the plurality of second weight values comprises:

calculating each of the plurality of second weight values based on equation below,

W

q

=

V

t

⁢

h

⁢

⌈

W

r

γ

×

2

N

-

1

⌋

γ

=

V

t

⁢

h

×

2

⌈

log

2

(

max

(

W

r

)

V

t

⁢

h

)

⌋

wherein Wq is a corresponding one of the plurality of second weight values, Wr is a corresponding one of the plurality of first weight values, and Vth is the threshold value.

7 . The computer program product of claim 6 , wherein the program codes are executable by the processor to perform operations further comprising:

training the first spiking neural network to obtain the plurality of first weight values and the threshold value,

wherein the plurality of first weight values are of M bits and the threshold value, and the plurality of second weight values are of N bits, M being greater than N.

8 . The computer program product of claim 6 , wherein the calculating the plurality of second weight values comprises:

calculating each of the plurality of second weight values according to a ratio between the threshold value and a corresponding one of the plurality of first weight values.

9 . The computer program product of claim 6 , wherein the calculating the plurality of second weight values comprises:

calculating each of the plurality of second weight values according to a product of the threshold value and a ratio between the threshold value and a corresponding one of the plurality of first weight values.

10 . The computer program product of claim 6 , wherein the retraining the first spiking neural network with the plurality of second weight values and the threshold value comprises:

applying training data on the first spiking neural network;

generating the membrane potential according to the training data and the plurality of second weight values;

generating the output spike by comparing the membrane potential with the threshold value; and

after generating the output spike, setting the membrane potential to have the threshold value.

11 . The computer program product of claim 6 , wherein the retraining the spiking neural network with the plurality of second weight values comprises:

generating a training factor K in the spiking neural network based on a comparison between output data and desire data corresponding to training data for retraining the spiking neural network,

wherein the threshold value is of (N+K) bits.

12 . The computer program product of claim 11 , wherein the retraining the spiking neural network with the plurality of second weight values further comprises:

generating the membrane potential according to training data and the plurality of second weight values;

comparing a (N+K+1)-th bit of the membrane potential with 0; and

in response to the comparison, generating the output spike.

13 . The computer program product of claim 6 , wherein the program codes are executable by the processor to perform operations further comprising:

output the updated threshold value for the second spiking neural network.

14 . A method, comprising:

generating a plurality of first weight values of M bits by training a spiking neural network (SNN) in a first device;

generating, based on a ratio between the plurality of first weight values and a threshold value of the SNN, a plurality of second weight values of N bits that range from a first value to a second value, wherein the first and second values are associated with the number N, wherein the number N is different from the number M, wherein the threshold value of the SNN is compared with a membrane potential of a neuron of the SNN to determine whether the neuron fires an output spike; and

saving the plurality of second weight values in a memory of a second device and performing a spiking neural network operation with the plurality of second weight values in the second device,

wherein the calculating the plurality of second weight values comprises:

calculating each of the plurality of second weight values based on an equation below,

W

q

=

⌈

W

r

V

th

⨯

2

N

-

1

⌋

wherein Wq is a corresponding one of the plurality of second weight values, Wr is a corresponding one of the plurality of first weight values, and Vth is the threshold value.

15 . The method of claim 14 , wherein the number M is greater than the number N.

16 . The method of claim 14 , further comprising:

retraining the spiking neural network with the plurality of second weight values to generate the updated threshold value,

wherein the updated threshold value is of (N+K) bits, K being a positive integer.

17 . The method of claim 16 , wherein the retraining the spiking neural network comprises:

generating the membrane potential according to training data and the plurality of second weight values;

when a (N+K+1)-th bit of the membrane potential is 1, generating the output spike; and

setting the membrane potential to have the threshold value.

18 . The method of claim 14 , wherein the generating the plurality of second weight values comprises:

rounding the plurality of second weight values to nearest integers.

19 . The method of claim 14 , wherein the threshold value is of the N bits.

20 . The method of claim 14 , wherein a maximum value of the plurality of second weight values equals to (2 N-1 ), and

a minimum value of the plurality of second weight values equals to −(2 N-1 ).