Method and electronic device of updating neural network model

1 . An electronic device of updating a neural network model, comprising:

a transceiver for receiving the neural network model and training data, wherein the neural network model comprises an input node, a first neuron connected to the input node, and a second neuron connected to the first neuron; and

a processor coupled to the transceiver, wherein the processor is configured to execute:

inputting the training data from the input node to the first neuron to output a first value and inputting the first value to the second neuron to output a first estimated value from the second neuron;

quantizing a first weight of the first neuron to generate a third neuron different from the first neuron, and quantizing a second weight of the second neuron to generate a fourth neuron connected to the third neuron, wherein the first weight corresponds to a first floating point number format, and the quantized first weight corresponds to one of an integer format or a second floating point number format different from the first floating point number format, wherein the third neuron is connected to the input node;

inputting the training data from the input node to the third neuron to output a second value and inputting the second value to the fourth neuron to output a second estimated value from the fourth neuron;

updating a first activation function of the first neuron and a second activation function of the second neuron according to the first estimated value and the second estimated value to generate an updated neural network model,

wherein the transceiver is configured for outputting the updated neural network model;

quantizing the second estimated value to generate a quantized second estimated value;

calculating a difference between the first estimated value and the quantized second estimated value; and

training a downstream neuron of the second neuron with an output of the fourth neuron in response to the difference being less than a threshold.

2 . The electronic device of updating the neural network model according to claim 1 , wherein the processor is further configured to execute:

deleting one bit of the second estimated value to generate the quantized second estimated value; and

updating the first activation function and the second activation function according to the first estimated value and the quantized second estimated value.

3 . The electronic device of updating the neural network model according to claim 2 , wherein the bit comprises at least one of a most significant bit and a least significant bit.

4 . The electronic device of updating the neural network model according to claim 1 , wherein the processor is further configured to execute:

deleting one bit of the second estimated value to generate the quantized second estimated value.

5 . The electronic device of updating the neural network model according to claim 4 , wherein the processor is further configured to execute:

training the downstream neuron with an output of the second neuron in response to the difference being greater than or equal to the threshold.

6 . The electronic device of updating the neural network model according to claim 1 , wherein the first activation function is a piecewise function.

7 . The electronic device of updating the neural network model according to claim 1 , further comprising:

a storage medium coupled to the processor and configured for storing the neural network model.

8 . The electronic device of updating the neural network model according to claim 1 , further comprising:

a storage medium coupled to the processor for storing a threshold of a number of iteration times, wherein the processor determines whether to stop updating the neural network model according to the threshold of the number of the iteration times.

9 . A method of updating a neural network model for an electronic device having a transceiver and a processor, comprising:

receiving the neural network model and training data through the transceiver, wherein the neural network model comprises an input node, a first neuron connected to the input node, and a second neuron connected to the first neuron;

inputting the training data from the input node to the first neuron by the processor to output a first value and inputting the first value to the second neuron output a first estimated value from the second neuron;

quantizing a first weight of the first neuron to generate a third neuron different from the first neuron, and quantizing a second weight of the second neuron to generate a fourth neuron connected to the third neuron, wherein the first weight corresponds to a first floating point number format, and the quantized first weight corresponds to one of an integer format or a second floating point number format different from the first floating point number format, wherein the third neuron node is connected to the input node;

inputting the training data from the input node to the third neuron to output a second value and inputting the second value to the fourth neuron to output a second estimated value from the fourth neuron;

updating a first activation function of the first neuron and a second activation function of the second neuron according to the first estimated value and the second estimated value to generate the updated neural network model;

outputting the updated neural network model;

quantizing the second estimated value to generate a quantized second estimated value;

calculating a difference between the first estimated value and the quantized second estimated value; and

training a downstream neuron of the second neuron with an output of the fourth neuron in response to the difference being less than a threshold.

10 . The method of updating the neural network model for the electronic device according to claim 9 , wherein the step of updating the first activation function of the first neuron and the second activation function of the second neuron according to the first estimated value and the second estimated value to generate the updated neural network model comprises:

deleting one bit of the second estimated value to generate a quantized second estimated value; and

updating the first activation function and the second activation function according to the first estimated value and the quantized second estimated value.

11 . The method of updating the neural network model for the electronic device according to claim 10 , wherein the bit comprises at least one of a most significant bit and a least significant bit.

12 . The method of updating the neural network model for the electronic device according to claim 9 , further comprising:

deleting one bit of the second estimated value to generate the quantized second estimated value.

13 . The method of updating the neural network model for the electronic device according to claim 12 , further comprising:

training the downstream neuron with an output of the second neuron in response to the difference being greater than or equal to the threshold.

14 . The method of updating the neural network model for the electronic device according to claim 9 , wherein the first activation function is a piecewise function.

15 . The method of updating the neural network model for the electronic device according to claim 9 , wherein the electronic device further comprises a storage medium, wherein the storage medium stores the neural network model.

16 . The method of updating the neural network model for the electronic device according to claim 9 , wherein the electronic device further comprises a storage medium, wherein the storage medium stores a threshold of a number of iteration times, wherein the processor determines whether to stop updating the neural network model according to the threshold of the number of the iteration times.