Memory device and operation method thereof
Provided is an operation method for a memory device, the memory device being used for implementing an Artificial Neural Network (ANN). The operation method includes: reading from the memory device a weight matrix of a current layer of a plurality of layers of the ANN to extract a plurality of neuro values; determining whether to perform calibration; when it is determined to perform calibration, recalculating and updating a mean value and a variance value of the neuro values; and performing batch normalization based on the mean value and the variance value of the neuro values.
1. An operation method for a memory device, the memory device being used for implementing an Artificial Neural Network (ANN), the operation method including:
reading from a plurality of memory cells of the memory device a weight matrix of a current layer of a plurality of layers of the ANN to extract a plurality of neuro values by the memory device;
determining whether to perform calibration by the memory device, wherein the plurality of neuro values are formed by currents generated by the plurality of memory cells of the memory device;
when it is determined to perform calibration, recalculating and updating a mean value and a variance value of the plurality of neuro values by the memory device; and
performing batch normalization based on the mean value and the variance value of the plurality of neuro values by the memory device,
the step of determining whether to perform calibration includes: determining to perform calibration when weight shift is induced;
when calibration of the mean value and the variance value of the plurality of the neuro values is determined, before batch normalization is performed, a plurality of weights are read out from the plurality of memory cells of the memory array and are used to extract the plurality of neuro values to recalculate and update the mean value and the variance value of the plurality of neuro values for performing batch normalization.
2. The operation method according to claim 1 , wherein an activation function is performed after batch normalization is performed.
3. The operation method according to claim 1 , wherein an activation function is performed before the mean value and the variance value of the plurality of neuro values are recalculated and updated.
4. The operation method according to claim 1 , wherein the step of determining whether to perform calibration includes: it is determined to perform calibration when a weight is written into the memory device first time.
5. The operation method according to claim 1 , wherein the step of determining whether to perform calibration includes: it is determined to perform calibration when triggered by a user.
6. A memory device for implementing an Artificial Neural Network (ANN), the memory device including:
a memory array, for storing respective weight matrix of a plurality of layers of the ANN; and
a hardware controller coupled to the memory array,
the hardware controller being configured for:
reading from a plurality of memory cells of the memory array a weight matrix of a current layer of the ANN to extract a plurality of neuro values by the memory device;
determining whether to perform calibration by the memory device, wherein the plurality of neuro values are formed by currents generated by the plurality of memory cells of the memory device;
when the hardware controller determines to perform calibration, recalculating and updating a mean value and a variance value of the plurality of neuro values by the memory device; and
performing batch normalization based on the mean value and the variance value of the plurality of neuro values by the memory device,
wherein the hardware controller determines to perform calibration when weight shift is induced;
when calibration of the mean value and the variance value of the plurality of the neuro values is determined, before batch normalization is performed, a plurality of weights are read out from the plurality of memory cells of the memory array and are used to extract the plurality of neuro values to recalculate and update the mean value and the variance value of the plurality of neuro values for performing batch normalization.
7. The memory device according to claim 6 , wherein the hardware controller is configured for: performing an activation function after batch normalization is performed.
8. The memory device according to claim 6 , wherein the hardware controller is configured for: performing an activation function before the mean value and the variance value of the plurality of neuro values are recalculated and updated.
9. The memory device according to claim 6 , wherein the hardware controller is configured for: determining to perform calibration when a weight is written into the memory device first time.
10. The memory device according to claim 6 , wherein the hardware controller is configured for: determining to perform calibration when triggered by a user.