System and method for training artificial neural networks

1. A method, comprising:

performing, at a computing device that includes one or more processors, magnetic random access memory (MRAM), and a non-transitory computer-readable storage medium storing instructions for execution by the one or more processors, a set of operations that includes:

receiving first data on which to train an artificial neural network (ANN);

using the MRAM, training the ANN by:

performing a first set of training iterations on the first data, wherein each of the first set of iterations includes writing values for a set of weights of the ANN to the MRAM using first write parameters corresponding to a first write error rate;

during the first set of training iterations, performing error detection that includes detecting an error in a respective weight stored in the MRAM;

replacing a value stored for the respective weight with a zero value prior to using the respective weight; and

after performing the first set of iterations, performing a second set of training iterations on the first data, wherein each of the second set of iterations includes writing values for the set of weights of the ANN to the MRAM using second write parameters corresponding to a second write error rate, wherein the second write error rate is lower than the first write error rate;

storing values for the trained ANN, wherein the trained ANN is configured to classify second data based on the stored values.

2. The method of claim 1 , wherein:

each of the first set of iterations includes writing values for a set of biases and a set of activations of the ANN; and

each of the second set of iterations includes writing values for the set of biases and the set of activations of the ANN.

3. The method of claim 1 , wherein:

the first write parameters include a first write pulse duration; and

the second write parameters include a second write pulse duration that is longer than the first write pulse duration.

4. The method of claim 1 , wherein:

the first write parameters include a first write current; and

the second write parameters include a second write current that is greater than the first write current.

5. The method of claim 1 , wherein:

writing the values for the set of weights of the ANN to the MRAM using the write parameters corresponding to the first write error rate includes writing the values without using an error-correcting code; and

writing the values for the set of weights of the ANN to the MRAM using the second write parameters corresponding to the second write error rate includes writing the values using an error-correcting code.

6. The method of claim 1 , wherein:

each of the first set of training iterations includes:

reading the values for the set of weights of the ANN from the MRAM;

using the one or more processors, performing a set of arithmetic operations to update the values for the set of weights of the ANN,

wherein the values for the set of weights of the ANN written to the MRAM for the iteration are the updated set of weights.

7. The method of claim 6 , wherein:

each of the second set of training iterations includes:

reading the values for the set of weights of the ANN from the MRAM;

using the one or more processors, performing the set of arithmetic operations to update the values for the set of weights of the ANN,

wherein the values for the set of weights of the ANN written to the MRAM for the iteration are the updated set of weights.

8. The method of claim 1 , wherein the MRAM is on the same chip as the one or more processors.

9. The method of claim 1 , wherein the first write parameters correspond to a bit error-rate below a threshold for convergence of the first set of training iterations.

10. The method of claim 9 , wherein the threshold for convergence is greater than: 10 −3 , 10 −5 , or 10 −7 .

11. The method of claim 1 , wherein the first write error rate is greater than: 10 −4 , 10 −6 , or 10 −8 .

12. The method of claim 1 , wherein the first set of training iterations includes more than 20%, 40%, 60%, 80%, or 95% of a total number of training iterations used for training the ANN.

13. The method of claim 1 , wherein:

each of the first set of iterations includes reading values for the set of weights of the ANN to the MRAM using first read parameters corresponding to a first read error rate; and

each of the second set of iterations includes reading values for the set of weights of the ANN to the MRAM using second read parameters corresponding to a second read error rate, wherein the second read error rate is lower than the first read error rate.

14. The method of claim 13 , wherein:

the first read parameters include a first read pulse duration; and

the second read parameters include a second read pulse duration that is longer than the first read pulse duration.

15. The method of claim 1 , wherein:

the first read parameters include a first read current; and

the second read parameters include a second read current that is greater than the first read current.

16. The method of claim 1 , further comprising, after training the ANN:

receiving second data; and

assigning scores to the second data using the stored values of the trained ANN.

17. A system, comprising:

one or more processors;

magnetic random access memory (MRAM);

write circuitry configured to write data to the MRAM; and

a non-transitory computer-readable storage medium storing instructions for execution by the one or more processors, including instructions for:

receiving first data on which to train an artificial neural network (ANN);

using the MRAM, training the ANN by:

performing a first set of training iterations on the first data, wherein each of the first set of iterations includes writing, using the write circuitry, values for a set of weights of the ANN to the MRAM using first write parameters corresponding to a first write error rate; and

after performing the first set of iterations, performing a second set of training iterations on the first data, wherein each of the second set of iterations includes writing, using the write circuitry, values for the set of weights of the ANN to the MRAM using second write parameters corresponding to a second write error rate, wherein the second write error rate is lower than the first write error rate;

storing values for the trained ANN, wherein the trained ANN is configured to classify second data based on the stored values;

after training the ANN receiving second data, and assigning scores to the second data using the stored values of the trained ANN.

18. A method, comprising:

performing, at a computing device that includes one or more processors, magnetic random access memory (MRAM), and a non-transitory computer-readable storage medium storing instructions for execution by the one or more processors, a set of operations that includes:

receiving first data on which to train an artificial neural network (ANN);

using the MRAM, training the ANN by:

performing a first set of training iterations on the first data, wherein each of the first set of iterations includes writing values for a set of weights of the ANN to the MRAM using first write parameters corresponding to a first write error rate, wherein the first set of training iterations includes more than 20%, 40%, 60%, 80%, or 95% of a total number of training iterations used for training the ANN; and

after performing the first set of iterations, performing a second set of training iterations on the first data, wherein each of the second set of iterations includes writing values for the set of weights of the ANN to the MRAM using second write parameters corresponding to a second write error rate, wherein the second write error rate is lower than the first write error rate;

storing values for the trained ANN, wherein the trained ANN is configured to classify second data based on the stored values.

19. The method of claim 17 , wherein:

each of the first set of iterations includes writing values for a set of biases and a set of activations of the ANN; and

each of the second set of iterations includes writing values for the set of biases and the set of activations of the ANN.

20. The method of claim 17 , wherein:

the first write parameters include a first write pulse duration; and

the second write parameters include a second write pulse duration that is longer than the first write pulse duration.