MTJ-based hardware synapse implementation for binary and ternary deep neural networks

1. A ternary synapse device comprising:

first and second magnetic tunnel junction (MTJ) devices, wherein each of said MTJ devices has a fixed layer port and a free layer port, and wherein said fixed layer ports of said first and second MTJ devices are connected to each other;

a first control circuit connected to the free layer port of said first MTJ device and configured to provide a first control signal; and

a second control circuit connected to the free layer port of said second MTJ device and configured to provide a second control signal;

wherein said ternary synapse device is configured to perform a gated XNOR (GXNOR) operation between a ternary synapse weight and one or more binary, or ternary activation values; and wherein an output of the GXNOR is represented by a sum of the output currents through both of said first and second MTJ devices.

2. The synapse device of claim 1 , wherein said synapse device is configured to store the ternary synapse weight represented by a state of the MTJ devices.

3. The synapse device of claim 2 , wherein said ternary synapse weight is defined as, and stored as a combination of respective resistance values of each of said first and second MTJ devices.

4. The synapse device of claim 2 , wherein said synapse device is further configured to perform in-situ stochastic update of said ternary or binary synapse weights.

5. An array of synapse devices comprising:

a plurality of ternary synapse devices of claim 1 , arranged in an array of rows and columns,

wherein all of said synapse devices arranged in any one of said columns share an input voltage,

wherein all of said synapse devices arranged in any one of said rows share said first and second control signals, and

wherein outputs of all of said synapse devices arranged in any one of said rows are connected.

6. The array of claim 4 , wherein said array forms a trainable neural network, wherein feedforward is calculated by connecting a row output to ground potential, and summing output currents from all synapses in that row based on Kirchoff's Current Law (KCL).

7. The array of claim 6 , wherein said neural network represents a synaptic weight matrix comprising all of said synapse weights of each of said synapse devices in said array.

8. The array of claim 7 , wherein an output vector of said neural network is calculated as a weighted sum of all of said input voltages multiplied by said synaptic weightings matrix.

9. A method comprising:

providing an array of synapse devices arranged in rows and columns, wherein each of said synapse devices comprises:

first and second magnetic tunnel junction (MTJ) devices, wherein each of said MTJ devices has a fixed layer port and a free layer port, and wherein said fixed layer ports of said first and second MTJ devices are connected to each other,

a first control circuit connected to the free layer port of said first MTJ device and configured to provide a first control signal, and

a second control circuit connected to the free layer port of said second MTJ device and configured to provide a second control signal,

wherein said first and second control circuits are configured to perform a gated XNOR (GXNOR) operation between synapse and activation values, and wherein an output of the GXNOR is represented by the output current through both of said first and second MTJ devices,

wherein all of said synapse devices arranged in any one of said columns share an input voltage,

wherein all of said synapse devices arranged in any one of said rows share said first and second control signals, and

wherein outputs of all of said synapse devices arranged in any one of said rows are connected; and

at a training stage, training said array of synapse devices by:

(i) inputting all of said input voltages associated with each of said columns,

(ii) setting said first and second control signals associated with each of said rows to perform said GXNOR operation, and

(iii) calculating an output vector of said array as a weighted sum of said input voltages multiplied by a synaptic weightings matrix comprising synapse weights of all of said synapse devices in said array.

10. The method of claim 9 , wherein said training further comprises comparing said output vector to a training dataset input, wherein said comparing leads to an adjustment of said synaptic weightings matrix.

11. The method of claim 9 , wherein each of said synapse devices is configured to store said synapse weight represented by a state of the MTJ devices, wherein said synapse weight is ternary or binary.

12. The method of claim 11 , wherein said synapse weight is defined as and stored as a combination of respective resistance values of each of said first and second MTJ devices.

13. The method of claim 11 , wherein each of said synapse devices is further configured to perform in-situ stochastic update of said ternary or binary synapse weights.

14. The method of claim 11 , wherein said array forms a trainable neural network.

15. The method of claim 14 , wherein said neural network represents said synaptic weight matrix comprising all of said synapse weights of each of said synapse devices in said array.

16. The method of claim 15 , wherein said output vector of said neural network is calculated as a weighted sum of all of said input voltages multiplied by said synaptic weightings matrix.

17. A computer memory structure comprising:

a plurality of ternary synapse devices, each comprising:

first and second magnetic tunnel junction (MTJ) devices, wherein each of said MTJ devices has a fixed layer port and a free layer port, and wherein said fixed layer ports of said first and second MTJ devices are connected to each other,

a first control circuit connected to the free layer port of said first MTJ device and configured to provide a first control signal, and

a second control circuit connected to the free layer port of said second MTJ device and configured to provide a second control signal,

wherein said ternary synapse devices are configured to perform a gated XNOR (GXNOR) operation between a ternary synapse weight and one or more binary, or ternary activation values; and wherein an output of the GXNOR is represented by a sum of the output currents through both of said first and second MTJ devices.

18. The array of claim 6 , wherein at a training stage, the first control signal and second control signal of a ternary synapse are applied as voltage pulses, wherein a sign and a duration of the voltage pulses are calculated according to a gradient-based update value of said ternary synapse, so as to ensure a required switching probability of MTJ devices of that ternary synapse.