Filtering hidden matrix training DNN
View Patent ↗In one aspect, a method of training a DNN includes transmitting an input vector x through a weight matrix W and reading a resulting output vector y, transmitting an error signal δ, transmitting the input vector x with the error signal δ through conductive row wires of a matrix A, and transmitting an input vector e i and reading a resulting output vector y′ as current output. The training also includes updating a hidden matrix H comprising an H value for RPU devices by iteratively adding the output vector y′ multiplied by the transpose of the input vector e i to each H value. The training also includes, when an H value reaches a threshold value, transmitting the input vector e i as a voltage pulse through the conductive column wires of the matrix W simultaneously with sign information of the H values that reached a threshold value as voltage pulses through the conductive row wires matrix W.
1. A deep neural network (DNN), comprising:
an A matrix comprising resistive processing unit (RPU) devices separating intersections between conductive row wires and conductive column wires, whereby the RPU devices comprise processed gradients for weighted connections between neurons in the DNN;
a weight matrix W comprising RPU devices separating intersections between conductive row wires and conductive column wires, whereby the RPU devices comprise weighted connections between neurons in the DNN; and
a hidden matrix comprising an H value for each RPU device in the weight matrix W, wherein each of the H values comprises a summation of iteratively added values from the matrix A until at least one H value is over a threshold.
2. The DNN of claim 1 , further comprising a reference array, interconnected with the A matrix array, that stores zero-weight conductance values for the RPU devices in the cross-point array.
3. The DNN of claim 1 , wherein the hidden matrix H comprises digitally stored values for each RPU device in the weight matrix W.