LOW POWER ANALOG CIRCUITRY FOR ARTIFICIAL NEURAL NETWORKS
A low power analog Long Short-Term Memory (LSTM) recurrent neural network has an input layer, an array of Adaptive Filter Unit for Analog LSTM, a linear projection layer, and an output layer. The output layer has multiple nonlinear amplifiers, a nonlinear element with a sigmoidal input-output characteristic function, and a time-constant adjustable, nonlinear, low pass filter that provides the memory function of the LSTM. The LSTM memory is used with mismatch-robust weights determined by learning by computation of optimal weights values, wherein the objective function minimizes misdetection probability, and used to process a signal to detect events.
1 . A current-mode low-pass filter, comprising:
a state input configured to receive an input current representing a state variable;
a state node configured to provide a filtered state current; and
a control input configured to receive a control signal z that sets a time constant of the low-pass filter to govern a rate at which the filtered state current is updated relative to the input current.
2 . The current-mode low-pass filter of claim 1 , wherein the control signal z is an analog signal.
3 . The current-mode low-pass filter of claim 1 , wherein a decrease in the control signal z increases the time constant.
4 . The current-mode low-pass filter of claim 1 , wherein an increase in the control signal z decreases the time constant.
5 . The current-mode low-pass filter of claim 1 , wherein the filter is configured as a first-order continuous-time low-pass filter.
6 . The current-mode low-pass filter of claim 1 , wherein the state node is coupled to a capacitive element configured to store the state variable.
7 . The current-mode low-pass filter of claim 6 , wherein the capacitive element is integrated on a semiconductor substrate.
8 . The current-mode low-pass filter of claim 1 , wherein the control input comprises a bias current source controlled by the control signal z.
9 . The current-mode low-pass filter of claim 1 , wherein the filter is configured to impede updating of the filtered state current when the control signal z is below a predetermined threshold.
10 . The current-mode low-pass filter of claim 1 , wherein the control signal z is generated in response to a nonlinear activation function, input to the nonlinear activation function including a weighted summation of one or more input signals to an analog recurrent neural network and one or more prior state signals of the analog recurrent neural network.
11 . The current-mode low-pass filter of claim 10 , wherein the nonlinear activation function has a sigmoidal or clipped softplus characteristic.
12 . The current-mode low-pass filter of claim 10 , wherein the nonlinear activation function is implemented using a diode-connected transistor.
13 . The current-mode low-pass filter of claim 1 , wherein the filtered state current saturates when the input current exceeds a predetermined threshold.
14 . The current-mode low-pass filter of claim 12 , wherein the filter is integrated in an analog recurrent neural network cell.
15 . The current-mode low-pass filter of claim 1 , wherein the filtered state current is fed back as an input to the low-pass filter to provide state retention over time.
16 . A method of controlling a state update using a current-mode low-pass filter, comprising:
receiving an input current at a state input of the current-mode low-pass filter, the input current representing a state variable;
applying a control signal z to a control input of the current-mode low-pass filter to set a time constant of the low-pass filter; and
producing, at a state node of the current-mode low-pass filter, a filtered state current that is updated relative to the input current at a rate governed by the time constant.
17 . The method of claim 16 , wherein applying the control signal z comprises decreasing the control signal z to increase the time constant and thereby impede updating of the filtered state current.
18 . The method of claim 16 , wherein applying the control signal z comprises increasing the control signal z to decrease the time constant and thereby increase a rate of updating of the filtered state current.
19 . The method of claim 16 , further comprising storing the filtered state current on a capacitive element coupled to the state node to retain the state variable over time.
20 . The method of claim 16 , further comprising generating the control signal z using a nonlinear activation function having a sigmoidal or clipped softplus characteristic, input to the nonlinear activation function including a weighted summation of one or more input signals to an analog recurrent neural network and one or more prior state signals of the analog recurrent neural network.