Integrated circuits for neural networks
An integrated circuit includes an analog network of analog components fabricated by a method. The method includes obtaining a neural network topology and weights of a trained neural network. The method also includes transforming the neural network topology to an equivalent analog network of analog components including operational amplifiers and resistors. Each operational amplifier represents an analog neuron, and each resistor represents a connection between analog neurons. The method also includes computing a weight matrix for the equivalent analog network based on the weights of the trained neural network. The method also includes generating a resistance matrix for the weight matrix. The method also includes generating lithographic masks for fabricating a circuit implementing the equivalent analog network based on the resistance matrix. The method also includes fabricating the circuit based on the one or more lithographic masks using a lithographic process.
1. An integrated circuit for implementing a neural network to provide an improved signal-to-noise ratio through arrangements of analog components, comprising:
an analog network of analog components fabricated by a method comprising the steps of:
obtaining a neural network topology and weights of a trained neural network;
transforming the neural network topology to an equivalent analog network of analog components including a plurality of operational amplifiers and a plurality of resistors, wherein each operational amplifier represents a respective analog neuron, and each resistor represents a respective connection between a respective first analog neuron and a respective second analog neuron;
computing a weight matrix for the equivalent analog network based on the weights of the trained neural network, wherein each element of the weight matrix represents a respective connection;
generating a resistance matrix for the weight matrix, wherein each element of the resistance matrix corresponds to a respective weight of the weight matrix;
generating one or more lithographic masks for fabricating a circuit implementing the equivalent analog network of analog components based on the resistance matrix; and
fabricating the circuit based on the one or more lithographic masks using a lithographic process, wherein:
the trained neural network was trained using training datasets from at least one of: sensor array measurements for detecting and analyzing gas mixtures, semiconductor device aging data for predicting device lifetime, battery performance data for monitoring battery health parameters, voice data for command recognition, biometric sensor data for physiological monitoring during physical activity, or motion sensor data for human activity recognition;
the equivalent analog network is restricted to a maximum number of input and output connections per analog neuron, a predetermined signal limit, a predetermined number of layers, a predetermined range for number of analog neurons, and a predetermined range for number of connections; and
the fabricated circuit implements the trained neural network to perform real-time detection, prediction, or monitoring tasks corresponding to a respective one of the training datasets.
2. The integrated circuit of claim 1 , further comprising one or more digital to analog converters configured to generate analog input for the equivalent analog network of analog components based on one or more digital signals.
3. The integrated circuit of claim 1 , further comprising an analog signal sampling module configured to process 1-dimensional or 2-dimensional analog inputs with a sampling frequency based on number of inferences of the integrated circuit.
4. The integrated circuit of claim 1 , further comprising a voltage converter module to scale down or scale up analog signals to match operational range of the plurality of operational amplifiers.
5. The integrated circuit of claim 1 , further comprising a tact signal processing module configured to process one or more frames obtained from a CCD camera.
6. The integrated circuit of claim 1 , wherein the trained neural network is a long short-term memory (LSTM) network, the integrated circuit further comprising one or more clock modules to synchronize signal tacts and to allow time series processing.
7. The integrated circuit of claim 1 , further comprising one or more analog to digital converters configured to generate digital signal based on output of the equivalent analog network of analog components.
8. The integrated circuit of claim 1 , wherein the circuit includes one or more signal processing modules configured to process 1-dimensional or 2-dimensional analog signals obtained from edge applications.
9. The integrated circuit of claim 1 , wherein:
the trained neural network is trained, using training datasets containing signals of arrays of gas sensors on different gas mixture, for selective sensing of different gases in a gas mixture containing predetermined amounts of gases to be detected;
the neural network topology is a 1-Dimensional Deep Convolutional Neural network (1D-DCNN) designed for detecting 3 binary gas components based on measurements by 16 gas sensors, and includes 16 sensor-wise 1-D convolutional blocks, 3 shared or common 1-D convolutional blocks and 3 dense layers; and
the equivalent analog network includes: (i) a maximum of 100 input and output connections per analog neuron, (ii) delay blocks to produce delay by a predetermined number of time steps, (iii) a signal limit of 5, (iv) 15 layers, (v) approximately 100,000 analog neurons, and (vi) approximately 4,900,000 connections.
10. The integrated circuit of claim 1 , wherein:
the trained neural network is trained, using training datasets containing thermal aging time series data for different MOSFETs, for predicting remaining useful life (RUL) of a MOSFET device;
the neural network topology includes 4 LSTM layers with 64 neurons in each layer, followed by two dense layers with 64 neurons and 1 neuron, respectively; and
the equivalent analog network includes: (i) a maximum of 100 input and output connections per analog neuron, (ii) a signal limit of 5, (iii) 18 layers, (iv) between 3,000 and 3,200 analog neurons, and (v) between 123,000 and 124,000 connections.
11. The integrated circuit of claim 1 , wherein:
the trained neural network is trained, using training datasets containing time series data including discharge and temperature data during continuous usage of different commercially available Li-Ion batteries, for monitoring state of health (SOH) and state of charge (SOC) of Lithium Ion batteries to use in battery management systems (BMS);
the neural network topology includes an input layer, 2 LSTM layers with 64 neurons in each layer, followed by an output dense layer with 2 neurons for generating SOC and SOH values; and
the equivalent analog network includes: (i) a maximum of 100 input and output connections per analog neuron, (ii) a signal limit of 5, (iii) 9 layers, (iv) between 1,200 and 1,300 analog neurons, and (v) between 51,000 and 52,000 connections.
12. The integrated circuit of claim 1 , wherein:
the trained neural network is trained, using training datasets containing time series data including discharge and temperature data during continuous usage of different commercially available Li-Ion batteries, for monitoring state of health (SOH) of Lithium Ion batteries to use in battery management systems (BMS);
the neural network topology includes an input layer with 18 neurons, a simple recurrent layer with 100 neurons, and a dense layer with 1 neuron; and
the equivalent analog network includes: (i) a maximum of 100 input and output connections per analog neuron, (ii) a signal limit of 5, (iii) 4 layers, (iv) between 200 and 300 analog neurons, and (v) between 2,200 and 2,400 connections.
13. The integrated circuit of claim 1 , wherein:
the trained neural network is trained, using training datasets containing speech commands, for identifying voice commands;
the neural network topology is a Depthwise Separable Convolutional Neural Network (DS-CNN) layer with 1 neuron; and
the equivalent analog network includes: (i) a maximum of 100 input and output connections per analog neuron, (ii) a signal limit of 5, (iii) 13 layers, (iv) approximately 72,000 analog neurons, and (v) approximately 2.6 million connections.
14. The integrated circuit of claim 1 , wherein:
the trained neural network is trained, using training datasets containing photoplethysmography (PPG) data, accelerometer data, temperature data, and electrodermal response signal data for different individuals performing various physical activities for a predetermined period of times and reference heart rate data obtained from ECG sensor, for determining pulse rate during physical exercises based on PPG sensor data and 3-axis accelerometer data;
the neural network topology includes two Conv1D layers each with 16 filters and a kernel of 20, performing time series convolution, two LSTM layers each with 16 neurons, and two dense layers with 16 neurons and 1 neuron, respectively; and
the equivalent analog network includes: (i) delay blocks to produce a predetermined number of time steps, (ii) a maximum of 100 input and output connections per analog neuron, (iii) a signal limit of 5, (iv) 16 layers, (v) between 700 and 800 analog neurons, and (vi) between 12,000 and 12,500 connections.
15. The integrated circuit of claim 1 , wherein:
the trained neural network is trained to classify different objects based on pulsed Doppler radar signal; and
the neural network topology includes multi-scale LSTM neural network.
16. The integrated circuit of claim 1 , wherein:
the trained neural network is trained to perform human activity type recognition, based on inertial sensor data;
the neural network topology includes three channel-wise convolutional networks each with a convolutional layer of 12 filters and a kernel dimension of 64, and each followed by a max pooling layer, and two common dense layers of 1024 neurons and N neurons, respectively, where N is a number of classes; and
the equivalent analog network includes: (i) delay blocks to produce a predetermined number of time steps, (ii) a maximum of 100 input and output connections per analog neuron, (iii) an output layer of 10 analog neurons, (iv) signal limit of 5, (v) 10 layers, (vi) between 1,200 and 1,300 analog neurons, and (vi) between 20,000 and 21,000 connections.
17. The integrated circuit of claim 1 , wherein the trained neural network is further trained to detect abnormal patterns of human activity based on accelerometer data that is merged with heart rate data using a convolution operation.