Granted Patent B2

US 12,632,738 · App. 17/299,102 · Granted May 19, 2026

Inventors: Loai Danial (Nazareth, IL); Shahar Kvatinsky (Hanaton, IL)

Assignee: TECHNION RESEARCH & DEVELOPMENT FOUNDATION LIMITED

G06N3/088G06N3/049G06N3/063

Abstract

Claims (32)

1 . An artificial neural network (NN) circuit comprising:

a plurality of interconnected neural network elements, each neural network element comprising:

a neuron circuit comprising:

a first delta-sigma modulator configured to receive input voltage signal and generate a modulated representation of the input voltage signal; and an averaging filter, wherein a transfer function of the neuron circuit serves as an activation function; and

at least one synapse device comprising a memristor connected to an output of said neuron circuit, wherein a conductance of the memristor represents an adjustable synaptic weight of said at least one synapse device; and

a memristor driver circuit comprising:

at least one second delta-sigma modulator configured to receive at least one stored weighted sum and at least one desired value form a training dataset;

at least one integrator module adapted to implement an averaging filter on output of said second delta-sigma modulator;

at least one subtractor configured to compute error signal; and

wherein said neural network (NN) is trained by an iterative process comprising:

a read phase comprising:

(i) inputting the at least input voltage signal into said neuron circuits; and

(ii) calculating and storing a respective weighted sum of outputs of said neuron circuits, based on said synaptic weights; and

an update phase comprising:

(i) feeding the at least one stored weighted sum and the at least one desired value from the training dataset to the second delta-sigma modulator to generate two modulated outputs;

(ii) calculating a cost function by subtracting the two modulated outputs using said subtractor; and

(iii) adjusting said synaptic weights to minimize the cost function based on a gradient descent algorithm.

2 . The artificial neural network (NN) circuit of claim 1 , wherein said plurality of interconnected neural elements form at least one trainable single-layer neural network, arranged as a memristive crossbar array comprising a synaptic weightings matrix.

3 . The artificial neural network (NN) circuit of claim 2 , wherein an output vector of said single-layer neural network comprises a plurality of weighted sum of said outputs of said neuron circuits multiplied by said synaptic weightings matrix.

4 . The artificial neural network (NN) circuit of claim 3 , further comprising an output circuit comprising at least one delta-sigma modulator, wherein said output circuit encodes said output vector.

5 . The artificial neural network (NN) circuit of claim 2 , wherein said neural network comprises two or more of said single-layer neural networks (NNs) to form a deep neural network (DNN).

6 . The artificial neural network (NN) circuit of claim 1 , wherein said iterative process continues until the at least one stored weighted sum of outputs corresponds to the desired value from the training dataset.

7 . The artificial neural network (NN) circuit of claim 1 , further comprising a plurality of input neuron circuits, a plurality of synapse devices, and at least one output neuron circuit, wherein, at the training stage, said neural network (NN) is trained by an unsupervised spike time-dependent plasticity (STDP) process, wherein outputs of said neuron circuits reflect spikes encoded in time.

8 . The artificial neural network (NN) circuit of claim 7 , wherein said STDP process comprises comparing pre-synaptic and post-synaptic outputs of said neuron circuits, wherein a difference detected in said comparison leads to long-term potentiation or long-term depression.

9 . The artificial neural network (NN) circuit of claim 1 , wherein the gradient descent algorithm is a synchronous algorithm, comprising:

a read clock cycle, during which the at least input voltage signal is passed through the synapse devices of the interconnected neural network elements to generate an output vector; and

a write clock cycle, during which a mean-square error cost function is calculated to update the synaptic weights.

10 . The artificial neural network (NN) circuit of claim 9 , wherein said delta-sigma modulators are configured to oversample input voltage signals, and wherein said neuron circuits further comprise:

a memory component, adapted to store the output vector obtained after the read cycle; and

an integrator module, coupled with said memory element, and adapted to: (i) implement the averaging filter on said output vector and (ii) be reset after every write cycle.

11 . The artificial neural network (NN) circuit of claim 1 , wherein said read phase reflects a feedforward operation of said neural network (NN), and said update phase reflects an error backpropagation operation of said neural network (NN).

12 . The artificial neural network (NN) circuit of claim 1 , wherein said memristor driver circuit further comprises at least one operational amplifier configured to amplify output signals from said memristors.

Assignments (1)

Continuity (2)

References Cited (28)