STDP WITH SYNAPTIC FATIGUE FOR LEARNING OF SPIKE-TIME-CODED PATTERNS IN THE PRESENCE OF PARALLEL RATE-CODING
A circuit implementing a spiking neural network that includes a learning component that can learn from temporal correlations in the spikes regardless of correlations in the rates. In some embodiments, the learning component comprises a rate-discounting component. In some embodiments, the learning rule computes a rate-normalized covariance (normcov) matrix, detects clusters in this matrix, and sets the synaptic weights according to these clusters. In some embodiments, a synapse with a long-term plasticity rule has an efficacy that is composed by a weight and a fatiguing component. In some embodiments, A Hebbian plasticity component modifies the weight component and a short-term fatigue plasticity component modifies the fatiguing component. The fatigue component increases with increases in the presynaptic spike rate. In some embodiments, the fatigue component increases are implemented in a spike-based manner. In some embodiments, the Hebbian plasticity is a spike-timing-dependent plasticity (STDP), resulting in a fatiguing STDP (FSTDP) synapse.
1 . A method comprising:
providing, by machine logic, at least one presynaptic artificial neuron generating a sequence of presynaptic spikes having a timing;
providing, by machine logic, a postsynaptic artificial neuron comprising a membrane potential;
providing, by machine logic, a learning rule component comprising a synaptic efficacy and a synaptic plasticity; and
providing, by machine logic, the learning rule configured to modify the synaptic efficacy by a learning rule;
wherein:
the learning rule is based on the timing and discounts presynaptic spike rates;
the learning rule comprises either: (i) a normalized covariance component configured to generate a normalized covariance matrix based on the sequence of presynaptic spikes and a clustering component configured to modify the efficacy according to clusters in the normalized covariance matrix, or (ii) at least one plastic synapse that is operatively connected to the at least one presynaptic artificial neuron to receive the sequence of presynaptic spikes and modify the postsynaptic membrane potential based on the synaptic efficacy and the sequence of presynaptic spikes;
the efficacy comprises a weighting and a rate-normalization, and wherein the at least one plastic synapse comprises a Hebbian plasticity component configured to modify the weighting, and a rate-dependent plasticity component configured to modify the rate-normalization;
the Hebbian plasticity component comprises a spike-timing-dependent plasticity rule and the rate-dependent plasticity component comprises a fatigue plasticity rule;
the sequence of presynaptic spikes is characterized by a spike rate and the fatigue plasticity rule is configured to modify the rate-normalization component so that each spike in the sequence of presynaptic spikes has a reduced effect on the membrane potential when the spike rate increases; and
the at least one plastic synapse comprises either: (i) a non-volatile memory element with volatile characteristics that match a form of fatigue, or (ii) a digital complementary metal-oxide semiconductor (CMOS) circuit or an analog CMOS circuit.