Apparatus and methods for rate-modulated plasticity in a neuron network
Apparatus and methods for activity based plasticity in a spiking neuron network adapted to process sensory input. In one approach, the plasticity mechanism of a connection may comprise a causal potentiation portion and an anti-causal portion. The anti-causal portion, corresponding to the input into a neuron occurring after the neuron response, may be configured based on the prior activity of the neuron. When the neuron is in low activity state, the connection, when active, may be potentiated by a base amount. When the neuron activity increases due to another input, the efficacy of the connection, if active, may be reduced proportionally to the neuron activity. Such functionality may enable the network to maintain strong, albeit inactive, connections available for use for extended intervals.
1. A non-transitory computerized spiking neuron apparatus comprising a plurality of computer-readable instructions configured to, when executed:
based on a response by a neuron:
increase efficacy of a connection configured to provide an input to the neuron prior to the response; and
adjust the efficacy of the connection configured to provide the input to the neuron subsequent to the response;
wherein the adjustment of the efficacy is determined based on a rate of the response by the neuron.
2. The apparatus of claim 1 wherein:
the input is characterized by a refresh period of 40 ms; and
the rate of the response is based on an average of the rate of responses by the neuron determined within a time window selected from a range between 1 s and 2000 s, inclusive.
3. The apparatus of claim 1 wherein:
the response comprises one or more spikes generated by the neuron based on the input;
the rate of the response comprises an average rate determined based on at least a number of the one or more spikes that occur within an interval;
the increase of the efficacy is characterized by a first rule; and
the adjustment of the efficacy comprises an increase of the efficacy based on a second rule, where a magnitude of the increase being determined is based at least on the average rate being below a threshold.
4. The apparatus of claim 3 wherein the adjustment of the efficacy further comprises a decrease in the efficacy, where a magnitude of the decrease being based on the average rate being above the threshold.
5. The apparatus of claim 4 wherein the magnitude of the decrease and the magnitude of the increase are each independent of a time interval between the response and the input.
6. The apparatus of claim 5 wherein the magnitude of the increase is configured to be smaller than the magnitude of the decrease, and the magnitude of the increase is characterized by the first rule.
7. The apparatus of claim 5 , wherein:
the input is characterized by a refresh rate of 25 Hz;
the threshold is selected from a range between 0.05 Hz and 0.2 Hz; and
a ratio of the magnitude of the increase based on the second rule to the magnitude of the decrease is selected from the range between 0.05 Hz and 0.2 Hz inclusive.
8. The apparatus of claim 1 , wherein the adjustment of the efficacy is based at least in part on a timer interval between the response and another response that precedes the response.
9. The apparatus of claim 1 wherein:
the efficacy increase comprises an increase of the efficacy based on a delay between the response and the input being below a threshold; and
the efficacy is maintained based on the delay between the response and the input being above the threshold.
10. The apparatus of claim 9 , wherein:
the input is characterized by a refresh rate;
the refresh rate comprises 40 ms; and
the threshold is selected from a range between 500 ms and 1000 ms, inclusive.
11. The apparatus of claim 1 wherein:
the input is characterized by a refresh rate of 40 ms; and
the efficacy increase comprises an increase of the efficacy when a delay between the response and the input is below a threshold, the threshold is selected from a range between 20 ms and 200 ms, inclusive.
12. The apparatus of claim 1 wherein:
the increase of the efficacy is characterized by a potentiation interval configured so that the efficacy of the connection configured to provide the input to the neuron within the potentiation interval prior to the response is increased; and
the efficacy of the connection configured to provide the input to the neuron outside the potentiation interval prior to the response is maintained at a value of the efficacy prior to the response.
13. The apparatus of claim 12 wherein the maintenance of the efficacy at the value of the efficacy prior to the response is characterized by changes to the value of less than 0.01%.
14. A non-transitory computer-readable medium having program code recorded thereon, the program code comprising:
program code to update a connection configured to provide a stimulus to an artificial neuron, the update configured to:
when the stimulus precedes a response generated by the neuron, potentiate the connection;
when the response precedes the stimulus:
potentiate the connection when a neuron activity is below a threshold level; and
depress the connection when the neuron activity is above the threshold level.
15. The apparatus of claim 14 , wherein:
the depression of the connection is configured to delay another response by the neuron subsequent to the response; and
the potentiation of the connection is configured to advance the another response.
16. The apparatus of claim 15 , wherein:
the connection is characterized by a latency configured to delay a provision of the stimulus to the neuron;
the connection potentiation comprises a reduction in the latency; and
the connection depression comprises an increase in the latency.
17. The apparatus of claim 14 , wherein:
the connection is characterized by a weight configured to affect generation of the response by the neuron such that a larger weight corresponds to a greater probability of the response being generated;
the connection potentiation comprises an increase in the weight; and
the connection depression comprises a reduction in the weight.
18. A method of managing a connection in a neuron network based on at least one signal from a neuron, the method comprising:
receiving at least one input via the connection;
sending the at least one signal at a time proximate to the received at least one input;
when the at least one signal is sent prior to the receiving of the at least one input, demoting the connection; and
when the at least one signal is sent after the receiving of the input, promoting the connection.
19. A sensory processing neuron network apparatus, comprising:
a neuron; and
a connection configured to provide an input to the neuron configured to generate a response based on the input;
wherein the connection is further configured to be:
potentiated when the input is within an interval from the response;
adjusted when the input occurs subsequent to the response, the adjustment being
determined based on activity of the neuron prior to the response,
wherein a neuron activity is determined based on at least a number of neuron responses within a time interval to precede a second input, the number of responses including the response.
20. The apparatus of claim 19 , wherein the each individual one of the number of neuron responses comprises a spike.
21. The apparatus of claim 19 , wherein:
at least a portion of a first interval corresponds to the input preceding the response; and
the remaining portion of the first interval corresponds to the input subsequent to the response.
22. The apparatus of claim 19 , wherein:
the input is characterized by a refresh rate of 25 Hz; and
the interval is selected from a range between 40 ms and 400 ms inclusive.
23. An apparatus for managing a connection in a neuron network based on at least one signal from a neuron, the apparatus comprising:
a memory; and
at least one processor coupled to the memory, the at least one processor configured:
to receive at least one input via the connection;
to send the at least one signal at a time proximate to the received at least one input;
to demote the connection when the at least one signal is sent prior to the receiving of the at least one input; and
to promote the connection when the at least one signal is sent after the receiving of the input.
24. An apparatus for managing a connection in a neuron network based on at least one signal from a neuron, the apparatus comprising:
means for receiving at least one input via the connection;
means for sending the at least one signal at a time proximate to the received at least one input;
means for demoting the connection when the at least one signal is sent prior to the receiving of the at least one input; and
means for promoting the connection when the at least one signal is sent after the receiving of the input.