Methods and apparatus for spiking neural computation
Certain aspects of the present disclosure provide methods and apparatus for spiking neural computation of general linear systems. One example aspect is a neuron model that codes information in the relative timing between spikes. However, synaptic weights are unnecessary. In other words, a connection may either exist (significant synapse) or not (insignificant or non-existent synapse). Certain aspects of the present disclosure use binary-valued inputs and outputs and do not require post-synaptic filtering. However, certain aspects may involve modeling of connection delays (e.g., dendritic delays). A single neuron model may be used to compute any general linear transformation x=AX+BU to any arbitrary precision. This neuron model may also be capable of learning, such as learning input delays (e.g., corresponding to scaling values) to achieve a target output delay (or output value). Learning may also be used to determine a logical relation of causal inputs.
1. A method for implementing a spiking neural network, comprising:
receiving at least one input value at a first artificial neuron;
based on the input value, determining a relative time between a first output spike time of the first artificial neuron and a reference time, wherein determining the relative time comprises encoding the input value as the relative time, wherein the encoding comprises calculating the relative time as a negative of a logarithm of the input value; and
emitting an output spike from the first artificial neuron based on the relative time.
2. The method of claim 1 , wherein the logarithm has a base equal to an exponential value of a coefficient of change of a membrane potential as a function of the membrane potential for the first artificial neuron.
3. The method of claim 1 , further comprising:
determining a delay in the at least one input based on a function modeled by the first artificial neuron; and
adjusting the relative time based on the delay such that the emitting comprises emitting the output spike based on the adjusted relative time.
4. The method of claim 3 , wherein the function comprises a linear transformation.
5. The method of claim 3 , wherein the function comprises multiplication by a scalar, wherein determining the delay comprises computing an absolute value of the scalar to determine the delay, wherein a synapse associated with the input to the first artificial neuron is used as an inhibitory synapse if the scalar is negative, and wherein the synapse associated with the input to the first artificial neuron is used as an excitatory synapse if the scalar is positive.
6. The method of claim 3 , wherein determining the delay comprises quantizing the delay to a desired temporal resolution and wherein adjusting the relative time comprises adjusting the relative time based on the quantized delay.
7. The method of claim 6 , wherein a precision of the function depends on the temporal resolution.
8. The method of claim 3 , further comprising:
determining an output value for the first artificial neuron based on a time difference between a time of the emitted output spike and the reference time, wherein the output value is an inverse of an exponential value of a coefficient of change of a membrane potential for the first artificial neuron, the exponential raised to the power of the time difference before taking the inverse; and
outputting the output value to a display.
9. The method of claim 3 , wherein the function is a learning function based on a homeostatic process or a target output delay.
10. The method of claim 1 , wherein the reference time comprises a second output spike time of the first artificial neuron, the second output spike time occurring before the first output spike time.
11. The method of claim 1 , wherein the reference time comprises a second output spike time of a second artificial neuron, wherein an output of the first artificial neuron is coupled to an input of the second artificial neuron, and wherein the second output spike time occurs before the first output spike time.
12. The method of claim 11 , wherein the first artificial neuron has a first coefficient of change of a first membrane potential for the first artificial neuron and wherein the second artificial neuron has a second coefficient of change of a second membrane potential for the second artificial neuron different than the first coefficient of change.
13. The method of claim 11 , wherein the second artificial neuron uses another reference time that is delayed from the reference time for the first artificial neuron.
14. The method of claim 1 , wherein the first artificial neuron comprises an anti-leaky-integrate-and-fire (ALIF) artificial neuron.
15. The method of claim 1 , wherein the first artificial neuron has an exponentially growing membrane potential and continues to depolarize in the absence of an inhibitory input and wherein an excitatory input causes the first artificial neuron to fire sooner than the first artificial neuron would fire without the excitatory input.
16. The method of claim 1 , further comprising:
resetting a membrane potential of the first artificial neuron to a nominal setting above zero after emitting the output spike.
17. The method of claim 1 , wherein receiving the input comprises sampling the input with a sampling rate based on a desired temporal resolution and wherein determining the relative time comprises quantizing the relative time to the temporal resolution.
18. An apparatus for implementing a spiking neural network, comprising:
a processing unit configured to receive at least one input value at a first artificial neuron;
determine, based on the input value, a relative time between a first output spike time of the first artificial neuron and a reference time, wherein the processing unit is configured to determine the relative time by encoding the input value as the relative time and wherein the encoding comprises calculating the relative time as a negative of a logarithm of the input value; and
emit an output spike from the first artificial neuron based on the relative time.
19. The apparatus of claim 18 , wherein the logarithm has a base equal to an exponential value of a coefficient of change of a membrane potential as a function of the membrane potential for the first artificial neuron.
20. The apparatus of claim 18 , wherein the processing unit is further configured to:
determine a delay in the at least one input based on a function modeled by the first artificial neuron; and
adjust the relative time based on the delay such that the output spike is emitted based on the adjusted relative time.
21. The apparatus of claim 20 , wherein the function comprises a linear transformation.
22. The apparatus of claim 20 , wherein the function comprises multiplication by a scalar, wherein the processing unit is configured to determine the delay by computing an absolute value of the scalar to determine the delay, wherein a synapse associated with the input to the first artificial neuron is used as an inhibitory synapse if the scalar is negative, and wherein the synapse associated with the input to the first artificial neuron is used as an excitatory synapse if the scalar is positive.
23. The apparatus of claim 20 , wherein the processing unit is configured to determine the delay by quantizing the delay to a desired temporal resolution and wherein the processing unit is configured to adjust the relative time by adjusting the relative time based on the quantized delay.
24. The apparatus of claim 23 , wherein a precision of the function depends on the temporal resolution.
25. The apparatus of claim 20 , wherein the processing unit is further configured to:
determine an output value for the first artificial neuron based on a time difference between a time of the emitted output spike and the reference time, wherein the output value is an inverse of an exponential value of a coefficient of change of a membrane potential for the first artificial neuron, the exponential raised to the power of the time difference before taking the inverse; and
output the output value to a display.
26. The apparatus of claim 20 , wherein the function is a learning function based on a homeostatic process or a target output delay.
27. The apparatus of claim 18 , wherein the reference time comprises a second output spike time of the first artificial neuron, the second output spike time occurring before the first output spike time.
28. The apparatus of claim 18 , wherein the reference time comprises a second output spike time of a second artificial neuron, wherein an output of the first artificial neuron is coupled to an input of the second artificial neuron, and wherein the second output spike time occurs before the first output spike time.
29. The apparatus of claim 28 , wherein the first artificial neuron has a first coefficient of change of a first membrane potential for the first artificial neuron and wherein the second artificial neuron has a second coefficient of change of a second membrane potential for the second artificial neuron different than the first coefficient of change.
30. The apparatus of claim 28 , wherein the second artificial neuron uses another reference time that is delayed from the reference time for the first artificial neuron.
31. The apparatus of claim 18 , wherein the first artificial neuron comprises an anti-leaky-integrate-and-fire (ALIF) artificial neuron.
32. The apparatus of claim 18 , wherein the first artificial neuron has an exponentially growing membrane potential and continues to depolarize in the absence of an inhibitory input and wherein an excitatory input causes the first artificial neuron to fire sooner than the first artificial neuron would fire without the excitatory input.
33. The apparatus of claim 18 , wherein the processing unit is further configured to:
reset a membrane potential of the first artificial neuron to a nominal setting above zero after the output spike is emitted.
34. The apparatus of claim 18 , wherein the processing unit is configured to receive the input by sampling the input with a sampling rate based on a desired temporal resolution and wherein the processing unit is configured to determine the relative time by quantizing the relative time to the temporal resolution.
35. An apparatus for implementing a spiking neural network, comprising:
means for receiving at least one input value at a first artificial neuron;
means for determining, based on the input value, a relative time between a first output spike time of the first artificial neuron and a reference time, wherein the means for determining the relative time is configured to encode the input value as the relative time and wherein encoding the input value comprises calculating the relative time as a negative of a logarithm of the input value; and
means for emitting an output spike from the first artificial neuron based on the relative time.
36. The apparatus of claim 35 , wherein the logarithm has a base equal to an exponential value of a coefficient of change of a membrane potential as a function of the membrane potential for the first artificial neuron.
37. The apparatus of claim 35 , further comprising:
means for determining a delay in the at least one input based on a function modeled by the first artificial neuron; and
means for adjusting the relative time based on the delay such that the means for emitting is configured to emit the output spike based on the adjusted relative time.
38. The apparatus of claim 37 , wherein the function comprises a linear transformation.
39. The apparatus of claim 37 , wherein the function comprises multiplication by a scalar, wherein the means for determining the delay is configured to compute an absolute value of the scalar to determine the delay, wherein a synapse associated with the input to the first artificial neuron is used as an inhibitory synapse if the scalar is negative, and wherein the synapse associated with the input to the first artificial neuron is used as an excitatory synapse if the scalar is positive.
40. The apparatus of claim 37 , wherein the means for determining the delay is configured to quantize the delay to a desired temporal resolution and wherein the means for adjusting the relative time is configured to adjust the relative time based on the quantized delay.
41. The apparatus of claim 40 , wherein a precision of the function depends on the temporal resolution.
42. The apparatus of claim 37 , further comprising:
means for determining an output value for the first artificial neuron based on a time difference between a time of the emitted output spike and the reference time, wherein the output value is an inverse of an exponential value of a coefficient of change of a membrane potential for the first artificial neuron, the exponential raised to the power of the time difference before taking the inverse; and
means for outputting the output value to a means for indicating the output value.
43. The apparatus of claim 37 , wherein the function is a learning function based on a homeostatic process or a target output delay.
44. The apparatus of claim 35 , wherein the reference time comprises a second output spike time of the first artificial neuron, the second output spike time occurring before the first output spike time.
45. The apparatus of claim 35 , wherein the reference time comprises a second output spike time of a second artificial neuron, wherein an output of the first artificial neuron is coupled to an input of the second artificial neuron, and wherein the second output spike time occurs before the first output spike time.
46. The apparatus of claim 45 , wherein the first artificial neuron has a first coefficient of change of a first membrane potential for the first artificial neuron and wherein the second artificial neuron has a second coefficient of change of a second membrane potential for the second artificial neuron different than the first coefficient of change.
47. The apparatus of claim 45 , wherein the second artificial neuron uses another reference time that is delayed from the reference time for the first artificial neuron.
48. The apparatus of claim 35 , wherein the first artificial neuron comprises an anti-leaky-integrate-and-fire (ALIF) artificial neuron.
49. The apparatus of claim 35 , wherein the first artificial neuron has an exponentially growing membrane potential and continues to depolarize in the absence of an inhibitory input and wherein an excitatory input causes the first artificial neuron to fire sooner than the first artificial neuron would fire without the excitatory input.
50. The apparatus of claim 35 , further comprising:
means for resetting a membrane potential of the first artificial neuron to a nominal setting above zero after the output spike is emitted.
51. The apparatus of claim 35 , wherein the means for receiving the input is configured to sample the input with a sampling rate based on a desired temporal resolution and wherein the means for determining the relative time is configured to quantize the relative time to the temporal resolution.
52. A computer-program product for implementing a spiking neural network, comprising a non-transitory computer-readable medium comprising instructions executable to:
receive at least one input value at a first artificial neuron;
determine, based on the input value, a relative time between a first output spike time of the first artificial neuron and a reference time, wherein determining the relative time comprises encoding the input value as the relative time and wherein the encoding comprises calculating the relative time as a negative of a logarithm of the input value; and
emit an output spike from the first artificial neuron based on the relative time.
53. The computer-program product of claim 52 , wherein the logarithm has a base equal to an exponential value of a coefficient of change of a membrane potential as a function of the membrane potential for the first artificial neuron.
54. The computer-program product of claim 52 , further comprising instructions executable to:
determine a delay in the at least one input based on a function modeled by the first artificial neuron; and
adjust the relative time based on the delay such that the emitting comprises emitting the output spike based on the adjusted relative time.
55. The computer-program product of claim 54 , wherein the function comprises a linear transformation.
56. The computer-program product of claim 54 , wherein the function comprises multiplication by a scalar, wherein determining the delay comprises computing an absolute value of the scalar to determine the delay, wherein a synapse associated with the input to the first artificial neuron is used as an inhibitory synapse if the scalar is negative, and wherein the synapse associated with the input to the first artificial neuron is used as an excitatory synapse if the scalar is positive.
57. The computer-program product of claim 54 , wherein determining the delay comprises quantizing the delay to a desired temporal resolution and wherein adjusting the relative time comprises adjusting the relative time based on the quantized delay.
58. The computer-program product of claim 57 , wherein a precision of the function depends on the temporal resolution.
59. The computer-program product of claim 54 , further comprising instructions executable to:
determine an output value for the first artificial neuron based on a time difference between a time of the emitted output spike and the reference time, wherein the output value is an inverse of an exponential value of a coefficient of change of a membrane potential for the first artificial neuron, the exponential raised to the power of the time difference before taking the inverse; and
output the output value to a display.
60. The computer-program product of claim 54 , wherein the function is a learning function based on a homeostatic process or a target output delay.
61. The computer-program product of claim 52 , wherein the reference time comprises a second output spike time of the first artificial neuron, the second output spike time occurring before the first output spike time.
62. The computer-program product of claim 52 , wherein the reference time comprises a second output spike time of a second artificial neuron, wherein an output of the first artificial neuron is coupled to an input of the second artificial neuron, and wherein the second output spike time occurs before the first output spike time.
63. The computer-program product of claim 62 , wherein the first artificial neuron has a first coefficient of change of a first membrane potential for the first artificial neuron and wherein the second artificial neuron has a second coefficient of change of a second membrane potential for the second artificial neuron different than the first coefficient of change.
64. The computer-program product of claim 62 , wherein the second artificial neuron uses another reference time that is delayed from the reference time for the first artificial neuron.
65. The computer-program product of claim 52 , wherein the first artificial neuron comprises an anti-leaky-integrate-and-fire (ALIF) artificial neuron.
66. The computer-program product of claim 52 , wherein the first artificial neuron has an exponentially growing membrane potential and continues to depolarize in the absence of an inhibitory input and wherein an excitatory input causes the first artificial neuron to fire sooner than the first artificial neuron would fire without the excitatory input.
67. The computer-program product of claim 52 , further comprising instructions executable to:
reset a membrane potential of the first artificial neuron to a nominal setting above zero after emitting the output spike.
68. The computer-program product of claim 52 , wherein receiving the input comprises sampling the input with a sampling rate based on a desired temporal resolution and wherein determining the relative time comprises quantizing the relative time to the temporal resolution.