Scalable neural hardware for the noisy-OR model of Bayesian networks
Embodiments of the invention relate to a scalable neural hardware for the noisy-OR model of Bayesian networks. One embodiment comprises a neural core circuit including a pseudo-random number generator for generating random numbers. The neural core circuit further comprises a plurality of incoming electronic axons, a plurality of neural modules, and a plurality of electronic synapses interconnecting the axons to the neural modules. Each synapse interconnects an axon with a neural module. Each neural module receives incoming spikes from interconnected axons. Each neural module represents a noisy-OR gate. Each neural module spikes probabilistically based on at least one random number generated by the pseudo-random number generator unit.
1. A method comprising:
at an electronic neuron of a neural core circuit:
receiving at least one incoming firing event from at least one electronic axon connected to the electronic neuron via at least one electronic synapse;
maintaining, on a memory device of the electronic neuron, at least one programmable probability value, wherein each programmable probability value represents a probability that the electronic neuron integrates an incoming firing event received; and
modeling the electronic neuron as a noisy-OR gate configured to perform one or more Bayesian computations by:
probabilistically integrating, via a computation circuit of the electronic neuron, the at least one incoming firing event received based on the at least one programmable probability value; and
probabilistically generating, via the computation circuit of the electronic neuron, an outgoing firing event based on a number of incoming firing events integrated and the at least one programmable probability value.
2. The method of claim 1 , wherein probabilistically integrating the at least one incoming firing event received comprises:
for each incoming firing event received from each electronic axon:
determining whether an electronic synapse connecting the electronic axon to the electronic neuron is a conducting synapse; and
integrating the incoming firing event in response to determining the electronic synapse is a conducting synapse.
3. The method of claim 2 , wherein probabilistically generating an outgoing firing event comprises:
determining a spiking probability of the electronic neuron spiking based on an exponential function, the number of incoming firing events integrated, and the at least one programmable probability value;
drawing a random number from a pseudo-random number generator of the electronic neuron;
determining whether a condition is satisfied by comparing the random number drawn against the spiking probability determined; and
generating an outgoing firing event in response to determining that the condition is satisfied.
4. The method of claim 3 , wherein the condition is satisfied if the random number drawn is less than the spiking probability determined, and the one or more Bayesian computations comprise at least one of statistical interference, recognizing patterns and classifying inputs.
5. The method of claim 1 , wherein probabilistically integrating the at least one incoming firing event received comprises:
for each incoming firing event received from each electronic axon:
determining whether an electronic synapse connecting the electronic axon to the electronic neuron is a conducting synapse; and
in response to determining the electronic synapse is a conducting synapse:
drawing a random number from a pseudo-random number generator of the electronic neuron;
determining whether a condition is satisfied by comparing the random number drawn against the at least one programmable probability value;
integrating the incoming firing event in response to determining that the condition is satisfied; and
generating an outgoing firing event in response to integrating the incoming firing event.
6. The method of claim 5 , wherein the condition is satisfied if the random number drawn is less than the at least one programmable probability value, and the one or more Bayesian computations comprise at least one of statistical interference, recognizing patterns and classifying inputs.
7. A system comprising a computer processor, a computer-readable hardware storage medium, and program code embodied with the computer-readable hardware storage medium for execution by the computer processor to implement a method comprising:
at an electronic neuron of a neural core circuit:
receiving at least one incoming firing event from at least one electronic axon connected to the electronic neuron via at least one electronic synapse;
maintaining, on a memory device of the electronic neuron, at least one programmable probability value, wherein each programmable probability value represents a probability that the electronic neuron integrates an incoming firing event received; and
modeling the electronic neuron as a noisy-OR gate configured to perform one or more Bayesian computations by:
probabilistically integrating, via a computation circuit of the electronic neuron, the at least one incoming firing event received based on the at least one programmable probability value; and
probabilistically generating, via the computation circuit of the electronic neuron, an outgoing firing event based on a number of incoming firing events integrated and the at least one programmable probability value.
8. The system of claim 7 , wherein probabilistically integrating the at least one incoming firing event received comprises:
for each incoming firing event received from each electronic axon:
determining whether an electronic synapse connecting the electronic axon to the electronic neuron is a conducting synapse; and
integrating the incoming firing event in response to determining the electronic synapse is a conducting synapse.
9. The system of claim 8 , wherein probabilistically generating an outgoing firing event comprises:
determining a spiking probability of the electronic neuron spiking based on an exponential function, the number of incoming firing events integrated, and the at least one programmable probability value;
drawing a random number from a pseudo-random number generator of the electronic neuron;
determining whether a condition is satisfied by comparing the random number drawn against the spiking probability determined; and
generating an outgoing firing event in response to determining that the condition is satisfied.
10. The system of claim 9 , wherein the condition is satisfied if the random number drawn is less than the spiking probability determined, and the one or more Bayesian computations comprise at least one of statistical interference, recognizing patterns and classifying inputs.
11. The system of claim 7 , wherein probabilistically integrating the at least one incoming firing event received comprises:
for each incoming firing event received from each electronic axon:
determining whether an electronic synapse connecting the electronic axon to the electronic neuron is a conducting synapse; and
in response to determining the electronic synapse is a conducting synapse:
drawing a random number from a pseudo-random number generator of the electronic neuron;
determining whether a condition is satisfied by comparing the random number drawn against the at least one programmable probability value;
integrating the incoming firing event in response to determining that the condition is satisfied; and
generating an outgoing firing event in response to integrating the incoming firing event.
12. The system of claim 11 , wherein the condition is satisfied if the random number drawn is less than the at least one programmable probability value, and the one or more Bayesian computations comprise at least one of statistical interference, recognizing patterns and classifying inputs.
13. A computer program product comprising a computer-readable hardware storage medium having program code embodied therewith, the program code being executable by a computer to implement a method comprising:
at an electronic neuron of a neural core circuit:
receiving at least one incoming firing event from at least one electronic axon connected to the electronic neuron via at least one electronic synapse;
maintaining, on a memory device of the electronic neuron, at least one programmable probability value, wherein each programmable probability value represents a probability that the electronic neuron integrates an incoming firing event received; and
modeling the electronic neuron as a noisy-OR gate configured to perform one or more Bayesian computations by:
probabilistically integrating, via a computation circuit of the electronic neuron, the at least one incoming firing event received based on the at least one programmable probability value; and
probabilistically generating, via the computation circuit of the electronic neuron, an outgoing firing event based on a number of incoming firing events integrated and the at least one programmable probability value.
14. The computer program product of claim 13 , wherein probabilistically integrating the at least one incoming firing event received comprises:
for each incoming firing event received from each electronic axon:
determining whether an electronic synapse connecting the electronic axon to the electronic neuron is a conducting synapse; and
integrating the incoming firing event in response to determining the electronic synapse is a conducting synapse.
15. The computer program product of claim 14 , wherein probabilistically generating an outgoing firing event comprises:
determining a spiking probability of the electronic neuron spiking based on an exponential function, the number of incoming firing events integrated, and the at least one programmable probability value;
drawing a random number from a pseudo-random number generator of the electronic neuron;
determining whether a condition is satisfied by comparing the random number drawn against the spiking probability determined; and
generating an outgoing firing event in response to determining that the condition is satisfied.
16. The computer program product of claim 15 , wherein the condition is satisfied if the random number drawn is less than the spiking probability determined, and the one or more Bayesian computations comprise at least one of statistical interference, recognizing patterns and classifying inputs.
17. The computer program product of claim 13 , wherein probabilistically integrating the at least one incoming firing event received comprises:
for each incoming firing event received from each electronic axon:
determining whether an electronic synapse connecting the electronic axon to the electronic neuron is a conducting synapse; and
in response to determining the electronic synapse is a conducting synapse:
drawing a random number from a pseudo-random number generator of the electronic neuron;
determining whether a condition is satisfied by comparing the random number drawn against the at least one programmable probability value;
integrating the incoming firing event in response to determining that the condition is satisfied; and
generating an outgoing firing event in response to integrating the incoming firing event.
18. The computer program product of claim 17 , wherein the condition is satisfied if the random number drawn is less than the at least one programmable probability value, and the one or more Bayesian computations comprise at least one of statistical interference, recognizing patterns and classifying inputs.