Elementary network description for efficient memory management in neuromorphic systems
A simple format is disclosed and referred to as Elementary Network Description (END). The format can fully describe a large-scale neuronal model and embodiments of software or hardware engines to simulate such a model efficiently. The architecture of such neuromorphic engines is optimal for high-performance parallel processing of spiking networks with spike-timing dependent plasticity. Methods for managing memory in a processing system are described whereby memory can be allocated among a plurality of elements and rules configured for each element such that the parallel execution of the spiking networks is most optimal.
1. A method for managing memory in a processing system, comprising:
allocating memory among a plurality of elements;
configuring rules for individual elements, wherein at least a portion of the rules associated with a first element type is configured to control updates to memory allocated to a second element type; and
providing a system clock defining a step interval during which a part of the rules is executed;
wherein:
execution of the part of the rules for two or more elements of a given type is order-independent during the step interval, the given type comprises one of the first element type or the second element type;
the plurality of elements are interconnected according to a graph representative of a neuronal network;
memory of a given element having allocated memory is updated consistent with a rule configured for the given element;
the plurality of elements comprises:
a plurality of units, individual units having an allocated memory; and
a plurality of doublets, individual doublets associated with a presynaptic unit and a postsynaptic unit; and
an individual doublet is operable to modify the memory of the postsynaptic unit.
2. The method of claim 1 , wherein the individual doublet modifies the memory of the postsynaptic unit in response to an event received from the presynaptic unit.
3. The method of claim 2 , wherein the event is generated by the presynaptic unit based on a condition associated with the presynaptic unit.
4. The method of claim 3 , wherein the condition of the unit comprises a value stored in the unit memory.
5. The method of claim 4 , wherein: the memory updates of the plurality of doublets include delayed updates.
6. The method of claim 2 , wherein reception of the event triggers a doublet event rule.
7. The method of claim 1 , wherein the order of execution of triggered doublet event rules is arbitrary during a given step interval.
8. The method of claim 1 , wherein the order of execution of the rules of the second portion of elements is arbitrary during the step interval.
9. The method of claim 1 , wherein a given doublet is configured to update the memory of its corresponding postsynaptic unit by atomic addition.
10. The method of claim 1 , wherein the plurality of elements comprises triplets configured to access the memory allocated to a pair of units and further configured to update the memory allocated to at least one of the units.
11. The method of claim 10 , wherein:
the order of execution of rules associated with the triplets is arbitrary during a given step interval, and
a given triplet is configured to update the memory of a connected unit by atomic addition.
12. A parallel processing system, comprising:
a plurality of elements, a given element having a memory configured to be updated according to a rule configured for the given element; and
a system clock defining a series of step intervals in which updates to memories of a first set of the plurality of elements are initiated and in which updates to memories of a second set of the plurality of elements are completed;
wherein:
the plurality of elements includes units and doublets, individual unit operable to modify its own memory, and individual doublet operable to modify its own memory and the memory of a unit;
execution of the rules for the doublets is order-independent; and
a given doublet is configured to receive one or more events from a presynaptic unit and to modify the memory of a postsynaptic unit.
13. The system of claim 12 , wherein the rules for the units are executed during a given step, and wherein execution of the rules for the units is order-independent.
14. The system of claim 13 , wherein the rule for a given unit is configured to generate an event based on a condition of the given unit.
15. The system of claim 14 , wherein the event is communicated to a given doublet.
16. The system of claim 14 , wherein the condition of the given unit comprises a value stored in the unit memory.
17. The system of claim 14 , wherein the updates to the memories of the second set of the elements include delayed updates.
18. The system of claim 12 , wherein the events trigger the rule of the given doublet.
19. The system of claim 18 , wherein the rules of at least one doublet is triggered after a delay configured for the least one doublet.
20. The system of claim 12 , wherein individual doublet is configured to update its corresponding postsynaptic unit by atomic addition.
21. The system of claim 12 , wherein:
the first set of the plurality of elements comprises the doublets; and
the second set of the plurality of elements comprises the units.
22. The system of claim 12 , wherein:
the first set of the plurality of elements comprises the units; and
the second set of the plurality of elements comprises the doublets.
23. A parallel processing system, comprising:
a plurality of elements, a given element having a memory configured to be updated according to a rule configured for the given element; and
a system clock defining a series of step intervals in which updates to memories of a first set of the elements are initiated and in which updates to memories of a second set of the elements are completed;
wherein:
the plurality of elements includes units and doublets, individual unit operable to modify its own memory, and individual doublet operable to modify its own memory and the memory of a unit; and
the plurality of elements includes at least one triplet operable to access memory of a pair of units and its own memory and to modify memory of at least one of the pair of units.
24. The system of claim 23 , wherein the plurality of elements includes at least one triplet operable to modify its own memory and the memory of two units.
25. The system of claim 24 , wherein execution of the rules of the at least one triplet is order-independent within a given step.
26. The system of claim 24 , wherein the triplet updates the memory of the two units by atomic addition.
27. A parallel processing system, comprising:
a plurality of elements, a given element having a memory configured to be updated according to a rule configured for the given element; and
a system clock defining a series of step intervals in which updates to memories of a first set of the elements are initiated and in which updates to memories of a second set of the elements are completed;
wherein:
the plurality of elements includes units and doublets, individual unit operable to modify its own memory, and individual doublet operable to modify its own memory and the memory of a unit; and
the updates to the memories of the second set of the elements include delayed updates.
28. The system of claim 27 , wherein execution of the rules for the doublets is order-independent.
29. The system of claim 28 , wherein the elements are interconnected according to a directed graph representative of a neuronal network.