Configurable and programmable sliding window based memory access in a neural network processor
A novel and useful neural network (NN) processing core adapted to implement artificial neural networks (ANNs) and incorporating configurable and programmable sliding window based memory access. The memory mapping and allocation scheme trades off random and full access in favor of high parallelism and static mapping to a subset of the overall address space. The NN processor is constructed from self-contained computational units organized in a hierarchical architecture. The homogeneity enables simpler management and control of similar computational units, aggregated in multiple levels of hierarchy. Computational units are designed with minimal overhead as possible, where additional features and capabilities are aggregated at higher levels in the hierarchy. On-chip memory provides storage for content inherently required for basic operation at a particular hierarchy and is coupled with the computational resources in an optimal ratio. Lean control provides just enough signaling to manage only the operations required at a particular hierarchical level. Dynamic resource assignment agility is provided which can be adjusted as required depending on resource availability and capacity of the device.
1. A method of connecting first resource elements with second resource elements in an integrated circuit (IC), the IC including a neural network (NN) processor circuit for performing neural network calculations for an artificial neural network (ANN) having one or more network layers, the method comprising:
establishing a plurality of access windows between said first resource elements and said second resource elements by, for each window:
limiting access of a first number of said first resource elements solely to a second number of said second resource elements;
limiting access of a third number of said second resource elements solely to a fourth number of said first resource elements; and
configuring said first number, said second number, said third number, and said fourth number such that said plurality of access windows overlap each other to form sliding, bounded access windows.
2. The method according to claim 1 , wherein said overlapping sliding access windows do not allow random access between said first resource elements and said second resource elements thereby significantly increasing access bandwidth within said NN processor circuit by reducing required control.
3. The method according to claim 1 , further comprising utilizing a scatter/gather technique to generate said overlapping sliding access windows.
4. The method according to claim 1 , wherein said first resource elements and said second resource elements comprise at least one of control elements, compute elements, and memory elements.
5. The method according to claim 1 , wherein each access window comprises a size and specific connectivity that are dynamically configurable and not fixed.
6. The method according to claim 1 , further comprising resolving contention arising from simultaneous access within an access window of a first resource element and a second resource element.
7. A method of windowing between compute elements and memory elements in an integrated circuit (IC), the IC including a neural network (NN) processor circuit for performing neural network calculations for an artificial neural network (ANN) having one or more network layers, the method comprising:
establishing a plurality of access windows between said compute elements and said memory elements by, for each window:
limiting access of each compute element solely to a first number of memory elements;
limiting access of each memory element solely to a second number of compute elements;
configuring said first number and said second number such that said plurality of access windows overlap each other to form sliding, bounded access windows thereby enabling memory sharing and pipelining in said NN processor circuit.
8. The method according to claim 7 , wherein said overlapping sliding access windows do not allow full mesh random access between said compute elements and said memory elements thereby allowing simultaneous access by substantially all compute elements to their corresponding memory elements resulting in a significant increase in memory access bandwidth within said NN processor circuit.
9. The method according to claim 7 , wherein access by a compute element is limited to only those memory elements required for that compute element to perform its function.
10. The method according to claim 7 , further comprising utilizing a scatter/gather technique to generate said overlapping sliding access windows.
11. The method according to claim 10 , wherein said scatter/gather technique comprises providing a plurality of splitters and/or demultiplexers, and a plurality of multiplexers connected and configured to generate said sliding, bounded access windows.
12. The method according to claim 7 , wherein each access window comprises a size and specific connectivity that are dynamically configurable and not fixed.
13. The method according to claim 7 , wherein said first number of memory elements and said second number of compute elements accessible within each sliding window is dynamically configurable and programmable.
14. The method according to claim 7 , further comprising providing sliding windows for at least one of control, ingress data, egress data, weights, intermediate results, activation function, processor, subcluster, cluster, processing units, compute elements, and memory elements.
15. The method according to claim 7 , further comprising resolving contention arising from simultaneous access to a memory element by more than one compute element.
16. The method according to claim 15 , wherein resolving contention comprises inhibiting one compute element from accessing its corresponding memory element until access by the other compute element is complete.
17. An apparatus for resource windowing in a neural network (NN) processor circuit for performing neural network calculations for an artificial neural network (ANN) having one or more network layers, comprising:
a plurality of compute elements;
a plurality of memory elements;
a first circuit coupled to said plurality of compute elements and said plurality of memory elements, said first circuit operative to establish a plurality of access windows between said compute elements and said memory elements by:
limiting, for each window, access of each compute element solely to a first number of memory elements;
limiting, for each access window, access of each memory element solely to a second number of compute elements;
a second circuit operative to configure said first number and said second number such that said plurality of access windows overlap each other to form sliding, bounded access windows thereby enabling memory sharing and pipelining in said NN processor circuit.
18. The apparatus according to claim 17 , wherein said first circuit and said second circuit do not permit full mesh random access between said compute elements and said memory elements thereby allowing simultaneous access by substantially all compute elements to their corresponding memory elements resulting in a significant increase in memory access bandwidth within said NN processor circuit.
19. The apparatus according to claim 17 , wherein said first circuit and said second circuit are operative to limit access by a compute element to only those memory elements required for that compute element to perform its function.
20. The apparatus according to claim 17 , wherein said first circuit and said second circuit comprises scatter/gather based circuitry to generate said overlapping sliding access windows.
21. The apparatus according to claim 17 , wherein said first number of memory elements and said second number of compute elements accessible within each sliding window is dynamically configurable and programmable.
22. The apparatus according to claim 17 , wherein each access window comprises a size and specific connectivity that are dynamically configurable and not fixed.
23. The apparatus according to claim 17 , wherein said first circuit and said second circuit is operative to provide sliding windows for at least one of control, ingress data, egress data, weights, intermediate results, activation function, processor, subcluster, cluster, processing units, compute elements, and memory elements.
24. The apparatus according to claim 17 , further comprising a third circuit operative to resolve contention arising from simultaneous access to a memory element by more than one compute element.
25. The apparatus according to claim 24 , wherein said third circuit is operative to resolve contention by inhibiting one compute element from accessing its corresponding memory element until access by the other compute element is complete.