Centralized-distributed mixed organization of shared memory for neural network processing

1. A processor, comprising:

a memory architecture having M-number of processing elements each having at least N-number of processing units and a local memory, wherein M, N comprising integers greater than 1 and the memory architecture comprising:

a first processing element of the M-number of processing elements comprising:

a first set of N-number of processing units configured to perform a computing operation, and

a first local memory configured to store data capable of being utilized by the N-number of processing units, wherein first local memory has a full write port, a full read port, a narrow write port, and a narrow read port, wherein the first local memory is configured to acquire private data via the full write port and to send private data via the full read port, and wherein the first local memory is configured to acquire shared data via the narrow write port and send shared data via the narrow read port; and

a data hub capable of receiving data from the M-number of processing elements and capable of providing shared data from one of the M-number of processing elements to other processing elements of the M-number of processing elements.

2. The processor of claim 1 , wherein the first processing element comprises:

a first multiplexer configured to receive shared data from the data hub and to provide the shared data to the first local memory.

3. The processor of claim 2 , wherein the data hub is configured to provide the shared data to a second multiplexer that is configured to provide the share data to one or more of the first set of N-number of processing units.

4. The processor of claim 1 , wherein the first local memory further comprises:

a plurality of local memory storage blocks that includes a first set of one or more storage blocks configured to store private data to be utilized by at least one of the first set of N-number of processing units and a second set of one or more storage blocks configured to store shared data received from the data hub.

5. The processor of claim 1 , wherein the N-number is identical to the M-number.

6. The processor of claim 1 , wherein the data hub is configured to distribute shared data using a distribution pattern.

7. The processor of claim 6 , wherein the distribution pattern includes a broadcasting pattern, an XORing pattern, a shifting pattern, or a rotating pattern.

8. The processor of claim 1 , wherein the data hub is configured to send shared data to the first of N-number of processing units.

9. A processor, comprising:

a memory architecture having M-number of processing elements each having at least N-number of processing units and a local memory, wherein M, N comprising integers greater than 1 and the memory architecture comprising:

a first processing element of the M-number of processing elements comprising:

a first set of N-number of processing units configured to perform a computing operation, and

a first local memory configured to store data capable of being utilized by the N-number of processing units; and

a data hub capable of receiving data from the M-number of processing elements and capable of providing shared data from one of the M-number of processing elements to other processing elements of the M-number of processing elements,

wherein the first local memory has a full write port, a full read port, a narrow write port, and a narrow read port,

wherein the first local memory comprises a plurality of local memory storage blocks that includes a first set of one or more storage blocks configured to store private data to be utilized by at least one of the first set of N-number of processing units and a second set of one or more storage blocks configured to store shared data received from the data hub, and

wherein the first set of one or more storage blocks are configured to acquire private data via the full write port and to send private data via the full read port and the second set of one or more storage blocks are configured to acquire shared data via the narrow write port and send shared data via the narrow read port.

10. The processor of claim 9 , wherein the data hub is configured to receive shared data from the first local memory via the narrow read port.

11. The processor of claim 9 , wherein the data hub is configured to send shared data to the first set of one or more storage blocks via the full write port.

12. The processor of claim 9 , wherein the data hub is configured to send shared data to the second set of one or more storage blocks via the narrow write port.

13. A method performed by a centralized data hub capable of communicating with M-number of processing elements each having at least N-number of processing units and a local memory, the method comprising:

acquiring data from a processing element of M-number of processing elements;

determining a distribution pattern for the acquired data; and

distributing the acquired data to at least one or more of the M-number of processing elements using the distribution pattern,

wherein the local memory has a full write port, a full read port, a narrow write port, and a narrow read port, wherein the local memory is configured to acquire private data via the full write port and to send private data via the full read port, and wherein the local memory is configured to acquire shared data via the narrow write port and send shared data via the narrow read port.

14. The method of claim 13 , wherein the distribution pattern includes a broadcasting pattern, an XORing pattern, a shifting pattern, or a rotating pattern.

15. The method of claim 14 , wherein the distribution pattern includes an offset indicating how to distribute acquired data using the distribution pattern.

16. The method of claim 13 , wherein determining the distribution pattern comprises:

receiving a distribution indication; and

determining the distribution pattern based on the distribution indication.

17. A method performed by a processing element of multiple processing elements coupled to a centralized data hub, the method comprising:

acquiring shared data from the centralized data hub;

acquiring private data local to the processing element;

storing the private data in a local memory of the processing element, wherein the local memory has a full write port, a full read port, a narrow write port, and a narrow read port, wherein the local memory is configured to acquire private data via the full write port and to send private data via the full read port, and wherein the local memory is configured to acquire shared data via the narrow write port and send shared data via the narrow read port;

providing the shared data and the stored private data as operands to one or more processing units of the processing element; and

generating one or more results from one or more operations performed by corresponding one or more processing units.

18. The method of claim 17 , further comprising storing the acquired shared data in the local memory.

19. The method of claim 18 , wherein storing the acquired shared data in the local memory comprises storing the shared data in a first set of one or more blocks in the local memory, and

wherein storing the private data in the local memory comprises storing the private data in a second set of one or more blocks in the local memory.

20. The method of claim 17 , wherein the acquired shared data is directly provided to the one or more processing units of the processing element without being stored in the local memory.

21. The method of claim 17 , wherein the private data that is acquired corresponds to data generated by the one or more processing units of the processing element.

22. The method of claim 17 , further comprising storing the generated one or more results in the local memory.