Memory-based distributed processor architecture
Distributed processors and methods for compiling code for execution by distributed processors are disclosed. In one implementation, a distributed processor may include a substrate; a memory array disposed on the substrate; and a processing array disposed on the substrate. The memory array may include a plurality of discrete memory banks, and the processing array may include a plurality of processor subunits, each one of the processor subunits being associated with a corresponding, dedicated one of the plurality of discrete memory banks. The distributed processor may further include a first plurality of buses, each connecting one of the plurality of processor subunits to its corresponding, dedicated memory bank, and a second plurality of buses, each connecting one of the plurality of processor subunits to another of the plurality of processor subunits.
1. A distributed processor on a memory chip, comprising:
a semiconductor substrate;
a plurality of processor subunits disposed on the semiconductor substrate, each processor subunit being configured to execute a series of instructions independent from other processor subunits, each series of instructions defining a series of tasks to be performed by a single processor subunit;
a corresponding plurality of memory banks disposed on the semiconductor substrate, each one of the plurality of processor subunits being connected to at least one dedicated memory bank not shared by any others of the plurality of processor subunits; and
a plurality of buses, each of the plurality of buses connecting one of the plurality of processor subunits to at least one other of the plurality of processor subunits,
wherein data transfers across at least one of the plurality of buses are predefined by the series of instructions included in a processor subunit connected to the at least one of the plurality of buses; and
wherein each of the plurality of processor subunits is configured to access data stored in at least its corresponding dedicated one of the plurality of memory banks and perform one or more calculations using at least one value included in the accessed data.
2. The distributed processor on a memory chip of claim 1 , wherein each series of instructions comprises a set of machine code defining a corresponding series of tasks.
3. The distributed processor on a memory chip of claim 2 , wherein the series of tasks are defined by a compiler configured to distribute a higher-level series of tasks amongst the plurality of logic circuits as a plurality of series of tasks.
4. The distributed processor on a memory chip of claim 3 , wherein the higher-level series of tasks comprises a set of instructions in a human-readable programming language.
5. The distributed processor on a memory chip of claim 1 , wherein the series of instructions included in the processor subunit connected to the at least one of the plurality of buses includes a sending task that comprises a command for the processor subunit connected to the at least one of the plurality of buses to write data to the at least one of the plurality of buses.
6. The distributed processor on a memory chip of claim 1 , wherein the series of instructions included in the processor subunit connected to the at least one of the plurality of buses includes a receiving task that comprises a command for the processor subunit connected to the at least one of the plurality of buses to read data from the at least one of the plurality of buses.
7. A distributed processor on a memory chip, comprising:
a semiconductor substrate;
a plurality of processor subunits disposed on the semiconductor substrate of the memory chip;
a plurality of memory banks disposed on the semiconductor substrate of the memory chip, wherein each one of the plurality of memory banks is configured to store data independent from data stored in other ones of the plurality of memory banks, and wherein each one of the plurality of processor subunits is connected to at least one dedicated memory bank from among the plurality of memory banks; and
a plurality of buses, wherein each one of the plurality of buses connects one of the plurality of processor subunits to one or more corresponding, dedicated memory banks from among the plurality of memory banks,
wherein data transfers across a particular one of the plurality of buses are controlled by a corresponding processor subunit connected to the particular one of the plurality of buses; and
wherein each of the plurality of processor subunits is configured to access data stored in at least its corresponding dedicated one of the plurality of memory banks and perform one or more calculations using at least one value included in the accessed data.
8. The distributed processor on a memory chip of claim 7 , wherein the data stored in each of the plurality of memory banks are defined by a compiler configured to distribute data amongst the plurality of memory banks.
9. The distributed processor on a memory chip of claim 8 , wherein the compiler is configured to distribute data defined in a higher-level series of tasks amongst the plurality of memory banks using a plurality of lower-level tasks distributed amongst corresponding processor subunits.
10. The distributed processor on a memory chip of claim 9 , wherein the higher-level series of tasks comprises a set of instructions in a human-readable programming language.
11. The distributed processor on a memory chip of claim 9 , wherein the lower-level series of tasks comprises a set of instructions in a machine code.
12. A distributed processor on a memory chip, comprising:
a semiconductor substrate;
a plurality of processor subunits disposed on the semiconductor substrate of the memory chip;
a plurality of memory banks disposed on the semiconductor substrate of the memory chip, wherein each one of the plurality of processor subunits is connected to at least one dedicated memory bank from among the plurality of memory banks, and wherein each memory bank of the plurality of memory banks is configured to store data independent from data stored in other ones of the plurality of memory banks, and wherein at least some of the data stored in one particular memory bank from among the plurality of memory banks comprises a duplicate of data stored in at least another one of the plurality of memory banks; and
a plurality of buses, wherein each one of the plurality of buses connects one of the plurality of processor subunits to one or more corresponding, dedicated memory banks from among the plurality of memory banks,
wherein data transfers across a particular one of the plurality of buses are controlled by a corresponding processor subunit connected to the particular one of the plurality of buses; and
wherein each of the plurality of processor subunits is configured to access data stored in at least its corresponding dedicated one of the plurality of memory banks and perform one or more calculations using at least one value included in the accessed data.
13. The distributed processor on a memory chip of claim 12 , wherein the at least some data duplicated across the one particular memory bank from among the plurality of memory banks and the at least another one of the plurality of memory banks is defined by a compiler configured to duplicate data across memory banks.
14. The distributed processor on a memory chip of claim 12 , wherein the at least some data duplicated across the one particular memory bank from among the plurality of memory banks and the at least another one of the plurality of memory banks comprises weights of a neural network.
15. The distributed processor on a memory chip of claim 14 , wherein each node in the neural network is defined by at least one processor subunit from among the plurality of processor subunits.
16. The distributed processor on a memory chip of claim 15 , wherein each node comprises machine code executed by the at least one processor subunit defining the node.