Unified memory for integrated circuit executing neural network
Some embodiments provide an integrated circuit (IC). The IC includes a microprocessor circuit for loading configuration for a neural network and generating instructions for executing the neural network based on the configuration. The IC includes a neural network inference circuit for executing a neural network for input data according to instructions received from the central processing circuit. The IC includes an input processing circuit for receiving data and preparing input data for the neural network inference circuit. The IC includes a unified memory accessible by the microprocessor circuit, neural network inference circuit, and input processing circuit.
1 . An integrated circuit (IC) comprising:
a microprocessor circuit for loading configuration for a neural network and generating instructions for executing the neural network based on the configuration;
a neural network inference circuit for executing the neural network for input data according to the instructions received from the microprocessor circuit, the neural network inference circuit comprising a plurality of cores;
an input processing circuit for receiving data and preparing the input data for the neural network inference circuit;
a unified memory accessible by the microprocessor circuit, the neural network inference circuit, and the input processing circuit, wherein the unified memory comprises a plurality of memory banks, and wherein a first core of the plurality of cores of the neural network inference circuit is configured to have exclusive access from among the plurality of cores to only a respective first subset of dedicated memory banks of the plurality of memory banks, and wherein a second core of the plurality of cores of the neural network inference circuit is configured to have exclusive access from among the plurality of cores to only a second subset of dedicated memory banks of the plurality of memory banks; and
at least a first direct connection from the first core of the plurality of cores to a first dedicated memory bank of a first subset of dedicated memory banks,
wherein the input processing circuit writes first input data to the first subset of dedicated memory banks and second input data to the second subset of dedicated memory banks, and
wherein a logical address of the first subset of dedicated memory banks is swapped with a logical address of the second subset of dedicated memory banks, after executing, by the neural network inference circuit, the first input data.
2 . The IC of claim 1 , wherein each memory bank of the plurality of memory banks is accessible by the input processing circuit and the microprocessor circuit.
3 . The IC of claim 1 , wherein each respective memory bank of the plurality of memory banks is associated with a respective core of the plurality of cores of the neural network inference circuit, and wherein each respective memory bank has a same memory configuration.
4 . The IC of claim 3 , wherein two memory banks have the same memory configuration when a number of physical memories for each of the two memory banks is the same and physical memories of each of the two memory banks have a same size.
5 . The IC of claim 1 , further comprising at least one direct connection from the first core of the plurality of cores to a first memory bank of a respective subset of dedicated memory banks a second direct connection from the second core of the plurality of cores to a second dedicated memory bank of the second subset of dedicated memory banks.
6 . The IC of claim 5 , further comprising a crossbar for providing access to each memory bank of the plurality of memory banks of the unified memory to the microprocessor circuit and the input processing circuit.
7 . The IC of claim 1 , wherein the neural network inference circuit writes weight values to the unified memory based on instructions provided from the microprocessor circuit at bootup of the IC.
8 . The IC of claim 7 , wherein the neural network inference circuit:
writes a set of input data to the unified memory based on instructions provided by the input processing circuit; and
executes the neural network for the input data to generate output data.
9 . The IC of claim 8 , wherein while executing the neural network for the set of input data the neural network inference circuit reads the set of input data along with a set of weight values from the unified memory, writes sets of intermediate activation values to the unified memory, and reads sets of intermediate activation values along with sets of weight values from the unified memory.
10 . The IC of claim 8 , wherein the microprocessor circuit stores in the unified memory a program comprising the instructions for the neural network inference circuit to execute the neural network.
11 . The IC of claim 10 , wherein the program is loaded from off-chip non-volatile storage at bootup of the IC.
12 . The IC of claim 1 , wherein the neural network inference circuit uses a first address space for addressing the unified memory while the microprocessor circuit and the input processing circuit use the first address space and a second address space for addressing the unified memory.
13 . The IC of claim 12 , wherein addresses in the first address space decode to specific physical memory locations.
14 . The IC of claim 13 , wherein:
a first address in the first address space is formatted to include (i) a first address prefix indicating that the first address is formatted in the first address space and (ii) a first physical address portion that refers to a first physical memory location and the first core of the plurality of cores, wherein the first core has exclusive access from among the plurality of cores to the first physical memory location; and
a second address in the second address space is formatted to include (i) a second address prefix indicating that the second address is formatted in the second address space and (ii) a first virtual address portion that refers to a first virtual memory address, wherein the first virtual memory address is used to identify a second physical memory location.
15 . The IC of claim 14 , wherein a mapping table is used to map the first virtual memory address to the second physical memory location.
16 . The IC of claim 15 , wherein the second address space allows a set of unused physical memory locations associated with two or more different cores of the plurality of cores to be treated as a contiguous block of virtual addresses used to identify the set of unused physical memory locations for access by the microprocessor circuit and the input processing circuit, wherein the set of unused physical memory locations are associated with static random access memory (SRAM).
17 . The IC of claim 12 , wherein the microprocessor circuit and the input processing circuit use the first address space for addressing the unified memory when reading and writing data that is also accessed by the neural network inference circuit.
18 . The IC of claim 17 , wherein the microprocessor circuit and the input processing circuit use the second address space for addressing the unified memory when reading and writing data that is not accessed by the neural network inference circuit.
19 . The IC of claim 1 , wherein the neural network is a first neural network, wherein the plurality of cores of the neural network inference circuit are separated into at least a first cluster of cores and a second cluster of cores.
20 . The IC of claim 19 , wherein the first cluster of cores is configured to execute the first neural network, and wherein the second cluster of cores is configured to execute a second neural network at a same time as the first neural network is executed.