Methods and apparatus for system-on-a-chip neural network processing applications
Methods and apparatus for multi-purpose neural network core and memory. The asynchronous/parallel nature of neural network tasks may allow a neural network IP core to dynamically switch between: a system memory (in whole or part), a neural network processor (in whole or part), and/or a hybrid of system memory and neural network processor. In one specific implementation, the multi-purpose neural network IP core has partitioned its sub-cores into a first set of neural network sub-cores, and a second set of memory sub-cores that operate as addressable memory space. Partitioning may be statically assigned at “compile-time”, dynamically assigned at “run-time”, or semi-statically assigned at “program-time” Any number of considerations may be used to partition the sub-cores; examples of such considerations may include, without limitation: thread priority, memory usage, historic usage, future usage, power consumption, performance, etc.
1 . A system-on-a-chip, comprising:
a system bus;
a first processor core coupled to the system bus;
a neural network core coupled to the system bus;
where the neural network core is partitioned into a first set of neural network sub-cores and a second set of memory sub-cores;
where each sub-core of the neural network core comprises a router and a memory; and
a translation logic comprising a neural network interface, a memory interface, and a packet-based interface,
where the neural network interface enables access to the first set of neural network sub-cores, the memory interface enables access to the second set of memory sub-cores, and the packet-based interface is coupled to at least a first sub-core of the neural network core.
2 . The system-on-a-chip of claim 1 , where the memory interface provides an addressable memory space to the system bus, where the addressable memory space is controlled by the first processor core.
3 . The system-on-a-chip of claim 2 , further comprising a second intellectual property core and where the addressable memory space is accessible by the second intellectual property core.
4 . The system-on-a-chip of claim 1 , where a first router of the first sub-core is configured to route at least one packet to a second router of a second sub-core.
5 . The system-on-a-chip of claim 1 , where the first set of neural network sub-cores and the second set of memory sub-cores are statically partitioned at compile-time.
6 . The system-on-a-chip of claim 1 , where the first set of neural network sub-cores and the second set of memory sub-cores are dynamically partitioned at run-time.
7 . The system-on-a-chip of claim 1 , where the system bus is characterized by a word size and the packet-based interface is characterized by a payload size smaller than the word size.
8 . A neural network core, comprising:
a plurality of sub-cores that is partitioned into a first set of neural network sub-cores and a second set of memory sub-cores,
where each sub-core of the plurality of sub-cores comprises a corresponding router and a corresponding memory; and
a translation logic comprising a neural network interface, a memory interface, and a packet-based interface,
where the neural network interface enables access to the first set of neural network sub-cores, the memory interface enables access to the second set of memory sub-cores, and the packet-based interface is coupled to at least a first sub-core of the neural network core.
9 . The neural network core of claim 8 , where each sub-core of the plurality of sub-cores communicate with other sub-cores of the plurality of sub-cores using an asynchronous handshake protocol.
10 . The neural network core of claim 9 , where the asynchronous handshake protocol comprises a start handshake that initiates communication, one or more data handshakes for each data packet, and an end handshake that terminates communication.
11 . The neural network core of claim 8 , where the corresponding memory of each sub-core comprises a first memory of a first bit width.
12 . The neural network core of claim 11 , where the corresponding memory of each sub-core comprises a second memory of a second bit width greater than the first bit width.
13 . The neural network core of claim 12 , where each sub-core of the plurality of sub-cores further comprises processing hardware coupled to the corresponding memory that is physically constructed to access the first memory with the first bit width and the second memory with the second bit width.
14 . The neural network core of claim 13 , where the neural network interface and the memory interface are memory mapped to a system bus with a third bit width greater than or equal to the second bit width.
15 . A method of operating a neural network core comprising a plurality of sub-cores, where each sub-core of the plurality of sub-cores comprises a router, a local memory, and a neural network processor logic that can be individually enabled and disabled, comprising:
partitioning the plurality of sub-cores of the neural network core into a first set of neural network sub-cores and a second set of memory sub-cores;
enabling the neural network processor logic for the first set of neural network sub-cores and disabling the neural network processor logic for the second set of memory sub-cores;
assigning a first range of memory addresses to the neural network core based on the first set of neural network sub-cores, where the first range of memory addresses has a first word length;
assigning a second range of memory addresses to system-wide memory based on the second set of memory sub-cores, where the second range of memory addresses has a second word length that is different than the first word length; and
enabling the first range of memory addresses and the second range of memory addresses.
16 . The method of claim 15 , where partitioning the neural network core is statically assigned at compile-time.
17 . The method of claim 15 , where partitioning the neural network core is dynamically assigned at run-time based on one or more of: a number of neural network threads, a thread priority, a memory usage, a historic usage, a predicted usage, a power consumption, or a performance requirement.
18 . The method of claim 15 , further comprising partitioning the neural network core into a third set of reserve sub-cores.
19 . The method of claim 18 , further comprising allocating at least one core of the third set of reserve sub-cores to the first set of neural network sub-cores based on a neural network thread status.
20 . The method of claim 18 , further comprising allocating at least one core of the third set of reserve sub-cores to the second set of memory sub-cores based on system-memory activity.