Neural network device with configurable shared memory
A neural network device includes a shared physical memory that has a plurality of independently accessible memory sections. The neural network device further includes a data processor core to execute instructions. The instructions include at least one instruction involving multiple memory access operations specifying respective logical memory addresses in a plurality of logical memories. During configuration of the neural network device for a particular application, respective memory sections of the plurality of independently accessible memory sections are assigned to respective logical memories of the plurality of logical memories. In accordance with the configuration, each logical memory address of the respective logical memory addresses is mapped to a physical address by providing an indication of a memory section of the plurality of independently accessible memory sections and a row address within the memory section.
1 . A neural network device comprising:
a shared physical memory comprising a plurality of independently accessible memory sections; and
a data processor core to execute instructions comprising:
at least one instruction that includes, in response to an event message, performing multiple memory access operations that specify respective logical memory addresses in a plurality of logical memories and causes updating of a state value of a neural network component to obtain an updated state value, the multiple memory access operations comprising accessing a first storage location storing a weight to weigh the event message and accessing a second storage location to read a current state value of the neural network component and write the updated state value, the first storage location being of a first logical memory of the plurality of logical memories that stores weight data, and the second storage location being of a second logical memory of the plurality of logical memories that stores state data, each logical memory address of the respective logical memory addresses being mapped, based on a configuration associated with a particular application that is to be maintained during runtime, to a physical address by providing an indication of a memory section of the plurality of independently accessible memory sections and a row address within the memory section, and a configuration stage in which the configuration is applied comprising assigning respective memory sections of the plurality of independently accessible memory sections of the shared physical memory to respective logical memories of the plurality of logical memories.
2 . The neural network device of claim 1 , wherein the plurality of independently accessible memory sections comprises independently accessible memory banks, and the assigning comprises assigning respective sets of the independently accessible memory banks to the respective logical memories.
3 . The neural network device of claim 1 , wherein the plurality of independently accessible memory sections comprises independently accessible memory banks, the assigning comprises assigning respective address ranges within the independently accessible memory banks to the respective logical memories, and concurrent access requests associated with a same memory bank of the independently accessible memory banks are serialized.
4 . The neural network device of claim 1 , wherein the plurality of independently accessible memory sections comprises single ported memory units.
5 . The neural network device of claim 1 , wherein the plurality of independently accessible memory sections is provided by identical memory units.
6 . The neural network device of claim 1 , further comprising a respective buffer for each of the plurality of independently accessible memory sections.
7 . The neural network device of claim 1 , wherein a load operation is prioritized over a store operation in response to concurrent access requests for the load operation and the store operation.
8 . The neural network device of claim 1 , wherein the at least one instruction comprises an instruction in which data words located contiguously in a logical memory of the plurality of logical memories are accessed in a single memory cycle.
9 . The neural network device of claim 1 , wherein the at least one instruction comprises an instruction that performs loading and storing of a plurality of data words in parallel using a single logical memory address of the respective logical memory addresses.
10 . A neural network processing method comprising:
assigning, during a configuration stage, respective memory sections of a plurality of independently accessible memory sections of a shared physical memory of a neural network device to respective logical memories of a plurality of logical memories to apply a configuration associated with a particular application that is to be maintained during runtime; and
executing, by a data processor core of the neural network device, at least one instruction that includes, in response to an event message, performing multiple memory access operations that specify respective logical memory addresses in the plurality of logical memories and causes updating of a state value of a neural network component to obtain an updated state value, the multiple memory access operations comprising accessing a first storage location storing a weight to weigh the event message and accessing a second storage location to read a current state value of the neural network component and write the updated state value, the first storage location being of a first logical memory of the plurality of logical memories that stores weight data, and the second storage location being of a second logical memory of the plurality of logical memories that stores state data, and each logical memory address of the respective logical memory addresses being mapped, based on the configuration, to a physical address by providing an indication of a memory section of the plurality of independently accessible memory sections and a row address within the memory section.
11 . The neural network processing method of claim 10 , wherein the plurality of independently accessible memory sections comprises independently accessible memory banks, and the assigning comprises assigning respective sets of the independently accessible memory banks to the respective logical memories.
12 . The neural network processing method of claim 10 , wherein the plurality of independently accessible memory sections comprises independently accessible memory banks, the assigning comprises assigning respective address ranges within the independently accessible memory banks to the respective logical memories, and concurrent access requests associated with a same memory bank of the independently accessible memory banks are serialized.
13 . The neural network processing method of claim 10 , wherein the plurality of independently accessible memory sections comprises single ported memory units.
14 . The neural network processing method of claim 10 , wherein the plurality of independently accessible memory sections is provided by identical memory units.
15 . The neural network processing method of claim 10 , wherein the neural network device comprises a respective buffer for each of the plurality of independently accessible memory sections, the neural network processing method comprising using a buffer of the respective buffers to temporarily buffer a memory access request to an independent accessible memory section associated with the buffer.
16 . The neural network processing method of claim 10 , further comprising automatically prioritizing a load operation over a store operation in response to concurrent access requests for the load operation and the store operation.
17 . The neural network processing method of claim 10 , wherein the at least one instruction comprises an instruction in which data words located contiguously in a logical memory of the plurality of logical memories are accessed in a single memory cycle.
18 . The neural network processing method of claim 10 , wherein the at least one instruction comprises an instruction that performs loading and storing of a plurality of data words in parallel using a single logical memory address of the respective logical memory addresses.
19 . A neural network system comprising a plurality of neural network devices, each neural network device comprising:
a shared physical memory comprising a plurality of independently accessible memory sections; and
a data processor core to execute instructions comprising:
at least one instruction that includes, in response to an event message, performing multiple memory access operations that specify respective logical memory addresses in a plurality of logical memories and causes updating of a state value of a neural network component to obtain an updated state value, the multiple memory access operations comprising accessing a first storage location storing a weight to weigh the event message and accessing a second storage location to read a current state value of the neural network component and write the updated state value, the first storage location being of a first logical memory of the plurality of logical memories that stores weight data, and the second storage location being of a second logical memory of the plurality of logical memories that stores state data, each logical memory address of the respective logical memory addresses being mapped, based on a configuration associated with a particular application that is to be maintained during runtime, to a physical address by providing an indication of a memory section of the plurality of independently accessible memory sections and a row address within the memory section, and a configuration stage in which the configuration is applied comprising assigning respective memory sections of the plurality of independently accessible memory sections of the shared physical memory to respective logical memories of the plurality of logical memories.
20 . The neural network system of claim 19 , further comprising a message exchange network with a respective network node for each neural network device.