Neural processing device and method for synchronization thereof
A neural processing device is provided. The neural processing device comprises a plurality of neural processors, a shared memory shared by the plurality of neural processors, a plurality of semaphore memories, and global interconnection. The plurality of neural processors generates a plurality of L3 sync targets, respectively. Each semaphore memory is associated with a respective one of the plurality of neural processors, and the plurality of semaphore memories receive and store the plurality of L3 sync targets, respectively. Synchronization of the plurality of neural processors is performed according to the plurality of L3 sync targets. The global interconnection connects the plurality of neural processors with the shared memory, and comprises an L3 sync channel through which an L3 synchronization signal corresponding to at least one L3 sync target is transmitted.
1. A neural processing device comprising:
first and second neural cores; and,
an L2 sync path configured to directly transmit an L2 synchronization signal between the first and second neural cores,
wherein the first neural core comprises,
a first local memory included in the first neural core and temporarily storing first data to be input and output,
a first load/store unit (LSU) included in the first neural core and moving the first data in the first local memory,
a processing unit included in the first neural core and performing an computation operation on the first data,
a data path through which the first data is transmitted between the local memory, the first LSU, and the processing unit, and
an L1 sync path configured to transmit an L1 synchronization signal, in the first neural core, between the first local memory, the first LSU, and the processing unit, wherein the L1 sync path is a separate path from the data path.
2. The neural processing device of claim 1 , further comprising a local interconnection configured to transmit second data between the first and second neural cores,
wherein the second data of the local interconnection and the L2 synchronization signal of the L2 sync path are transmitted independently of each other.
3. The neural processing device of claim 1 , wherein the first LSU comprises,
a local memory store unit for storing to the first local memory;
a local memory load unit for loading from the first local memory;
a first neural core store unit for performing storage externally from the first neural core; and
a first neural core load unit that performs an external load on the first neural core.
4. The neural processing device of claim 1 , wherein the first LSU comprises,
a local memory store unit for storing to the first local memory;
a first neural core store unit for performing storage externally from the neural core;
wherein the L1 synchronization signal is transmitted between the local memory store unit and the first neural core store unit through the L1 sync path.
5. The neural processing device of claim 4 , wherein the L1 synchronization signal comprises,
an L1 sync generation signal transmitted when the first neural core store unit is switched to a ready state for L1 synchronization;
an L1 sync request signal synchronized in correspondence with the L1 synchronization signal,
wherein the L1 sync request signal comprises a first L1 sync request signal generated by the first neural core store unit.
6. The neural processing device of claim 1 , wherein the L2 sync path is transmitted from the first neural core store unit in the first neural core to the second neural core.
7. The neural processing device of claim 6 , further comprising,
a first neural processor comprising the first and second neural core,
a first and second semaphore memory for receiving and storing L3 sync target inside the first neural processor, wherein the each of first and second semaphore memory is performed the L3 synchronization of the first neural processor according to the L3 sync target.
8. A neural processing device comprising:
first and second neural cores;
a L2 shared memory shared by the first and second neural core;
a local interconnection configured to connect the first and second neural core and the L2 shared memory and transmit data; and
an L2 sync path is independent of the local interconnection, directly transmit an L2 synchronization signal between the first and second neural core and the L2 shared memory,
wherein the L2 sync path is physically separated from the local interconnection such that the data of the local interconnection and the L2 synchronization signal of the L2 sync path are transmitted independently of each other.
9. The neural processing device of claim 8 , wherein the first neural core comprises a first LSU moving data within the first neural core, wherein the first LSU comprises a first neural core store unit,
wherein second neural core comprises a second LSU moving data within the second neural core, wherein the second LSU comprises a second neural core load unit,
wherein the L2 synchronization signal is transmitted between the first neural core store unit and the second neural core load unit through the L2 sync path.
10. The neural processing device of claim 9 , wherein the L2 synchronization signal comprises,
a send L2 sync transmitted to the first neural core store unit, and
a receive L2 sync generated to perform a load operation in correspondence with the send L2 sync,
wherein the receive L2 sync comprises a first receive L2 sync generated by the second neural core load unit.
11. The neural processing device of claim 8 , further comprising,
a first neural processor comprising the first and second neural cores,
a second neural processor,
a shared memory shared by the first and second neural processors,
a global interconnection configured to connect the first and second neural processors and the shared memory and comprises an L3 sync channel configured to transmit an L3 synchronization signal between the first and second neural processors.
12. The neural processing device of claim 11 , wherein the first neural core comprises an L1 sync path configured to transmit an L1 synchronization signal inside the first neural core.
13. A method for synchronizing a neural processing device including first and second neural cores, and a local interconnection configured to transmit data between the first and second neural cores, the method comprising:
transmitting an L1 synchronization signal within the first neural core through an L1 sync path separate from a data path through which a first data is transmitted within the first neural core;
directly transmitting, by the first neural core, an L2 synchronization signal from the first neural cores to the second neural core through an L2 sync path; and
after performing an L2 synchronization based on the L2 synchronization signal, receiving, by the second neural core, the first data through the local interconnection,
wherein the L2 sync path is physically separated from the local interconnection such that the first data of the local interconnection and the L2 synchronization signal of the L2 sync path are transmitted independently of each other.
14. The method for synchronizing a neural processing device of claim 13 , wherein transmitting the L1 synchronization signal comprises,
storing, by a local memory store unit of the first neural core second data in the local memory,
transmitting, by the local memory store unit, an L1 sync generation signal to a first neural core store unit, and
synchronizing an L1 sync request signal of the first neural core store unit.
15. The method for synchronizing a neural processing device of claim 14 , wherein synchronizing an L1 sync request signal comprises,
maintaining a ready state for receiving the L1 sync generation signal, and
receiving, by the first neural core store unit, the L1 sync generation signal.
16. The method for synchronizing a neural processing device of claim 14 , wherein transmitting the L2 synchronization signal comprises,
transmitting, by the first neural core store unit of the first neural core, a send L2 sync to a second neural core load unit of the second neural core;
generating, by the second neural core load unit, a receive L2 sync.
17. The method for synchronizing a neural processing device of claim 13 , wherein receiving the first data comprises:
after generation of a receive L2 sync, performing, by the second neural core, a first data request to a local memory of the first neural core, and
performing, by the second neural core, a first data return for the first data request.
18. The method for synchronizing a neural processing device of claim 13 , wherein the neural processing device comprises,
a first neural processor comprising the first and second neural cores, and
a second neural processor,
the method further comprising,
generating, by the first neural processor, an L3 sync target for an L3 synchronization,
arranging the L3 sync targets in order of virtual IDs of the first and second neural processors, and
identifying a physical ID of the second neural processor using the L3 sync target and a VPID table, wherein the VPID table is a conversion table between the virtual IDs and the physical ID of the neural processor.
19. The method for synchronizing a neural processing device of claim 13 , wherein the neural processing device further comprises a global interconnection comprising an L3 sync channel, wherein the method further comprises:
storing an L3 synchronization signal according to the L3 sync target in a first semaphore memory of the second neural processor through the L3 sync channel, and
performing, by the second neural processor, the L3 synchronization according to the value of the first semaphore memory.