Cascade streaming between data processing engines in an array
Examples herein describe techniques for communicating directly between cores in an array of data processing engines. In one embodiment, the array is a 2D array where each of the data processing engines includes one or more cores. In addition to the cores, the data processing engines can include a memory module (with memory banks for storing data) and an interconnect which provides connectivity between the cores. Using the interconnect, however, can add latency when transmitting data between the cores. In the embodiments herein, the array includes core-to-core communication links that directly connect one core in the array to another core. The cores can use these communication links to bypass the interconnect and the memory module to transmit data directly.
1. A method of processing data in an array of data processing engines in a system on a chip (SOC), the array of data processing engines including a first data processing engine having a first core and a first memory external to the first core and a second data processing engine having a second core and a second memory external to the second core, the method comprising:
processing the data in the first core; and
transmitting the processed data directly from the first core to the second core using a core-to-core communication link that is dedicated to communication between the first core and the second core and bypasses the memories in the first and the second data processing engines, wherein the first core and second core being directly adjacent cores in an array of cores, and the second core further processes the data processed by the first core.
2. The method of claim 1 , further comprising:
executing a first task in the first core to generate the data; and
executing a second task in the second core to process the data received from the first core, wherein the first and second tasks are sub-tasks for a same kernel.
3. The method of claim 1 , wherein transmitting the data directly from the first core to the second core is performed without storing the data in memory external to the first and second cores.
4. The method of claim 1 , wherein the first core comprises a plurality of transmitters and the second core comprises a plurality of receivers, wherein the core-to-core communication link comprises a plurality of connections coupling the plurality of transmitters to the plurality of receivers.
5. The method of claim 4 , further comprising:
determining a type of the data being generated at the first core; and
activating at least one of the plurality of transmitters based on the type of the data and deactivating at least one of the plurality of transmitters for transmitting the data to the second core.
6. The method of claim 4 , wherein the first core comprises a plurality of transceivers that comprises the plurality of transmitters and receivers to receive and transmit data from the second core using the core-to-core communication link, wherein each of the plurality of transceivers comprises a multiply accumulator (MAC) unit.
7. The method of claim 1 , wherein the first and second data processing engines each comprises an interconnect, wherein the interconnects of the first and second data processing engines are communicatively coupled.
8. The method of claim 7 , wherein the core-to-core communication link is separate from, and independent of, the interconnects in the first and second data processing engines.
9. The method of claim 7 , wherein the interconnects in the first and second data processing engines comprise a streaming network.
10. The method of claim 1 , further comprising:
transmitting data directly from the second core to a third core using a second core-to-core communication link, wherein the third core is disposed in a third data processing engine in the array, and wherein the first data processing engine directly neighbors the second data processing engine in the array and the second data processing engine directly neighbors the third data processing engine in the array.
11. A system on a chip (SoC), comprising:
a first data processing engine in an array of data processing engines, the first data processing engine including a first core and a first memory external to the first core;
a second data processing engine in the array, the second data processing engine including a second core and a second memory external to the second core; and
a core-to-core communication link coupled to the first core at a first end and the second core at a second end, the core-to-core communication link being dedicated to communication between the first core and the second core, the first core and second core being directly adjacent cores in an array of cores, wherein the first core is configured to transmit data directly to the second core using the core-to-core communication link bypassing the memories in the first and second data processing engines, wherein the first core processes the data and the second core further processes the data.
12. The SoC of claim 11 , wherein the first core is configured to execute a first task to generate the data and the second core is configured to execute a second task to process the data received from the first core, wherein the first and second tasks are sub-tasks for a same kernel.
13. The SoC of claim 11 , wherein transmitting the data directly from the first core to the second core using the core-to-core communication link is performed without storing the data in memory external to the first and second cores.
14. The SoC of claim 11 , wherein the first core comprises a plurality of transmitters and the second core comprises a plurality of receivers, wherein the core-to-core communication link comprises a plurality of connections coupling the plurality of transmitters to the plurality of receivers.
15. The SoC of claim 14 , wherein the first core is configured to:
determine a type of the data being generated at the first core; and
activating at least one of the plurality of transmitters based on the type of the data and deactivating at least one of the plurality of transmitters for transmitting the data to the second core.
16. The SoC of claim 14 , wherein the first core comprises a plurality of transceivers that comprises the plurality of transmitters and receivers to transmit and receive data from the second core using the core-to-core communication link, wherein each of the plurality of transceivers comprises a multiply accumulator (MAC) unit.
17. The SoC of claim 11 , wherein the first and second data processing engines each comprises an interconnect, wherein the interconnects of the first and second data processing engines are communicatively coupled.
18. The SoC of claim 17 , wherein the core-to-core communication link is separate from, and independent of, the interconnects in the first and second data processing engines.
19. The SoC of claim 17 , wherein the interconnects in the first and second data processing engines comprise a streaming network.
20. The SoC of claim 11 , further comprising:
a third data processing engine in the array, the third data processing engine comprises a third core; and
a second core-to-core communication link coupled to the second core at a first end and to the third core at a second end, wherein the second core transmits data directly to the third core using the second core-to-core communication link, and wherein the first data processing engine directly neighbors the second data processing engine in the array and the second data processing engine directly neighbors the third data processing engine in the array.
21. An integrated circuit comprising:
an array of data processing engines (DPEs), each DPE of the array of DPEs comprising:
a core comprising hardened logic and a program memory, the hardened logic being configured to execute instructions stored in the program memory to process data; and
a memory including memory banks; and
a plurality of core-to-core communication links, each core-to-core communication link of the plurality of core-to-core communication links being coupled to respective cores of a corresponding neighboring pair of DPEs of the array of DPEs, the respective core-to-core communication link being dedicated to communication between the cores of the corresponding neighboring pair of DPEs, the respective core-to-core communication link being configured to communicate data directly between the cores of the corresponding neighboring pair of DPEs bypassing the memories of the corresponding neighboring pair of DPEs,
wherein a first core of a first neighboring pair of DPEs of the array of DPEs is configured to process the data and a second core of the first neighboring pair of DPEs is configured to further process the data.