Time-multiplexed use of reconfigurable hardware

1. A method for executing applications in a system comprising general hardware and reconfigurable hardware, the method comprising:

accessing a first execution file comprising metadata storing a first priority indicator associated with a first application, and a second execution file comprising metadata storing a second priority indicator associated with a second application;

interleaving use of the reconfigurable hardware by the first application and the second application, and scheduling the interleaving to take into account (i) workload of the reconfigurable hardware and (ii) the first priority indicator and the second priority indicator associated with the first application and the second application, respectively;

when the reconfigurable hardware is used by a first one of the first and second applications, using the general hardware by either the first one or a second one of the first and second applications;

configuring the reconfigurable hardware in a first configuration, when the reconfigurable hardware is used by the first application;

configuring the reconfigurable hardware in a second configuration that is different from the first configuration, when the reconfigurable hardware is used by the second application;

the first execution file further comprises a first configuration file, and the second execution file further comprises a second configuration file;

the first configuration file comprises configuration information usable to configure the reconfigurable hardware in the first configuration;

the second configuration file comprises configuration information usable to configure the reconfigurable hardware in the second configuration;

receiving a data flow graph associated with the first application; and

compiling the data flow graph, to generate the first execution file comprising (i) the first configuration file and (ii) the metadata storing the first priority indicator.

2. The method of claim 1 , wherein:

the reconfigurable hardware comprises an array of configurable units, the array of configurable units comprising a first configurable unit including a multi-stage SIMD (Single Instruction, Multiple Data) pipeline; and

configuring the reconfigurable hardware in the first and second configurations comprise configuring the multi-stage SIMD differently in the first and second configurations.

3. The method of claim 1 , wherein:

the reconfigurable hardware comprises an array of configurable units that includes at least a configurable switch, a first Pattern Compute Unit (PCU), a second PCU, and a Pattern Memory Unit (PMU); and

configuring the reconfigurable hardware in the first and second configurations comprise configuring the configurable switch differently in the first and second configurations, such that the configurable switch interconnects the first PCU with the PMU in the first configuration, and the configurable switch interconnects the second PCU with the PMU in the second configuration.

4. The method of claim 1 , wherein:

the reconfigurable hardware comprises an array of configurable units that includes at least a first compute unit, a second compute unit, and a configurable memory unit; and

configuring the reconfigurable hardware in the first and second configurations comprise

configuring the memory unit differently in the first and second configurations, such that the memory unit stores data from the first compute unit in the first configuration, and the memory unit stores data from the second compute unit in the second configuration.

5. The method of claim 1 , wherein:

configuring the reconfigurable hardware in the first configuration comprises configuring a reconfigurable unit of the reconfigurable hardware to implement first one or more of a plurality of functions; and

configuring the reconfigurable hardware in the second configuration comprises configuring the reconfigurable unit of the reconfigurable hardware to implement second one or more of the plurality of functions,

wherein the plurality of functions comprises two or more of Rectified Linear Units (ReLU), leaky ReLU, convolution, transpose convolution, hyperbolic tangent, sigmoid, softmax, element-wise addition, matrix multiplication, General Matrix Multiply (GeMM), layer normalization, batch normalization, loss function, and tensor shape modifier.

6. The method of claim 1 , wherein:

the first execution file comprises (i) a first one or more sections that are to be executed by the general hardware and (ii) a second one or more sections that are to be executed by the reconfigurable hardware;

the second execution file comprises (i) a third one or more sections that are to be executed by the general hardware and (ii) a fourth one or more sections that are to be executed by the reconfigurable hardware; and

interleaving use of the reconfigurable hardware comprises:

scheduling execution of at least one of the second one or more sections of the first application by the reconfigurable hardware simultaneously with execution of at least one of the third one or more sections of the second application by the general hardware.

7. The method of claim 6 , wherein interleaving use of the reconfigurable hardware further comprises:

scheduling execution of at least one of the fourth one or more sections of the second application by the reconfigurable hardware simultaneously with execution of at least one of the first one or more sections of the first application by the general hardware.

8. The method of claim 6 , wherein interleaving use of the reconfigurable hardware further comprises:

time-multiplexing use of the reconfigurable hardware by the first application and the second application, such that the second one or more sections of the first application are executed by the reconfigurable hardware, followed by execution of the fourth one or more sections of the second application by the reconfigurable hardware.

9. The method of claim 1 , wherein interleaving use of the reconfigurable hardware comprises:

first loading a first configuration file of the first application in a local memory of the reconfigurable hardware from an off-chip memory, where the off-chip memory and the reconfigurable hardware are respectively in two different integrated circuit chips;

subsequent to the first loading, first executing at least part of the first configuration file of the first application by a reconfigurable processor of the reconfigurable hardware;

while performing the first executing, receiving a switch command to switch usage of the reconfigurable processor of the reconfigurable hardware from the first application to the second application;

in response to the switch command, invalidating the first configuration file loaded in the local memory;

subsequent to the invalidating, loading a second configuration file of the second application in the local memory of the reconfigurable hardware; and

second executing at least part of the second configuration file of the second application by the reconfigurable processor of the reconfigurable hardware.

10. The method of claim 1 , wherein interleaving use of the reconfigurable hardware comprises:

loading a first configuration file of the first application and a second configuration file of the first application in a local memory of the reconfigurable hardware from an off-chip memory, where the off-chip memory and the reconfigurable hardware are in two different integrated circuit chips;

loading the first configuration file of the first application from the local memory of the reconfigurable hardware to a reconfigurable processor of the reconfigurable hardware;

executing at least part of the first configuration file of the first application by the reconfigurable processor of the reconfigurable hardware;

while executing at least part of the first configuration file, receiving a switch command to switch usage of the reconfigurable processor of the reconfigurable hardware from the first application to the second application;

in response to the switch command, invalidating the first configuration file loaded in the reconfigurable processor;

subsequent to the invalidating, loading the second configuration file of the second application from the local memory of the reconfigurable hardware to the reconfigurable processor of the reconfigurable hardware; and

executing at least part of the second configuration file of the second application by the reconfigurable processor of the reconfigurable hardware.

11. The method of claim 1 , wherein interleaving use of the reconfigurable hardware comprises:

simultaneously loading a first configuration file of the first application and a second configuration file of the first application in a local memory of the reconfigurable hardware;

loading the first configuration file of the first application from the local memory of the reconfigurable hardware to a reconfigurable processor of the reconfigurable hardware;

executing at least part of the first configuration file of the first application by the reconfigurable processor of the reconfigurable hardware;

while executing at least part of the first configuration file, receiving a switch command to switch usage of the reconfigurable processor of the reconfigurable hardware from the first application to the second application;

in response to the switch command, loading the second configuration file of the second application from the local memory of the reconfigurable hardware to the reconfigurable processor of the reconfigurable hardware; and

executing at least part of the second configuration file of the second application by the reconfigurable processor of the reconfigurable hardware.

12. The method of claim 1 , wherein interleaving use of the reconfigurable hardware comprises:

simultaneously loading a first configuration file of the first application and a second configuration file of the second application in a first buffer and a second buffer, respectively, within a reconfigurable processor of the reconfigurable hardware;

during a first execution phase, outputting, via a multiplexer, the first configuration file of the first application from the first buffer, and executing the first configuration file of the first application by the reconfigurable processor;

at an end of the first execution phase, receiving a switch command to switch usage of the reconfigurable processor of the reconfigurable hardware from the first application to the second application; and

in response to the switch command and during a second execution phase, outputting, via the multiplexer, the second configuration file of the second application from the second buffer, and executing the second configuration file of the second application by the reconfigurable processor.

13. The method of claim 1 , further comprising:

compiling a first dataflow graph associated with the first application, to generate the first execution file that includes (i) a first plurality of configuration files to be executed by the reconfigurable hardware and (ii) a first plurality of host execution files to be executed by the general hardware; and

compiling a second dataflow graph associated with the second application, to generate the second execution file that includes (i) a second plurality of configuration files to be executed by the reconfigurable hardware and (ii) a second plurality of host execution files to be executed by the general hardware.

14. The method of claim 13 , further comprising:

while compiling to generate the first execution file, defining a plurality of switching points within the first plurality of configuration files,

wherein a first switching point is between two consecutive configuration files, and wherein a second switching point is within a configuration file, and

wherein usage of the reconfigurable hardware is switched from or to the first application at one or more switching points of the plurality of switching points.

15. The method of claim 14 , wherein the reconfigurable hardware is to synchronize data with an off-chip memory at a switching point, irrespective of whether a switching of the reconfigurable hardware from or to the first application occurs at the switching point, and wherein the off-chip memory and the reconfigurable hardware are respectively within two different integrated circuit chips.

16. The method of claim 1 , wherein interleaving use of the reconfigurable hardware further comprises:

interleaving use of the reconfigurable hardware, further taking into account a state of one or both the first application and the second application; and

dynamically updating the interleaved use of the reconfigurable hardware, in response to a change in state of at least one of the first application and the second application,

wherein each of the first and second application is, at a specific time, in a respective one of a ready state, a waiting state, or a blocked state.

17. The method of claim 1 , wherein the first priority indicator comprises a first latency sensitivity associated with the first application, and wherein the second priority indicator comprises a second latency sensitivity associated with the second application.

18. The method of claim 17 , wherein the first latency sensitivity associated with the first application provides an indication of a first target maximum latency for the first application, and wherein the second latency sensitivity associated with the second application provides an indication of a second target maximum latency for the second application.

19. The method of claim 1 , wherein the first priority indicator comprises a first target Quality of Service (QoS) identifier associated with the first application, and wherein the second priority indicator comprises a second target Quality of Service (QoS) identifier associated with the second application.

20. The method of claim 1 , wherein the first priority indicator comprises a first deadline timestamp by which execution of the first application is to be targeted to be completed.

21. The method of claim 1 , wherein using the general hardware by either the first one or the second one of the first and second applications comprises:

when the reconfigurable hardware is used by the first application, using the general hardware by the first application, such that a section of the first application is executed by the reconfigurable hardware in parallel with execution of another section of the first application by the general hardware.

22. The method of claim 1 , wherein using the general hardware by either the first one or the second one of the first and second applications comprises:

when the reconfigurable hardware is used by the first application, using the general hardware by the second application, such that a section of the first application is executed by the reconfigurable hardware in parallel with execution of a section of the second application by the general hardware.