Programmable and hierarchical control of execution of GEMM operation on accelerator

1. An apparatus for controlling execution of a GEMM (general matrix multiply) operation on an accelerator comprising multiple computation units, a first memory device, and a second memory device, the apparatus comprising:

a global controller comprising circuitry configured to:

determine an execution manner of the GEMM operation, the execution manner comprising partition information of the GEMM operation and computation unit allocation information of the partitioned GEMM operation,

generate one or more instructions to compute the partitioned GEMM operation on one or more allocated computation units among the multiple computation units, and

issue the one or more generated instructions;

a plurality of local controllers including a first local controller and a second local controller, the first local controller being configured to control data movement between the computation units and the first memory device and the second local controller being configured to control data movement between the first memory device and the second memory device; and

a plurality of queues including a first queue and a second queue, wherein the global controller and the plurality of local controllers are coupled via the plurality of queues and wherein the one or more issued instructions are received by at least one of the first queue and the second queue to enable the at least one of the first local controller and the second local controller to execute the one or more instructions.

2. The apparatus of claim 1 , wherein the global controller comprising circuitry configured to determine the execution manner of the GEMM operation further comprises:

circuitry configured to determine an execution manner of the GEMM operation based on a size of the GEMM operation, available computation units among the multiple computation units, and computation capacities of the available computation units.

3. The apparatus of claim 1 , wherein the global controller comprising circuitry configured to determine the execution manner of the GEMM operation further comprises:

circuitry configured to determine an execution manner of the GEMM operation by running a software program.

4. The apparatus of claim 1 , wherein the plurality of queues further comprises a third queue configured to receive, from the first local controller, an execution response of an instruction issued to the first local controller.

5. The apparatus of claim 4 , wherein the execution response comprises an execution result of whether the execution has failed or succeeded and a reason of failure when the execution has failed.

6. The apparatus of claim 4 , wherein the global controller comprising circuitry configured to determine the execution manner of the GEMM operation further comprises:

circuitry configured to determine an execution manner of the GEMM operation based on the execution response.

7. The apparatus of claim 1 , wherein at least one of the first local controller and the second local controller is a DMA (Dynamic Memory Access) unit on the accelerator.

8. The apparatus of claim 1 , wherein the global controller is configured to adapt to change in a memory subsystem of the accelerator.

9. The apparatus of claim 1 , wherein the one or more instructions comprise a series of instructions defining the one or more allocated computation units to execute the partitioned GEMM operation, timing of execution of the partitioned GEMM operation, and timing of data movement among the one or more allocated computation units, the first memory device, and the second memory device.

10. A method for controlling execution of a GEMM (general matrix multiply) operation on an accelerator comprising multiple computation units, a first memory device, and a second memory device, the method comprising:

determining an execution manner of the GEMM operation, the execution manner comprising partition information of the GEMM operation and computation unit allocation information of the partitioned GEMM operation;

generating one or more instructions to compute the partitioned GEMM operation on one or more allocated computation units among the multiple computation units; and

issuing the one or more instructions to at least one of a first queue and a second queue, which enables at least one of a first local controller and a second local controller to execute the one or more instructions, wherein the first local controller is configured to control data movement between the computation units and the first memory device according to an instruction issued to the first queue and the second local controller is configured to control data movement between the first memory device and the second memory device according to an instruction issued to the second queue.

11. The method of claim 10 , wherein determining an execution manner of the GEMM operation comprises:

determining an execution manner of the GEMM operation based on a size of the GEMM operation, available computation units among the multiple computation units, and computation capacities of the available computation units.

12. The method of claim 10 , wherein determining an execution manner of the GEMM operation comprises:

determining an execution manner of the GEMM operation by running a software program.

13. The method of claim 10 , further comprising:

receiving, from the first local controller, an execution response of an instruction issued to the first local controller.

14. The method of claim 13 , wherein the execution response comprises an execution result of whether the execution has failed or succeeded and a reason of failure when the execution has failed.

15. The method of claim 10 , wherein at least one of the first local controller and the second local controller is a DMA (Dynamic Memory Access) unit on the accelerator.

16. A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computing device to perform a method for controlling execution of a GEMM (general matrix multiply) operation on an accelerator comprising multiple computation units, a first memory device, and a second memory device, the method comprising:

determining an execution manner of the GEMM operation, the execution manner comprising partition information of the GEMM operation and computation unit allocation information of the partitioned GEMM operation;

generating one or more instructions to compute the partitioned GEMM operation on one or more allocated computation units among the multiple computation units; and

issuing the one or more instructions to at least one of a first queue and a second queue, which enables at least one of a first local controller and a second local controller to execute the one or more instructions, wherein the first local controller is configured to control data movement between the computation units and the first memory device according to an instruction issued to the first queue and the second local controller is configured to control data movement between the first memory device and the second memory device according to an instruction issued to the second queue.

17. The computer readable medium of claim 16 , wherein the set of instructions that are executable by at least one processor of the computing device to cause the computing device to further perform:

determining an execution manner of the GEMM operation based on a size of the GEMM operation, available computation units among the multiple computation units, and computation capacities of the available computation units.

18. The computer readable medium of claim 16 , wherein the set of instructions that are executable by at least one processor of the computing device to cause the computing device to further perform:

determining an execution manner of the GEMM operation by running a software program.

19. The computer readable medium of claim 16 , wherein the set of instructions that are executable by at least one processor of the computing device to cause the computing device to further perform:

receiving, from the first local controller, an execution response of an instruction issued to the first local controller.

20. The computer readable medium of claim 19 , wherein the execution response comprises an execution result of whether the execution has failed or succeeded and a reason of failure when the execution has failed.

21. The computer readable medium of claim 16 , wherein at least one of the first local controller and the second local controller is a DMA (Dynamic Memory Access) unit on the accelerator.

22. A terminal, comprising:

a host unit; and

an accelerator for controlling execution of a GEMM (general matrix multiply) operation, the accelerator comprising:

multiple computation units, a first memory device, and a second memory device;

a global controller comprising circuitry configured to:

determine an execution manner of the GEMM operation, the execution manner comprising partition information of the GEMM operation and computation unit allocation information of the partitioned GEMM operation,

generate one or more instructions to compute the partitioned GEMM operation on one or more allocated computation units among the multiple computation units, and

issue the one or more generated instructions;

a plurality of local controllers including a first local controller and a second local controller, the first local controller being configured to control data movement between computation units and the first memory device and the second local controller being configured to control data movement between the first memory device and the second memory device; and

a plurality of queues including a first queue and a second queue, wherein the global controller and the plurality of local controllers are coupled via the plurality of queues and wherein the one or more issued instructions are received by at least one of the first queue and the second queue to enable the at least one of the first local controller and the second local controller to execute the one or more instructions.