Method for interfacing with hardware accelerators
The present disclosure relates to a method for executing a computation task composed of at least one set of operations where subsets of pipelineable operations of the set of operations are determined in accordance with a pipelining scheme. A single routine may be created for enabling execution of the determined subsets of operations by a hardware accelerator. The routine has, as arguments, a value indicative of input data and values of configuration parameters of the computation task, where a call of the routine causes a scheduling of the subsets of operations on the hardware accelerator in accordance with the values of the configuration parameters. Upon receiving input data of the computation task, the routine may be called to cause the hardware accelerator to perform by the computation task in accordance with the scheduling.
1. A computer implemented method for executing a computation task composed of at least one set of operations, the method comprising:
determining subsets of pipelineable operations of the set of operations in accordance with a pipelining scheme;
creating a single routine for enabling execution of the determined subsets of operations by a hardware accelerator, the routine having as arguments a value indicative of input data and values of configuration parameters of the computation task, wherein a call of the routine causes a scheduling of the subsets of operations on the hardware accelerator in accordance with the values of the configuration parameters;
upon receiving input data of the computation task, calling the routine, thereby causing the hardware accelerator to perform the computation task in accordance with the scheduling.
2. The method of claim 1 , the computation task comprising any one of: training a deep neural network, performing inference using a trained neural network, matrix vector multiplication and matrix-matrix multiplication.
3. The method of claim 2 , wherein the at least one set of operations comprises a first set of operations for forward propagation, a second set of operations for backward propagation and a third set of operations for both forward and backward propagation of the training; the method further comprising: generating for each of the first, second and third sets of operations a respective composite operation, wherein calling the routine comprises performing a single application programming interface (API) call for each composite operation of at least part of the generated composition operations.
4. The method of claim 2 , the configuration parameters comprising parameters descriptive of the structure of the deep neural network and parameters required for configuring the training of the deep neural network.
5. The method of claim 1 , further comprising providing an application programming interface, API, to the hardware accelerator, and creating the routine using the API, wherein the calling of the routine is a single API call.
6. The method of claim 1 , further comprising providing a computation graph descriptive of the computation task, the computation task involving a deep neural network, determining the at least one set of operations by parsing the computation graph for identifying the at least one set of operations using nodes of the computation graph, generating a user graph such that each set of the at least one set of operations is represented by a node of the user graph, wherein calling the routine comprises identifying each node of the user graph that represents a respective set of operations, and for each identified node performing a single API call for the set of operation represented by the identified node.
7. The method of claim 1 , further comprising receiving an output from the hardware accelerator indicative of the result of the computation task.
8. The method of claim 1 , wherein the pipelining scheme is provided such that each of the subsets of operations comprises operations independent from each other that can be executed in parallel.
9. The method of claim 1 , wherein the hardware accelerator operates in accordance with the pipelining scheme using memristor crossbar arrays, wherein the subsets of pipelineable operations are determined such that each subset of operations of the subsets may be performed in parallel on different crossbar arrays of the memristor crossbar arrays.
10. The method of claim 1 , wherein the hardware accelerator operates in accordance with the pipelining scheme using memristor crossbar arrays, the computation task comprising training a deep neural network, wherein each layer of the deep neural network is associated with two crossbar arrays of the hardware accelerator, the two crossbar arrays comprising identical values, wherein causing the hardware accelerator to perform the computation task comprises: for each layer of the deep neural network using one crossbar array of the two crossbar array for forward propagation and the other crossbar array only for the backward propagation.
11. A computer program product comprising a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code configured for:
determining subsets of pipelineable operations of at least one set of operations of a computation task in accordance with a pipelining scheme;
creating a single routine for enabling execution of the determined subsets of operations by a hardware accelerator, the routine having as arguments a value indicative of input data and values of configuration parameters of the computation task, wherein a call of the routine causes a scheduling of the subsets of operations on the hardware accelerator in accordance with the values of the configuration parameters;
upon receiving input data of the computation task calling the routine, thereby causing the hardware accelerator to perform the computation task in accordance with the scheduling.
12. The computer program product of claim 11 , the computation task comprising one of: training a deep neural network, matrix vector multiplication and matrix-matrix multiplication.
13. The computer program product of claim 12 , at least one set of operations comprises a first set of operations for forward propagation, a second set of operations for backward propagation and a third set of operations for both forward and backward propagation of the training, the method comprising: generating for each of the first, second and third sets of operations a respective composite operation, wherein calling the routine comprises performing a single application programming interface (API) call for each composite operation of at least part of the generated composition operations.
14. The computer program product of claim 12 , the configuration parameters comprising parameters descriptive of the structure of the deep neural network and parameters required for configuring the training of the deep neural network.
15. The computer program product of claim 11 , further configured for creating the routine using an API to the hardware accelerator.
16. The computer program product of claim 11 , further configured for determining the at least one set of pipelineable operations by parsing a computation graph for identifying the at least one set of operations using nodes of the computation graph, generating a user graph such that each set of the at least one set of operations is represented by a node of the user graph, wherein calling the routine comprises identifying each node of the user graph that represents a respective set of operations, and for each identified node performing a single API call for the set of operation represented by the identified node.
17. The computer program product of claim 11 , further configured for receiving an output from the hardware accelerator indicative of the result of the computation task.
18. The computer program product of claim 11 , the pipelining scheme being provided such that each subset of the subsets comprises operations independent from each other that can be executed in parallel.
19. A computer system comprising:
at least one processor; and
at least one memory comprising computer-readable program instructions, wherein loading the computer-readable program instructions into the at least one processor and executing the computer-readable program instructions by the at least one processor causes the computer system to perform:
determining subsets of pipelineable operations of at least one set of operations of a computation task in accordance with a pipelining scheme;
creating a single routine for enabling execution of the determined subsets of operations by a hardware accelerator, the routine having as arguments a value indicative of input data and values of configuration parameters of the computation task, wherein a call of the routine causes a scheduling of the subsets of operations on the hardware accelerator in accordance with the values of the configuration parameters;
upon receiving input data of the computation task calling the routine, thereby causing the hardware accelerator to perform the computation task in accordance with the scheduling.