Full asynchronous execution queue for accelerator hardware
A method for providing an asynchronous execution queue for accelerator hardware includes replacing a malloc operation in an execution queue to be sent to an accelerator with an asynchronous malloc operation that returns a unique reference pointer. Execution of the asynchronous malloc operation in the execution queue by the accelerator allocates a requested memory size and adds an entry to a look-up table accessible by the accelerator that maps the reference pointer to a corresponding memory address.
1. A method for providing an asynchronous execution queue for accelerator hardware, the method comprising:
replacing a malloc operation in an execution queue to be sent to an accelerator with an asynchronous malloc operation that returns a unique reference pointer, whereby execution of the asynchronous malloc operation in the execution queue by the accelerator allocates a requested memory size and adds an entry to a look-up table accessible by the accelerator that maps the reference pointer to a corresponding memory address; and
replacing a free operation in the execution queue to be sent to the accelerator with an asynchronous free operation that uses the reference pointer and schedules a free instruction into the execution queue to be sent to the accelerator, whereby execution of the asynchronous free operation in the execution queue by the accelerator frees the corresponding memory address and removes the entry from the look-up table.
2. The method according to claim 1 , wherein the reference pointer includes a reference id and an offset.
3. The method according to claim 2 , wherein the reference pointer is 64 bits in which 32 bits comprise the reference id and 32 bits comprise the offset.
4. The method according to claim 1 , further comprising providing an asynchronous variadic operation which copies data in a variadic argument from original code in the execution queue into a variadic arguments stack and returns a memory offset, the variadic arguments stack being copied together to the accelerator along with the execution queue.
5. The method according to claim 1 , wherein the reference pointer is used by kernel calls and memcopy operations to determine the corresponding memory address using the look-up table.
6. The method according to claim 1 , wherein the accelerator is decoupled from a host of the accelerator and is housed in a separate enclosure together with other accelerators, the host being remotely connected to the accelerator enclosure using external interconnects.
7. The method according to claim 1 , wherein the look-up table is implemented in the accelerator by software or as an extension to a memory management unit (MMU) of the accelerator.
8. A system comprising one or more processors which, alone or in combination, are configured to provide for execution of the following steps:
replacing a malloc operation in an execution queue to be sent to an accelerator with an asynchronous malloc operation that returns a unique reference pointer, whereby execution of the asynchronous malloc operation in the execution queue by the accelerator allocates a requested memory size and adds an entry to a look-up table accessible by the accelerator that maps the reference pointer to a corresponding memory address; and
replacing a free operation in the execution queue to be sent to the accelerator with an asynchronous free operation that uses the reference pointer and schedules a free instruction into the execution queue to be sent to the accelerator, whereby execution of the asynchronous free operation in the execution queue by the accelerator frees the corresponding memory address and removes the entry from the look-up table.
9. The system according to claim 8 , wherein the system is decoupled from the accelerator, which is housed in a separate enclosure together with other accelerators, the system being remotely connected to the accelerator enclosure using external interconnects.
10. The system according to claim 8 , being configured to implement the look-up table in the accelerator using software.
11. The system according to claim 8 , wherein the look-up table is an extension to a memory management unit (MMU) of the accelerator.
12. A tangible, non-transitory computer-readable medium having instructions thereon which, upon being executed by one or more processors, alone or in combination, provide for execution of the following steps:
replacing a malloc operation in an execution queue to be sent to an accelerator with an asynchronous malloc operation that returns a unique reference pointer, whereby execution of the asynchronous malloc operation in the execution queue by the accelerator allocates a requested memory size and adds an entry to a look-up table accessible by the accelerator that maps the reference pointer to a corresponding memory address; and
replacing a free operation in the execution queue to be sent to the accelerator with an asynchronous free operation that uses the reference pointer and schedules a free instruction into the execution queue to be sent to the accelerator, whereby execution of the asynchronous free operation in the execution queue by the accelerator frees the corresponding memory address and removes the entry from the look-up table.
13. A method for providing an asynchronous execution queue for accelerator hardware, the method comprising:
replacing a malloc operation in an execution queue to be sent to an accelerator with an asynchronous malloc operation that returns a unique reference pointer, whereby execution of the asynchronous malloc operation in the execution queue by the accelerator allocates a requested memory size and adds an entry to a look-up table accessible by the accelerator that maps the reference pointer to a corresponding memory address,
wherein the reference pointer includes a reference id and an offset, and wherein the reference pointer is 64 bits in which 32 bits comprise the reference id and 32 bits comprise the offset.
14. A method for providing an asynchronous execution queue for accelerator hardware, the method comprising:
replacing a malloc operation in an execution queue to be sent to an accelerator with an asynchronous malloc operation that returns a unique reference pointer, whereby execution of the asynchronous malloc operation in the execution queue by the accelerator allocates a requested memory size and adds an entry to a look-up table accessible by the accelerator that maps the reference pointer to a corresponding memory address; and
providing an asynchronous variadic operation which copies data in a variadic argument from original code in the execution queue into a variadic arguments stack and returns a memory offset, the variadic arguments stack being copied together to the accelerator along with the execution queue.