Systems and methods for distributing work between a host and an accelerator using a shared memory
Provided is a method for data processing, the method including generating, by a host, a work request in a queue in a shared memory, reading, by an accelerator circuit, the work request from the queue in the shared memory, the shared memory being a physical memory that is common to the host and the accelerator circuit, and performing, by the accelerator circuit, an operation on data in the shared memory based on the work request.
1 . A method for data processing, the method comprising:
generating, by a host, a work request in a queue in a shared memory, the work request being associated with at least one of branchy code, a system call, an exception, or a network message;
reading, by an accelerator circuit, the work request from the queue in the shared memory, the shared memory being a physical memory that is common to the host and the accelerator circuit;
performing, by the accelerator circuit, an operation on data in the shared memory based on the work request by performing a function, associated with the operation, on the data, the function being located in the shared memory and being identified by the work request; and
processing, by the host, a reverse offload request from the accelerator circuit based on the work request being associated with the at least one of the branchy code, the system call, the exception, or the network message.
2 . The method of claim 1 , wherein a latency associated with processing second data in the shared memory by the host and by the accelerator circuit is less than a total latency associated with processing the second data, by the host, in a first location associated with the host and processing the second data, by the accelerator circuit, in a second location associated with the accelerator circuit, the first location and the second location being coupled via a network connection, at least one of the first location or the second location being external to the shared memory,
the processing of the second data in the shared memory by the host and by the accelerator circuit comprising performing, by the host, a first task associated with the operation, and performing, by the accelerator circuit, the function,
the processing of the second data, by the host, in the first location comprising performing, by the host, the first task associated with the operation in the first location, and
the processing of the second data, by the accelerator circuit, in the second location comprising performing, by the accelerator circuit, the function in the second location.
3 . The method of claim 1 , wherein:
the work request comprises a pointer indicating a location of a function in the shared memory, the function being associated with the operation; and
the accelerator circuit performs the function to process the work request.
4 . The method of claim 3 , wherein:
the work request comprises a pointer indicating a location of the data in the shared memory; and
the accelerator circuit performs the function on the data to process the work request.
5 . The method of claim 1 , wherein the work request comprises a compute kernel.
6 . The method of claim 1 , wherein the shared memory comprises cacheable memory comprising high-bandwidth memory (HBM).
7 . The method of claim 1 , wherein the accelerator circuit is a first accelerator circuit of two or more accelerator circuits that process work requests generated by the host; and
the method further comprises:
processing, by the host, work-request result data from the first accelerator circuit; and
processing, by the host, work-request result data from a second accelerator circuit of the two or more accelerator circuits.
8 . The method of claim 1 , further comprising generating, by the accelerator circuit, a completion notification in the shared memory.
9 . The method of claim 1 , further comprising processing, by the host, a reverse offload request from the accelerator circuit based on the work request being associated with at least one of branchy code, a system call, an exception, or a network message.
10 . A system for data processing, comprising:
a host;
an accelerator circuit; and
a shared memory communicably coupled to the host and to the accelerator circuit, wherein:
the host is configured to generate a work request in a queue in the shared memory, the work request being associated with at least one of branchy code, a system call, an exception, or a network message;
the accelerator circuit is configured to read the work request from the queue in the shared memory, the shared memory being a physical memory that is common to the host and the accelerator circuit;
the accelerator circuit is configured to perform an operation on data in the shared memory based on the work request by performing a function, associated with the operation, on the data, the function being located in the shared memory and being identified by the work request; and
the host is configured to process a reverse offload request from the accelerator circuit based on the work request being associated with the at least one of the branchy code, the system call, the exception, or the network message.
11 . The system of claim 10 , wherein:
the work request comprises a pointer indicating a location of a function in the shared memory, the function being associated with the operation; and
the accelerator circuit performs the function to process the work request.
12 . The system of claim 11 , wherein:
the work request comprises a pointer indicating a location of the data in the shared memory; and
the accelerator circuit performs the function on the data to process the work request.
13 . The system of claim 10 , wherein the work request comprises a compute kernel.
14 . The system of claim 10 , wherein the shared memory comprises cacheable memory comprising high-bandwidth memory (HBM).
15 . The system of claim 10 , wherein:
the accelerator circuit is a first accelerator circuit of two or more accelerator circuits that process work requests generated by the host; and
the host is configured to:
process work-request result data from the first accelerator circuit; and
process work-request result data from a second accelerator circuit of the two or more accelerator circuits.
16 . The system of claim 10 , wherein the accelerator circuit is configured to generate a completion notification in the shared memory.
17 . The system of claim 10 , wherein the host is configured to process a reverse-offload request from the accelerator circuit based on the work request being associated with at least one of branchy code, a system call, an exception, or a network message.
18 . A device for data processing, comprising:
a processing circuit; and
memory storing instructions:
which, when executed by the processing circuit, cause the processing circuit to perform:
reading a work request, generated by a host, from a queue in a shared memory, the shared memory being a physical memory that is common to the host and the processing circuit, and the work request being associated with at least one of branchy code, a system call, an exception, or a network message; and
performing an operation on data in the shared memory based on the work request by performing a function, associated with the operation, on the data, the function being located in the shared memory and being identified by the work request; and
which, when executed by the host, cause the host to perform:
processing a reverse offload request based on the work request being associated with the at least one of the branchy code, the system call, the exception, or the network message.
19 . The device of claim 18 , wherein:
the work request comprises a pointer indicating a location of a function in the shared memory, the function being associated with the operation; and
the processing circuit performs the function to process the work request.
20 . The device of claim 19 , wherein:
the work request comprises a pointer indicating a location of the data in the shared memory; and
the processing circuit performs the function on the data to process the work request.