COMMUNICATION PROTOCOL, AND A METHOD THEREOF FOR ACCELERATING ARTIFICIAL INTELLIGENCE PROCESSING TASKS
A system and method for communicating artificial intelligence (AI) tasks between AI resources are provided. The method comprises establishing a connection between a first AI resource and a second AI resource; encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol, wherein the at least one request data frame is encapsulated at the first AI resource; and transporting the at least one request data frame over a network using a transport protocol to the second AI resource, wherein the transport protocol provisions the transport characteristics of the AI task, and wherein the transport protocol is different than the communication protocol.
1 . A method for accelerating artificial intelligence (AI) task execution using a disaggregated computing infrastructure, comprising:
receiving from an AI client a request for an AI task to be performed, wherein the request is structured according to an AI over Fabric (AIoF) protocol;
performing a direct memory access (DMA) read operation over a network to retrieve input data for the AI task directly from client memory;
translating the DMA operation at an AIoF protocol level into a direct data transfer operation; and
transferring, based on the direct data transfer operation, a request data frame from the AI client to an AI server over a network, wherein the AI server is configured to process the AI task based on a received data.
2 . The method of claim 1 , further comprising:
receiving, from the AI server, a response data frame including results of the AI task, wherein the response frame is structured according to the AIoF protocol; and
writing the response data directly to the client memory using a DMA write operation over the network.
3 . The method of claim 1 , wherein the AI task includes metadata defining task characteristics, memory address ranges, and scatter-gather list (SGL) descriptors associated with client memory.
4 . The method of claim 3 , further comprising:
performing the DMA operation, based on a received memory addresses and SGL descriptors.
5 . The method of claim 1 , wherein the request data frame includes a header and a payload, wherein the header includes job metadata, priority, input and output scatter-gather descriptors, and AI compute graph identifiers.
6 . The method of claim 2 , further comprising:
implementing a flow control mechanism to transfer request data frames and a response data frame.
7 . The method of claim 6 , wherein the flow control mechanism includes any one of:
message-based flow control and a credit-based flow control.
8 . The method of claim 9 , further comprising:
encrypting the request and response data frames using a per-client security association.
9 . The method of claim 1 , wherein the AIoF protocol is defined with a shared memory over network, and where the method further comprises:
copying data from the memory of the AI client to a network-attached artificial intelligence accelerator (NA-AIA) memory via the network, wherein the data includes AI model and application data.
10 . The method of claim 1 , wherein transferring the request data frame over the network is performed using a transport control protocol (TCP).
11 . A non-transitory computer-readable medium storing a set of instructions for accelerating artificial intelligence (AI) task execution using a disaggregated computing infrastructure, the set of instructions comprising:
one or more instructions that, when executed by one or more processing circuitries of a device, cause the device to:
receive from an AI client a request for an AI task to be performed, wherein the request is structured according to an AI over Fabric (AIoF) protocol;
perform a direct memory access (DMA) read operation over a network to retrieve input data for the AI task directly from client memory;
translate the DMA operation at an AIoF protocol level into a direct data transfer operation; and
transfer, based on the direct data transfer operation, a request data frame from the AI client to an AI server over a network, wherein the AI server is configured to process the AI task based on a received data.
12 . A system for accelerating artificial intelligence (AI) task execution using a disaggregated computing infrastructure comprising:
a processing circuitry;
a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:
receive from an AI client a request for an AI task to be performed, wherein the request is structured according to an AI over Fabric (AIoF) protocol;
perform a direct memory access (DMA) read operation over a network to retrieve input data for the AI task directly from client memory;
translate the DMA operation at an AIoF protocol level into a direct data transfer operation; and
transfer, based on the direct data transfer operation, a request data frame from the AI client to an AI server over a network, wherein the AI server is configured to process the AI task based on a received data.
13 . The system of claim 12 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:
receive, from the AI server, a response data frame including results of the AI task, wherein the response frame is structured according to the AIoF protocol; and
write the response data directly to the client memory using a DMA write operation over the network.
14 . The system of claim 13 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:
implement a flow control mechanism to transfer request data frames and a response data frame.
15 . The system of claim 14 , wherein the flow control mechanism includes any one of:
message-based flow control and a credit-based flow control.
16 . The system of claim 12 , wherein the AI task includes metadata defining task characteristics, memory address ranges, and scatter-gather list (SGL) descriptors associated with client memory.
17 . The system of claim 16 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:
perform the DMA operation, based on a received memory addresses and SGL descriptors.
18 . The system of claim 12 , wherein the request data frame includes a header and a payload, the header includes job metadata, priority, input and output scatter-gather descriptors, and AI compute graph identifiers.
19 . The system of claim 12 , wherein the AIoF protocol is defined with a shared memory over network, and where the method further comprises:
copying data from the memory of the AI client to a network-attached artificial intelligence accelerator (NA-AIA) memory via the network, wherein the data includes AI model and application data.
20 . The system of claim 19 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:
encrypt the request and response data frames using a per-client security association.
21 . The system of claim 12 , wherein transferring the request data frame over the network is performed using a transport control protocol (TCP).