Communication protocol, and a method thereof for accelerating artificial intelligence processing tasks
A method and system for communicating artificial intelligence (AI) tasks between AI resources are provided. The method includes establishing a connection between a first AI resource and a second AI resource; encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol, wherein the at least one request data frame is encapsulated at the first AI resource; transporting the at least one request data frame over a network using a transport protocol to the second AI resource, wherein the transport protocol is different than the communication protocol; and using a credit-based flow control mechanism to transfer messages between the first AI resource and the second AI resource over the transport protocol, thereby avoiding congestion on compute resources.
1. A method for communicating artificial intelligence (AI) tasks between AI resources, comprising:
establishing a connection between a first AI resource and a second AI resource;
encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol, wherein the at least one request data frame is encapsulated at the first AI resource;
transporting the at least one request data frame over a network using a transport protocol to the second AI resource, wherein the transport protocol is different than the communication protocol; and
using a credit-based flow control mechanism to transfer messages between the first AI resource and the second AI resource over the transport protocol, thereby avoiding congestion on compute resources;
wherein the first AI resource is an AI client and the second AI resource is at least one AI server.
2. The method of claim 1 , wherein the transport protocol provisions transport characteristics of the AI task.
3. The method of claim 1 , further comprising:
transferring using the credit-based flow control mechanism credit information, wherein the credit information controls scheduling and availability of the compute resources.
4. The method of claim 3 , wherein the credit information is included in a header portion of a message of the transport protocol.
5. The method of claim 4 , wherein the credit information includes a number of credits allocated per AI job, wherein the AI task includes one or more AI jobs.
6. The method of claim 5 , wherein the credits are allocated to the client AI.
7. The method of claim 2 , wherein the at least one AI server is configured to accelerate execution of the AI task.
8. The method of claim 1 , wherein the transport protocol is any one of: a Transmission Control Protocol (TCP), a remote direct memory access (RDMA), a RDMA over converged Ethernet (RoCE), NVMe or NVMeoF, and an InfiniBand.
9. A system for communicating artificial intelligence (AI) tasks between AI resources comprising:
one or more processors configured to:
establish a connection between a first AI resource and a second AI resource;
encapsulate a request to process an AI task in at least one request data frame compliant with a communication protocol, wherein the at least one request data frame is encapsulated at the first AI resource;
transport the at least one request data frame over a network using a transport protocol to the second AI resource, wherein the transport protocol is different than the communication protocol; and
use a credit-based flow control mechanism to transfer messages between the first AI resource and the second AI resource over the transport protocol, thereby avoiding congestion on compute resources;
wherein the first AI resource is an AI client and the second AI resource is at least one AI server.
10. The system of claim 9 , wherein the transport protocol provisions transport characteristics of the AI task.
11. The system of claim 10 , wherein at least one AI server configured to accelerate execution of the AI task.
12. The system of claim 9 , wherein the one or more processors are further configured to:
transfer using the credit-based flow control mechanism credit information, wherein the credit information controls scheduling and availability of the compute resources.
13. The system of claim 12 , wherein the credit information is included in a header portion of a message of the transport protocol.
14. The system of claim 13 , wherein the credit information includes a number of credits allocated per AI job, the AI task includes one or more AI jobs.
15. The system of claim 14 , wherein the first AI resource is a client AI, and the credits are allocated to the client AI.
16. The system of claim 9 , wherein the transport protocol is any one of:
a transmission control protocol (TCP), a remote direct memory access (RDMA), a RDMA over converged Ethernet (RoCE), NVMe or NVMeoF, and an InfiniBand.
17. A non-transitory computer-readable medium storing a set of instructions for communicating artificial intelligence (AI) tasks between AI resources, the set of instructions comprising:
one or more instructions that, when executed by one or more processors of a device, cause the device to:
establish a connection between a first AI resource and a second AI resource;
encapsulate a request to process an AI task in at least one request data frame compliant with a communication protocol, wherein the at least one request data frame is encapsulated at the first AI resource;
transport the at least one request data frame over a network using a transport protocol to the second AI resource, wherein the transport protocol is different than the communication protocol; and
use a credit-based flow control mechanism to transfer messages between the first AI resource and the second AI resource over the transport protocol, thereby avoiding congestion on compute resources;
wherein the first AI resource is an AI client and the second AI resource is at least one AI server.