Control plane proxy server for managing use of network connecting endpoint processing units
Some embodiments provide a method of managing communication between EPUs connected through a network having multiple forwarding elements. At a set of configuration servers, the method assigns, for each EPU, a CP proxy server to configure the network interface of the EPU to forward data messages through the network. The method distributes configuration data to CP proxy servers. Each particular CP proxy server (i) through one or more forwarding elements and from the network interface of each EPU assigned to the particular CP proxy server, receives requests for scheduling parameters for a data message flow that the network interface has to forward through the network, (ii) uses the received configuration data to generate the scheduling parameters for each EPU, and (iii) provides, through one or more forwarding elements, the generated scheduling parameters to the network interface of each EPU to use to forward the data message flow through the network.
1 . A method of managing communication between endpoint processing units (EPUs) connected through a network comprising a plurality of forwarding elements, the method comprising:
at a set of one or more configuration servers:
assigning, for each EPU, one CP proxy server to configure a network interface of the EPU to forward data messages through the network; and
distributing configuration data to CP proxy servers, each particular CP proxy server (i) through one or more forwarding elements, receiving, from the network interface of each EPU assigned to the particular CP proxy server, requests for a set of scheduling parameters for a data message flow that the network interface has to forward through the network, (ii) using the distributed configuration data to generate the set of scheduling parameters for each EPU, and (iii) providing, through one or more forwarding elements, the generated set of scheduling parameters to the network interface of each EPU to use to forward the data message flow through the network.
2 . The method of claim 1 , wherein the EPUs are graphics processing units (GPUs).
3 . The method of claim 1 , wherein the EPUs comprise at least one of graphics processing units (GPUs), tensor processing units (TPUs) and central processing units (CPUs).
4 . The method of claim 1 , wherein for each request sent from a particular network interface, the particular CP proxy server receives the request from a forwarding element that is a last hop in a path from the particular network interface to a destination of the data message flow for which the particular network interface sends the request.
5 . The method of claim 4 , wherein the particular CP proxy server provides the generated set of scheduling parameters for each request to the forwarding element from which the particular CP proxy server receives the request, in order for the forwarding element to forward the generated set of scheduling parameters in a reply back to the particular network interface that sent the request.
6 . The method of claim 5 , wherein each request from each particular network interface and each reply back to the particular network interface is sent as an in-band control message through the forwarding elements that form the network connecting the EPUs.
7 . The method of claim 4 , wherein each forwarding element has a data plane circuit that (i) intercepts each request when the forwarding element is the last hop for that request and (ii) for said interception, has a policy-based rule to identify all requests addressed to a destination that is one hop away from the forwarding element to a CP proxy server to process as a control plane message.
8 . The method of claim 1 , wherein based on the distributed configuration data, each CP proxy server provides the set of scheduling parameters for a data message flow to each network interface.
9 . The method of claim 8 , wherein each set of scheduling parameters for each EPU network interface controls at least one of a launch time and rate for forwarding a data message flow storing a result computed by the EPU to the result's destinations in the network.
10 . The method of claim 8 , wherein each set of scheduling parameters is part of the distributed configuration data.
11 . The method of claim 8 , wherein each CP proxy server generates at least a subset of the scheduling parameters from the distributed configuration data.
12 . A non-transitory machine readable medium storing a program that when executed by at least one processor manages communication between graphics processing units (GPUs) connected through a network comprising a plurality of forwarding elements, the program comprising sets of instructions for:
assigning, for each GPU, one CP proxy server to configure a network interface of the GPU to forward data messages through the network; and
distributing configuration data to CP proxy servers, each particular CP proxy server (i) through one or more forwarding elements, receiving, from the network interface of each GPU assigned to the particular CP proxy server, requests for a set of scheduling parameters for a data message flow that the network interface has to forward through the network, (ii) using the distributed configuration data to generate the set of scheduling parameters for each GPU, and (iii) providing, through one or more forwarding elements, the generated set of scheduling parameters to the network interface of each GPU to use to forward the data message flow through the network.
13 . The non-transitory machine readable medium of claim 12 , wherein for each request sent from a particular network interface, the particular CP proxy server receives the request from a forwarding element that is a last hop in a path from the particular network interface to a destination of the data message flow for which the particular network interface sends the request.
14 . The non-transitory machine readable medium of claim 13 , wherein the particular CP proxy server provides the generated set of scheduling parameters for each request to the forwarding element from which the particular CP proxy server receives the request, in order for the forwarding element to forward the generated set of scheduling parameters in a reply back to the particular network interface that sent the request.
15 . The non-transitory machine readable medium of claim 14 , wherein each request from each particular network interface and each reply back to the particular network interface is sent as an in-band control message through the forwarding elements that form the network connecting the GPUs.
16 . The non-transitory machine readable medium of claim 13 , wherein each forwarding element has a data plane circuit that (i) intercepts each request when the forwarding element is the last hop for that request and (ii) for said interception, has a policy-based rule to identify all requests addressed to a destination that is one hop away from the forwarding element to a CP proxy server to process as a control plane message.
17 . The non-transitory machine readable medium of claim 12 , wherein based on the distributed configuration data, each CP proxy server provides the set of scheduling parameters for a data message flow to each network interface.
18 . The non-transitory machine readable medium of claim 17 , wherein each set of scheduling parameters for each GPU network interface controls at least one of a launch time and rate for forwarding a data message flow storing a result computed by the GPU to the result's destinations in the network.
19 . The non-transitory machine readable medium of claim 17 , wherein each set of scheduling parameters is part of the distributed configuration data.