In-network message aggregation for efficient small message transport
Aggregation of small payloads from multiple packets may improve bandwidth efficiency of a network, particularly a high-performance compute cluster with thousands of network endpoints and distributed data. Aggregation is context-based and a packet header is reduced because the common components that are shared by the aggregated messages are included once within the header. Execution contexts are explicitly created and destroyed by application programs. Each participating endpoint stores context-specific properties until the context is destroyed, so that the properties are not included in the header. Aggregation may be performed at different hierarchical levels by switches and/or endpoints.
1. A computer-implemented method, comprising:
identifying messages that share a context created by a source application program;
aggregating the identified messages for inclusion in a single packet;
constructing the single packet including the aggregated identified messages and an identifier for the context, wherein the identifier references properties needed by a destination endpoint to interpret at least one of the identified messages in the single packet and the properties are stored, by the source application program, at each destination endpoint that participates in the context;
extracting, at a switch within a network, a first message and a second message of the aggregated identified messages from the single packet, wherein the properties are irrelevant for routing the single packet through the switch;
transmitting the first message and the identifier towards a first destination endpoint; and
transmitting the second message and the identifier towards a second destination endpoint.
2. The computer-implemented method of claim 1 , further comprising:
receiving the first message at the first destination endpoint; and
using the identifier, accessing the properties stored in a table within the first destination endpoint.
3. The computer-implemented method of claim 1 , wherein the properties comprise at least one of properties of packets, response requirements, or notification mechanisms.
4. The computer-implemented method of claim 1 , wherein the properties comprise at least one of a number of processes in a collective, a location where at least one of the identified messages will be stored, or attributes associated with reliability, ordering, atomicity, or data type.
5. The computer-implemented method of claim 1 , wherein the first message is aggregated with other messages identified as sharing the context and included in a second packet constructed at the switch.
6. The computer-implemented method of claim 1 , wherein the single packet is constructed in response to determining that either a timer expired or a maximum number of the identified messages are aggregated.
7. The computer-implemented method of claim 6 , wherein at least one of the timer and the maximum number is adjusted as a processing workload changes.
8. The computer-implemented method of claim 6 , wherein the maximum number corresponds to the context and a second maximum number corresponds to an additional context.
9. The computer-implemented method of claim 1 , wherein the single packet is constructed in response to determining that an aggregation buffer is not available for an additional context.
10. The computer-implemented method of claim 1 , wherein constructing the single packet comprises encoding a common portion of addresses of destination endpoints for the aggregated identified messages into the single packet.
11. The computer-implemented method of claim 10 , wherein constructing the single packet further comprises encoding message-specific portions of the addresses of the destination endpoints for the aggregated identified messages into the single packet.
12. The computer-implemented method of claim 1 , wherein constructing the single packet comprises encoding a next hop for the aggregated identified messages into the single packet.
13. The computer-implemented method of claim 1 , wherein at least one of the steps of identifying, aggregating, and constructing is performed on a server or in a data center to generate content that is streamed to a user device.
14. The computer-implemented method of claim 1 , wherein at least one of the steps of identifying, aggregating, and constructing is performed within a cloud computing environment.
15. The computer-implemented method of claim 1 , wherein at least one of the steps of identifying, aggregating, and constructing is performed for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle.
16. The computer-implemented method of claim 1 , wherein at least one of the steps of identifying, aggregating, and constructing is performed on a virtual machine comprising a portion of a graphics processing unit.
17. A system, comprising:
a processor that is connected to a network, wherein the processor executes a source application program that creates a context;
a device within the network that is configured to:
identify messages that share the context;
aggregate the identified messages for inclusion in a single packet;
construct the single packet including the aggregated identified messages and an identifier for the context, wherein the identifier references properties needed by a destination endpoint to interpret at least one of the identified messages in the single packet and the properties are stored, by the source application program, at each destination endpoint that participates in the context;
extract, at a switch within a network, a first message and a second message of the aggregated identified messages from the single packet, wherein the properties are irrelevant for routing the single packet through the switch;
transmit the first message and the identifier towards a first destination endpoint; and
transmit the second message and the identifier towards a second destination endpoint.
18. The system of claim 17 , wherein the first destination endpoint is further configured to:
receive the first message; and
using the identifier, access the properties stored in a table within the first destination endpoint.
19. A non-transitory computer-readable media storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
identifying messages that share a context created by a source application program;
aggregating the identified messages for inclusion in a single packet;
constructing the single packet including the aggregated identified messages and an identifier for the context, wherein the identifier references properties needed by a destination endpoint to interpret at least one of the identified messages in the single packet and the properties are stored, by the source application program, at each destination endpoint that participates in the context;
extracting, at a switch within a network, a first message and a second message of the aggregated identified messages from the single packet, wherein the properties are irrelevant for routing the single packet through the switch;
transmitting the first message and the identifier towards a first destination endpoint; and
transmitting the second message and the identifier towards a second destination endpoint.
20. A computer-implemented method for operation of a device connected to a network, wherein a plurality of source endpoints and a plurality of destination endpoints are coupled to the network, and wherein a message includes a payload that is associated with a context of one or more contexts created by a source application program, and wherein the method comprises:
identifying messages that share a first context;
aggregating into a packet the identified messages and a context identifier corresponding to the first context, wherein the identifier references properties needed by a destination endpoint to interpret at least one of the identified messages in the single packet and the properties are stored, by the source application program, at each destination endpoint of the plurality of destination endpoints that participates in the context;
extracting, at a switch within a network, a first message and a second message of the aggregated identified messages from the single packet, wherein the properties are irrelevant for routing the single packet through the switch;
transmitting the first message and the identifier towards a first destination endpoint; and
transmitting the second message and the identifier towards a second destination endpoint.
21. A computer-implemented method of claim 20 , wherein the packet includes a common portion of addresses of each destination endpoint of the plurality of destination endpoints associated with the identified messages and message-specific portions of the addresses of each destination endpoint of the plurality of destination endpoints associated with the identified messages.