Increased packet scheduling throughput and efficiency using úber batching
Described embodiments improve the performance of a computer network via selectively forwarding packets to bypass quality of service (QoS) processing, avoiding processing delays during critical periods of high demand, increasing throughput and efficiency may be increased by sacrificing a small amount of QoS accuracy. QoS processing may be applied to a subset of packets of a flow or connection, referred to herein as “lazy” processing or lazy byte batching. Packets that bypass QoS processing may be immediately forwarded with the same QoS settings as packets of the flow for which QoS processing is applied, resulting in tremendous overhead savings with only minimal decline in accuracy. In case of backlog, packets may be collected together into an aggregated or ‘uber’ packet, with QoS processing applied based on a virtual size of the aggregated packet.
1. A method for improving network efficiency, comprising:
receiving, by a device, a plurality of packets of a flow to be transmitted via a connection;
determining, by the device, that the flow is not backlogged;
transmitting, by the device, a first subset of the received plurality of packets of the flow, responsive to the determination that the flow is not backlogged;
subsequently determining, by the device, that the flow is backlogged;
aggregating, by the device, a second subset of the plurality of packets in a virtual aggregate packet having a virtual packet size equal to a sum of sizes of the second subset of the plurality of packets, responsive to the determination that the flow is backlogged;
determining, by the device, that a throughput credit for the virtual aggregate packet exceeds a threshold, the throughput credit based upon the virtual packet size;
applying quality of service (QoS) processing to the virtual aggregate packet, by the device, responsive to the throughput credit for the virtual aggregate packet exceeding the threshold; and
transmitting, by the device, the virtual aggregate packet.
2. The method of claim 1 , wherein the throughput credit is incremented for each packet of the second subset of the plurality of packets aggregated in the virtual aggregate packet; and wherein the throughput credit is reset upon applying QoS processing to the virtual aggregate packet.
3. The method of claim 1 , further comprising, for each packet of the second subset of the plurality of packets aggregated in the virtual aggregate packet, incrementing the throughput credit by a size of the packet.
4. The method of claim 1 , wherein aggregating the second subset of the plurality of packets in the virtual aggregate packet further comprises:
for each packet of the second subset of the plurality of packets:
determining that the throughput credit for the virtual aggregate packet does not exceed the threshold,
adding the packet to the virtual aggregate packet, responsive to the determination that the throughput credit is less than the threshold, and
incrementing the throughput credit based on the packet.
5. The method of claim 1 , wherein aggregating the second subset of the plurality of packets in the virtual aggregate packet further comprises:
determining that a first packet of the second subset of the plurality of packets is an initial packet of the virtual aggregate packet; and
responsive to the determination that the first packet is the initial packet, setting the virtual packet size of the virtual aggregate packet equal to a size of the first packet.
6. The method of claim 5 , wherein aggregating the second subset of the plurality of packets in the virtual aggregate packet further comprises:
determining that a second packet of the second subset of the plurality of packets is not the initial packet of the virtual aggregate packet; and
responsive to the determination that the second packet is not the initial packet, adding a size of the second packet to the virtual packet size of the virtual aggregate packet.
7. The method of claim 1 , further comprising concatenating each packet of the second subset of the plurality of packets in an aggregate packet.
8. The method of claim 1 , wherein the virtual aggregate packet comprises parameters of each packet of the second subset of the plurality of packets.
9. The method of claim 1 , further comprising:
subsequently receiving, by the device, an additional packet of the flow;
determining, by the device, that the flow is no longer backlogged;
determining, by the device, that a sum of the throughput credit and a size of the additional packet does not exceed the threshold; and
transmitting the additional packet via the connection while bypassing QoS processing of the device.
10. A system for improving network efficiency, comprising:
a device, comprising a network interface, a processor, and a memory storing a throughput credit for a flow;
wherein the network interface is configured to receive a plurality of packets of the flow; and
wherein the processor is configured to:
determine that the flow is not backlogged,
transmit a first subset of the received plurality of packets of the flow, responsive to the determination that the flow is not backlogged;
subsequently determine that the flow is backlogged,
aggregate a second subset of the plurality of packets in a virtual aggregate packet having a virtual packet size equal to a sum of sizes of the second subset of the plurality of packets, responsive to the determination that the flow is backlogged,
determine that a throughput credit for the virtual aggregate packet exceeds a threshold, the throughput credit based upon a number of packets added to the virtual aggregate packet,
apply quality of service (QoS) processing to the virtual aggregate packet, responsive to the throughput credit for the virtual aggregate packet exceeding the threshold, and
transmit the virtual aggregate packet.
11. The system of claim 10 , wherein the throughput credit is incremented for each packet of the second subset of the plurality of packets; and wherein the throughput credit is reset upon applying QoS processing to the virtual aggregate packet.
12. The system of claim 10 , wherein the processor is further configured to increment the throughput credit, for each packet of the second subset of the plurality of packets added to the virtual aggregate packet, by a size of the packet.
13. The system of claim 10 , wherein the processor is further configured to:
for each packet of the second subset of the plurality of packets:
determine that the throughput credit for the virtual aggregate packet does not exceed the threshold,
add the packet to the virtual aggregate packet, responsive to the determination that the throughput credit does not exceed the threshold, and
increment the throughput credit based on said packet.
14. The system of claim 10 , wherein the processor is further configured to:
determine that a first packet of the second subset of the plurality of packets is an initial packet of the virtual aggregate packet; and
responsive to the determination that the first packet is the initial packet, set the virtual packet size of the virtual aggregate packet equal to a size of the first packet.
15. The system of claim 14 , wherein the processor is further configured to:
determine that a second packet of the second subset of the plurality of packets is not the initial packet of the virtual aggregate packet; and
responsive to the determination that the second packet is not the initial packet, add a size of the second packet to the virtual packet size of the virtual aggregate packet.
16. The system of claim 10 , wherein the processor is further configured to concatenate each of the second subset of the plurality of packets to generate an aggregate packet.
17. The system of claim 10 , wherein the virtual aggregate packet comprises parameters of each packet of the second subset of the plurality of packets.
18. The system of claim 10 , wherein the network interface is further configured to subsequently receive an additional packet of the flow; and wherein the processor is further configured to:
identify that the flow is no longer backlogged;
determine, that a sum of the throughput credit and a size of the additional packet does not exceed the threshold, responsive to the identification; and
transmit the additional packet via the connection while bypassing QoS processing of the device, responsive to the determination that the sum of the throughput credit and the size of the additional packet does not exceed the threshold.