Increasing QoS throughput and efficiency through lazy byte batching
Described embodiments improve the performance of a computer network via selectively forwarding packets to bypass quality of service (QoS) processing, avoiding processing delays during critical periods of high demand, increasing throughput and efficiency may be increased by sacrificing a small amount of QoS accuracy. QoS processing may be applied to a subset of packets of a flow or connection, referred to herein as “lazy” processing or lazy byte batching. Packets that bypass QoS processing may be immediately forwarded with the same QoS settings as packets of the flow for which QoS processing is applied, resulting in tremendous overhead savings with only minimal decline in accuracy.
1. A method for improving network efficiency via throughput credits, comprising:
receiving, by a device, a plurality of packets of a flow;
determining, by the device, that a throughput credit for the flow does not exceed a predetermined threshold;
transmitting a first subset of the plurality of packets to bypass quality of service (QoS) processing, by the device, responsive to the determination that the throughput credit for the flow does not exceed the predetermined threshold, the throughput credit incremented for each transmitted packet of the first subset;
determining, by the device, that the throughput credit for the flow exceeds the predetermined threshold, responsive to incrementing the throughput credit for each transmitted packet of the first subset; and
applying QoS processing to a second subset of the plurality of packets, by the device, responsive to the determination that the throughput credit for the flow exceeds the predetermined threshold, the throughput credit reset upon applying QoS processing to the second subset, by:
placing the second subset of the plurality of packets in one or more QoS queues according to a virtual size of each packet of the second subset of the plurality of packets, each virtual size comprising a sum of the size of the corresponding packet and the throughput credit for the flow incremented for each transmitted packet of the first subset.
2. The method of claim 1 , wherein the second subset comprises a single packet.
3. The method of claim 1 , wherein the QoS processing is based on a virtual packet size equal to a sum of the sizes of the plurality of packets.
4. The method of claim 1 , further comprising, for each transmitted packet of the first subset, incrementing the throughput credit by a size of said packet.
5. The method of claim 1 , wherein transmitting the first subset of the plurality of packets further comprises placing each of the first subset of the plurality of packets in an output transmission buffer of the device, the output transmission buffer separate from one or more QoS queues.
6. The method of claim 1 , wherein selecting the QoS queue further comprises scheduling transmission of packets of the flow and packets of at least one additional flow of a plurality of flows according to the virtual size of the flow and a virtual size of each at least one additional flow.
7. The method of claim 1 , wherein transmitting the first subset of the plurality of packets to bypass QoS processing is performed responsive to determining, by the device, that a throughput for the flow exceeds a second threshold.
8. A system for improving network efficiency via throughput credits, comprising:
a device, comprising a network interface, a quality of service (QoS) processor, and a memory storing a throughput credit for a flow;
wherein the network interface is configured to receive a plurality of packets of the flow; and
wherein the QoS processor is configured to determine that the throughput credit for the flow does not exceed a predetermined threshold; and
wherein the network interface is further configured to transmit a first subset of the plurality of packets to bypass the QoS processor, responsive to the determination that the throughput credit for the flow does not exceed the predetermined threshold, the throughput credit incremented for each transmitted packet of the first subset; and
wherein the QoS processor is further configured to:
determine that the throughput credit for the flow exceeds the predetermined threshold, responsive to incrementing the throughput credit for each transmitted packet of the first subset
apply QoS processing to a second subset of the plurality of packets, responsive to the determination that the throughput credit for the flow exceeds the predetermined threshold, the throughput credit reset upon applying QoS processing to the second subset, by:
placing the second subset of the plurality of packets in one or more QoS queues according to a virtual size of each packet of the second subset of the plurality of packets, each virtual size comprising a sum of the size of the corresponding packet and the throughput credit for the flow incremented for each transmitted packet of the first subset.
9. The system of claim 8 , wherein the second subset comprises a single packet.
10. The system of claim 8 , wherein the QoS processing is based on a virtual packet size equal to a sum of the sizes of the plurality of packets.
11. The system of claim 8 , wherein the device is configured to, for each transmitted packet of the first subset, increment the throughput credit by a size of said packet.
12. The system of claim 8 , wherein the device is configured to place each of the first subset of the plurality of packets in an output transmission buffer of the network interface, the output transmission buffer separate from one or more QoS queues.
13. The system of claim 8 , wherein the QoS processor is further configured to schedule transmission of packets of the flow and packets of at least one additional flow of a plurality of flows according to the virtual size of the flow and a virtual size of each at least one additional flow.
14. The system of claim 8 , wherein the device is configured to determine that a throughput for the flow exceeds a second threshold; and wherein transmitting the first subset of the plurality of packets to bypass QoS processing is performed responsive to the determination.
15. A non-transitory computer readable medium comprising instructions that, when executed by a processor of a computing device, cause the device to:
receive a plurality of packets of a flow;
determine that a throughput credit for the flow does not exceed a predetermined threshold;
transmit a first subset of the plurality of packets to bypass a QoS processor of the computing device, responsive to the determination that the throughput credit for the flow does not exceed the predetermined threshold, the throughput credit incremented for each transmitted packet of the first subset;
determine that the throughput credit for the flow exceeds the predetermined threshold, responsive to incrementing the throughput credit for each transmitted packet of the first subset; and
apply QoS processing to a second subset of the plurality of packets, responsive to the determination that the throughput credit for the flow exceeds the predetermined threshold, the throughput credit reset upon applying QoS processing to the second subset, by:
placing the second subset of the plurality of packets in one or more QoS queues according to a virtual size of each packet of the second subset of the plurality of packets, each virtual size comprising a sum of the size of the corresponding packet and the throughput credit for the flow incremented for each transmitted packet of the first subset.
16. The computer-readable medium of claim 15 , further comprising instructions that cause the device to place each of the first subset of the plurality of packets in an output transmission buffer of the device, the output transmission buffer separate from one or more QoS queues.
17. The computer-readable medium of claim 15 , further comprising instructions that cause the device to apply QoS processing based on a virtual packet size equal to a sum of the sizes of the plurality of packets.
18. The computer-readable medium of claim 15 , further comprising instructions that cause the device to schedule transmission of packets of the flow and packets of at least one additional flow of a plurality of flows according to the virtual size of the flow and a virtual size of each at least one additional flow.
19. The computer-readable medium of claim 15 , further comprising instructions that cause the device to determine that a throughput for the flow exceeds a second threshold; and transmit the first subset of the plurality of packets to bypass QoS processing, responsive to the determination.