Layer 4 load aware load balancing
Load aware load balancing may be provided. Flow duration data associated with a plurality of flows associated with a plurality of servers may be obtained. Then a plurality of queue lengths respectively associated with the plurality of servers may be obtained. Next, a Shortest Expected Delay (SED) score may be determined for each of the plurality of servers based on the flow duration data and the plurality of queue lengths. A flow may then be assigned to a one of the plurality of servers having the lowest SED score.
1. A method comprising:
inferring, by a computing device, server processing speed data associated with a plurality of servers from flow duration data associated with a plurality of flows associated with the plurality of servers, wherein inferring the server processing speed data comprises using a Softmax normalization function on the flow duration data associated with the plurality of flows associated with the plurality of servers;
obtaining a plurality of queue lengths respectively associated with the plurality of servers;
determining a Shortest Expected Delay (SED) score for each of the plurality of servers from the inferred server processing speed data and the plurality of queue lengths; and
assigning a flow to the one of the plurality of servers having a lowest SED score.
2. The method of claim 1 , further comprising obtaining the flow duration data wherein obtaining the flow duration data comprises determining an average flow duration for each of the plurality of servers.
3. The method of claim 2 , wherein obtaining the flow duration data comprises deriving a normalization of the average flow duration for each of the plurality of servers.
4. The method of claim 3 , wherein obtaining the flow duration data comprises using a Kalman Filter on the normalization of the average flow duration for each of the plurality of servers.
5. The method of claim 1 , further comprising:
incrementing a one of the plurality of queue lengths when its corresponding server of the plurality of servers is assigned a new flow; and
decrementing the one of the plurality of queue lengths when a flow ends on its corresponding server of the plurality of servers.
6. The method of claim 1 , further comprising refreshing the server processing speed data periodically.
7. The method of claim 6 , wherein refreshing the server processing speed data periodically comprises refreshing the server processing speed data every 200 ms.
8. A system comprising:
a memory storage; and
a processing unit coupled to the memory storage, wherein the processing unit is operative to:
infer server processing speed data associated with a plurality of servers from flow duration data associated with a plurality of flows associated with the plurality of servers, wherein the processing unit being operative to infer the server processing speed data comprises the processing unit being operative to use a Softmax normalization function on the flow duration data associated with the plurality of flows associated with the plurality of servers;
obtain a plurality of queue lengths respectively associated with the plurality of servers;
determine a Shortest Expected Delay (SED) score for each of the plurality of servers from the inferred server processing speed data and the plurality of queue lengths; and
assign a flow to the one of the plurality of servers having a lowest SED score.
9. The system of claim 8 , wherein the processing unit is further operative to obtain the flow duration data wherein the processing unit being operative to obtain the flow duration data comprises the processing unit being operative to determine an average flow duration for each of the plurality of servers.
10. The system of claim 9 , wherein the processing unit being operative to obtain the flow duration data comprises the processing unit being operative to derive a normalization of the average flow duration for each of the plurality of servers.
11. The system of claim 10 , wherein the processing unit being operative to obtain the flow duration data comprises the processing unit being operative to use a Kalman Filter on the normalization of the average flow duration for each of the plurality of servers.
12. The system of claim 8 , comprising the processing unit being further operative to:
increment a one of the plurality of queue lengths when its corresponding server of the plurality of servers is assigned a new flow; and
decrement the one of the plurality of queue lengths when a flow ends on its corresponding server of the plurality of servers.
13. A non-transitory computer-readable medium that stores a set of instructions which when executed by a processor perform a method executed by the set of instructions comprising:
inferring, by a computing device, server processing speed data associated with a plurality of servers from flow duration data associated with a plurality of flows associated with the plurality of servers, wherein inferring the server processing speed data comprises using a Softmax normalization function on the flow duration data associated with the plurality of flows associated with the plurality of servers;
obtaining a plurality of queue lengths respectively associated with the plurality of servers;
determining a Shortest Expected Delay (SED) score for each of the plurality of servers from the inferred server processing speed data and the plurality of queue lengths; and
assigning a flow to the one of the plurality of servers having a lowest SED score.
14. The non-transitory computer-readable medium of claim 13 , further comprising obtaining the flow duration data wherein obtaining the flow data comprises determining an average flow duration for each of the plurality of servers.
15. The non-transitory computer-readable medium of claim 14 , wherein obtaining the flow duration data comprises deriving a normalization of the average flow duration for each of the plurality of servers.
16. The non-transitory computer-readable medium of claim 15 , wherein obtaining the flow duration data comprises using a Kalman Filter on the normalization of the average flow duration for each of the plurality of servers.
17. The non-transitory computer-readable medium of claim 13 , further comprising: incrementing a one of the plurality of queue lengths when its corresponding server of the plurality of servers is assigned a new flow; and decrementing the one of the plurality of queue lengths when a flow ends on its corresponding server of the plurality of servers.
18. The non-transitory computer-readable medium of claim 13 , further comprising refreshing the server processing speed data periodically.
19. The non-transitory computer-readable medium of claim 18 , wherein refreshing the server processing speed data periodically comprises refreshing the server processing speed data every 200 ms.
20. The system of claim 8 , further comprising refreshing the server processing speed data periodically.