Methods and systems for dynamic load balancing of processing resources in distributed environments
Systems and method are provided for load balancing in distributed environments. A computing device may instantiate a first quantity of partitions within a processing node. The first set of partitions can be configured to support a service accessible by multiple user devices. The computing device may determine that a load value corresponding to the first quantity of partitions is greater than a threshold and in response, cause an autoscaler to instantiate a second quantity of partitions. The quantity of petitions in the second quantity of partitions may be determined based on the first quantity of partitions. The computing device may then modify the autoscaler based on an indication that the second quantity of partitions has been instantiated. Modifying the autoscaler can include adjusting the threshold to reduce a likelihood that a subsequent load value is greater than the threshold.
1. A method comprising:
instantiating a first set of partitions within a processing node, the first set of partitions configured to support a service accessible by a plurality of users, the first set of partitions including a first quantity of partitions;
determining that a load value corresponding to the first set of partitions is greater than a threshold;
instantiating, in response to determining that the load value is greater than the threshold and by an autoscaler, a second set of partitions including a second quantity of partitions determined based on a scaling value, wherein the scaling value decreases as a quantity of partitions that have been instantiated increases; and
modifying the autoscaler based on an indication that the second set of partitions have been instantiated, wherein modifying the autoscaler includes adjusting the threshold to reduce a likelihood that a subsequent load value is greater than the threshold.
2. The method of claim 1 , wherein the first quantity of partitions is selected based on n-modular redundancy.
3. The method of claim 1 , wherein the second quantity of partitions is equal to the first quantity of partitions.
4. The method of claim 1 , further comprising:
determining that the load value is greater than the threshold for a predetermined time interval, wherein instantiating the second set of partitions is further in response to determining that the load value is greater than the threshold for the predetermined time interval.
5. The method of claim 1 , wherein each partition of the first set of partitions includes a set of containers, and wherein each container includes an environment configured to execute one or more applications accessible to one or more users.
6. A system comprising:
one or more processors; and
a machine-readable storage medium storing instructions that when executed by the one or more processors, cause the one or more processors to perform operations including:
instantiating a first set of partitions within a processing node, the first set of partitions configured to support a service accessible by a plurality of users, the first set of partitions including a first quantity of partitions;
determining that a load value corresponding to the first set of partitions is greater than a threshold;
instantiating, in response to determining that the load value is greater than the threshold and by an autoscaler, a second set of partitions including a second quantity of partitions determined based on a scaling value, wherein the scaling value decreases as a quantity of partitions that have been instantiated increases; and
modifying the autoscaler based on an indication that the second set of partitions have been instantiated, wherein modifying the autoscaler includes adjusting the threshold to reduce a likelihood that a subsequent load value is greater than the threshold.
7. The system of claim 6 , wherein the first quantity of partitions is selected based on n-modular redundancy.
8. The method of claim 4 , wherein a length of the predetermined time interval is determined based on a quantity of the partitions that have been instantiated.
9. The system of claim 6 , wherein the second quantity of partitions is equal to the first quantity of partitions.
10. The system of claim 6 , wherein the operations further include:
determining that the load value is greater than the threshold for a predetermined time interval, wherein instantiating the second set of partitions is further in response to determining that the load value is greater than the threshold for the predetermined time interval.
11. The system of claim 10 , wherein a length of the predetermined time interval is determined based on a quantity of the partitions that have been instantiated.
12. The system of claim 6 , wherein each partition of the first set of partitions includes a set of containers, and wherein each container includes an environment configured to execute one or more applications accessible to one or more users.
13. A non-transitory computer-readable storage medium storing instructions that when executed by one or more processors, cause the one or more processors to perform operations including:
instantiating a first set of partitions within a processing node, the first set of partitions configured to support a service accessible by a plurality of users, the first set of partitions including a first quantity of partitions;
determining that a load value corresponding to the first set of partitions is greater than a threshold;
instantiating, in response to determining that the load value is greater than the threshold and by an autoscaler, a second set of partitions including a second quantity of partitions determined based on a scaling value, wherein the scaling value decreases as a quantity of partitions that have been instantiated increases; and
modifying the autoscaler based on an indication that the second set of partitions have been instantiated, wherein modifying the autoscaler includes adjusting the threshold to reduce a likelihood that a subsequent load value is greater than the threshold.
14. The non-transitory computer-readable storage medium of claim 13 , wherein the first quantity of partitions is selected based on n-modular redundancy.
15. The non-transitory computer-readable storage medium of claim 13 , wherein the second quantity of partitions is equal to the first quantity of partitions.
16. The non-transitory computer-readable storage medium of claim 13 , wherein the operations further include:
determining that the load value is greater than the threshold for a predetermined time interval, wherein instantiating the second set of partitions is further in response to determining that the load value is greater than the threshold for the predetermined time interval.
17. The non-transitory computer-readable storage medium of claim 16 , wherein a length of the predetermined time interval is determined based on a quantity of the partitions that have been instantiated.