Systems and methods for load balancing across media server instances
A system and method are presented for load balancing across media server instances. In an embodiment, media is broken out into a multi-tenanted service allowing the media to be scaled independently of the number of organizations supported on a cloud-based collaboration platform. Scaling may occur in a scaling-out or a scaling-in operation. States for a media service may comprise in-service, quiescing, quiesced, failed, etc. The states may be used to monitor sessions associated with an instance and determine which media instances to terminate during a scaling-in operation. In an embodiment, new instances may be added to a collection of media instances in response to an increased workload in a scaling-out operation.
1. A method for load balancing a workload across a dynamically resized collection of computing resources providing long running services in a system for handling a media request, the method comprising the steps of:
receiving, by a media manager, a request for a resource;
selecting an instance of a group of instances, by a datastore operatively coupled to the media manager, wherein the instance meets a first criteria, and refraining from selecting any instance of the group of instances that meets a second criteria, each instance of the group being configured to maintain a collection of sessions associated with the instance meeting the first criteria by:
in response to being forwarded the request, adding a session associated with the request to the collection of sessions; and
in response to an instance state change, removing the session from the collection of sessions;
detecting, by the media manager, a scaling-in operation within a sub-group of instances meeting the first criteria;
transitioning at least one instance of the sub-group of instances meeting the first criteria to a sub-group of instances meeting the second criteria;
maintaining each instance of the sub-group of instances meeting the second criteria while the collection of sessions associated with the instance meeting the second criteria comprises at least one session; and
transitioning the instance meeting the second criteria to a sub-group of instances meeting a third criteria when the collection of sessions associated with the instance meeting the second criteria is empty.
2. The method of claim 1 , wherein the instance meeting the first criteria comprises an in-service instance.
3. The method of claim 1 , wherein the instance meeting the second criteria comprises a quiescing instance, wherein the quiescing instances further comprises a quiescing state.
4. The method of claim 1 , wherein the instance meeting the third criteria comprises a quiesced instance.
5. The method of claim 1 , further comprising terminating instances in the sub-group of instances meeting the third criteria.
6. The method of claim 5 , wherein the termination is performed by one of: the instance itself directly or through a third-party process.
7. The method of claim 3 , wherein the quiescing state is indefinite.
8. The method of claim 3 , wherein the quiescing state ends after a pre-determined interval of time.
9. The method of claim 3 , wherein the quiescing state is ended by an explicit stimulus.
10. The method of claim 9 , wherein the explicit stimulus comprises at least one of: session ending, disconnection, resource expiration, and time expiration.
11. The method of claim 1 , wherein an instance is selected based on load metrics associated with the instances in the datastore.
12. The method of claim 11 , further comprising:
determining that the load metric satisfies a threshold;
determining that the sub-group of instances meeting the second criteria comprises at least one instance; and
transitioning at least one instance from the sub-group of instances meeting the second criteria to the sub-group of instances meeting the first criteria.
13. The method of claim 1 , wherein the workload comprises real-time media streams of indeterminate duration.
14. The method of claim 13 , wherein the media streams comprise at least one of: email, vmail, chat, video, text-messaging, web, social media, and co-browsing.
15. The method of claim 1 , wherein the instance state change comprises a comparison by the media manager of a current state with a prior state of the instance.
16. The method of claim 1 , wherein the detected scaling-in operation comprises a load condition which satisfies a scaling-in threshold.
17. The method of claim 16 , wherein the detecting of the satisfied load condition is performed by a monitoring service wherein the monitoring service monitors computing load.
18. The method of claim 17 , wherein the monitoring service is a third-party service.
19. The method of claim 1 , wherein the transitioning is performed by the instance itself.