Systems and methods for thread management to optimize resource utilization in a distributed computing environment
Systems and methods for embodiments for load attenuating thread pools (LATP) that may be associated with a service deployed in distributed computer environment, where that service utilizes a shared resource. A LATP includes a thread pool comprising a number of worker threads servicing requests handled by a service that includes such a LATP. The thread pool is managed by a thread pool manager of the LATP that can attenuate (herein used to mean add, remove or leave unchanged) the number of worker threads in the thread pool based on a resource utilization metric associated with the shared resource.
1. An distributed computing system, comprising:
a processor;
a non-transitory, computer-readable storage medium, including computer instructions for:
a set of first lower priority services, each lower priority service comprising a load attenuated thread pool comprising a thread pool manager and a thread pool comprising a set of threads, each thread of the thread pool utilizing a shared resource deployed at a computing platform accessible over a computing network by the lower priority service;
a set of higher priority services, each higher priority service adapted to process work items utilizing the shared resource,
wherein each thread pool manager of each service is independently adapted to:
at a time interval:
determine a resource utilization metric associated with the shared resource;
evaluate the resource utilization metric associated with the shared resource against a policy defining a capacity full threshold and a capacity available threshold to determine if the resource utilization metric falls into a capacity exceeded zone, a capacity full zone or a capacity available zone;
when the resource utilization metric falls into the capacity exceeded zone, determining a removal thread number and removing that removal thread number of threads from an existing number of the set of threads of the thread pool;
when the resource utilization metric falls into the capacity available zone, determining a thread addition number and adding that thread addition number of threads to the existing number of the set of threads of the thread pool; and
when the resource utilization metric falls into the capacity full zone leaving the set of threads unchanged.
2. The system of claim 1 , wherein determine the removal thread number comprises applying a back-off ratio defined in the policy to the existing number of the set of threads or a maximum number of threads as defined in the policy.
3. The system of claim 1 , wherein the resource utilization metric is a hardware or network utilization metric associated with a total load placed on the shared resource.
4. The system of claim 3 , wherein the resource utilization metric is CPU utilization of the computing platform on which the shared resource is deployed.
5. The system of claim 1 , wherein the remote computing platform is a cloud based computing platform.
6. The system of claim 1 , further comprising a thread manager updater adapted to access the shared resource, obtain the resource utilization metric, and store the resource utilization metric at a storage location, wherein each thread pool manager accesses the resource utilization metric at the storage location.
7. The system of claim 1 , wherein the set of higher priority services are associated with user interface requests of the distributed computing system.
8. A method for thread management in a distributed computing system, comprising:
providing a set of first lower priority services, each lower priority service comprising a load attenuated thread pool comprising a thread pool manager and a thread pool comprising a set of threads, each thread of the thread pool utilizing a shared resource deployed at a computing platform accessible over a computing network by the lower priority service;
providing a set of higher priority services, each higher priority service adapted to process work items utilizing the shared resource,
wherein each thread pool manager of each service is independently adapted to:
at a time interval:
determine a resource utilization metric associated with the shared resource;
evaluate the resource utilization metric associated with the shared resource against a policy defining a capacity full threshold and a capacity available threshold to determine if the resource utilization metric falls into a capacity exceeded zone, a capacity full zone or a capacity available zone;
when the resource utilization metric falls into the capacity exceeded zone, determining a removal thread number and removing that removal thread number of threads from an existing number of the set of threads of the thread pool;
when the resource utilization metric falls into the capacity available zone, determining a thread addition number and adding that thread addition number of threads to the existing number of the set of threads of the thread pool; and
when the resource utilization metric falls into the capacity full zone leaving the set of threads unchanged.
9. The method of claim 8 , wherein determine the removal thread number comprises applying a back-off ratio defined in the policy to the existing number of the set of threads or a maximum number of threads as defined in the policy.
10. The method of claim 8 , wherein the resource utilization metric is a hardware or network utilization metric associated with a total load placed on the shared resource.
11. The method of claim 10 , wherein the resource utilization metric is CPU utilization of the computing platform on which the shared resource is deployed.
12. The method of claim 8 , wherein the remote computing platform is a cloud based computing platform.
13. The method of claim 8 , further comprising a thread manager updater adapted to access the shared resource, obtain the resource utilization metric, and store the resource utilization metric at a storage location, wherein each thread pool manager accesses the resource utilization metric at the storage location.
14. The method of claim 8 , wherein the set of higher priority services are associated with user interface requests of the distributed computing system.
15. A non-transitory computer readable medium, comprising instructions for thread management in a distributed computing system, including instructions for:
providing a set of first lower priority services, each lower priority service comprising a load attenuated thread pool comprising a thread pool manager and a thread pool comprising a set of threads, each thread of the thread pool utilizing a shared resource deployed at a computing platform accessible over a computing network by the lower priority service;
providing a set of higher priority services, each higher priority service adapted to process work items utilizing the shared resource,
wherein each thread pool manager of each service is independently adapted to:
at a time interval:
determine a resource utilization metric associated with the shared resource;
evaluate the resource utilization metric associated with the shared resource against a policy defining a capacity full threshold and a capacity available threshold to determine if the resource utilization metric falls into a capacity exceeded zone, a capacity full zone or a capacity available zone;
when the resource utilization metric falls into the capacity exceeded zone, determining a removal thread number and removing that removal thread number of threads from an existing number of the set of threads of the thread pool;
when the resource utilization metric falls into the capacity available zone, determining a thread addition number and adding that thread addition number of threads to the existing number of the set of threads of the thread pool; and
when the resource utilization metric falls into the capacity full zone leaving the set of threads unchanged.
16. The non-transitory computer readable medium of claim 15 , wherein determine the removal thread number comprises applying a back-off ratio defined in the policy to the existing number of the set of threads or a maximum number of threads as defined in the policy.
17. The non-transitory computer readable medium of claim 15 , wherein the resource utilization metric is a hardware or network utilization metric associated with a total load placed on the shared resource.
18. The non-transitory computer readable medium of claim 17 , wherein the resource utilization metric is CPU utilization of the computing platform on which the shared resource is deployed.
19. The non-transitory computer readable medium of claim 15 , wherein the remote computing platform is a cloud based computing platform.
20. The non-transitory computer readable medium of claim 15 , further comprising a thread manager updater adapted to access the shared resource, obtain the resource utilization metric, and store the resource utilization metric at a storage location, wherein each thread pool manager accesses the resource utilization metric at the storage location.
21. The non-transitory computer readable medium of claim 15 , wherein the set of higher priority services are associated with user interface requests of the distributed computing platform.