Oversubscription scheduling
A system for adjusting oversubscription loading includes an interface and a processor. The interface is configured to receive a set of performance data from a set of worker systems. The processor is configured to determine a feedback indication for a worker system of the set of worker systems based at least in part on the set of performance data. The feedback indication is used to adjust an oversubscription controller on the worker system. The processor is configured to provide the feedback indication to the worker system.
1. A system for adjusting oversubscription loading, comprising:
an interface configured to:
receive a set of performance data from a set of worker systems; and
a processor configured to:
determine a feedback indication for a worker system of the set of worker systems based at least in part on the set of performance data, wherein the set of performance data includes a stability indicator, a software experiment data, and a task interference, and wherein the feedback indication is used to adjust an oversubscription controller on the worker system, wherein the determining of the feedback indication comprises to:
perform the following:
A) determine a response time for an application performing requests sent to the application to obtain the stability indicator; and
in response to a determination that the response time is greater than or equal to a maximum response time threshold, include the stability indicator in the feedback indicator;
B) compare tasks over a period of time with no potentially interfering workloads present on a worker with tasks over the period of time with one or more other interfering workloads present on the same worker to obtain the software experiment data; and
in response to a determination that a decrease in tasks over the period of time is greater than or equal to a maximum decrease threshold, include the software experiment data in the feedback indicator; and
C) determine whether two or more tasks that are scheduled on the same worker use a shared resource to obtain the task interference; and
in response to a determination that two or more tasks that are scheduled on the same worker use the shared resource, include the task interference in the feedback indicator;
provide the feedback indication to the worker system;
adjust the oversubscription controller based on the provided feedback indication; and
evict a job based on the adjusted oversubscription controller.
2. The system of claim 1 , wherein the classification of performance data comprises a processor intensive classification.
3. The system of claim 1 , wherein the classification of performance data comprises a memory intensive classification.
4. The system of claim 1 , wherein the classification of performance data comprises a network access intensive classification.
5. The system of claim 1 , wherein the feedback indication is used to adjust a set point on the oversubscription controller.
6. The system of claim 5 , wherein the set point comprises a desired processor load set point.
7. The system of claim 5 , wherein the set point comprises a desired memory usage set point.
8. The system of claim 5 , wherein the set point comprises a desired network usage set point.
9. The system of claim 1 , wherein the feedback indication is used to adjust an eviction criteria on the oversubscription controller.
10. The system of claim 9 , wherein the eviction criteria comprises newest first.
11. The system of claim 9 , wherein the eviction criteria comprises most processor use first.
12. The system of claim 9 , wherein the eviction criteria comprises most memory use first.
13. The system of claim 9 , wherein the eviction criteria comprises most network use first.
14. A method for adjusting oversubscription loading, comprising:
receiving a set of performance data from a set of worker systems;
determining, using a processor, a feedback indication for a worker system of the set of worker systems based at least in part on the set of performance data, wherein the set of performance data includes a stability indicator, a software experiment data, and a task interference, and wherein the feedback indication is used to adjust an oversubscription controller on the worker system, wherein the determining of the feedback indication comprises:
performing the following:
A) determining a response time for an application performing requests sent to the application to obtain the stability indicator; and
in response to a determination that the response time is greater than or equal to a maximum response time threshold, including the stability indicator in the feedback indicator;
B) comparing tasks over a period of time with no potentially interfering workloads present on a worker with tasks over the period of time with one or more other interfering workloads present on the same worker to obtain the software experiment data; and
in response to a determination that a decrease in tasks over the period of time is greater than or equal to a maximum decrease threshold, including the software experiment data in the feedback indicator; and
C) determining whether two or more tasks that are scheduled on the same worker use a shared resource to obtain the task interference; and
in response to a determination that two or more tasks that are scheduled on the same worker use the shared resource, including the task interference in the feedback indicator;
providing the feedback indication to the worker system;
adjusting the oversubscription controller based on the provided feedback indication; and
evicting a job based on the adjusted oversubscription controller.
15. A computer program product for oversubscription loading, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
receiving a set of performance data from a set of worker systems;
determining a feedback indication for a worker system of the set of worker systems based at least in part on the set of performance data, wherein the set of performance data includes a stability indicator, a software experiment data, and a task interference, and wherein the feedback indication is used to adjust an oversubscription controller on the worker system, wherein the determining of the feedback indication comprises:
performing the following:
A) determining a response time for an application performing requests sent to the application to obtain the stability indicator; and
in response to a determination that the response time is greater than or equal to a maximum response time threshold, including the stability indicator in the feedback indicator;
B) comparing tasks over a period of time with no potentially interfering workloads present on a worker with tasks over the period of time with one or more other interfering workloads present on the same worker to obtain the software experiment data; and
in response to a determination that a decrease in tasks over the period of time is greater than or equal to a maximum decrease threshold, including the software experiment data in the feedback indicator; and
C) determining whether two or more tasks that are scheduled on the same worker use a shared resource to obtain the task interference; and
in response to a determination that two or more tasks that are scheduled on the same worker use the shared resource, including the task interference in the feedback indicator;
providing the feedback indication to the worker system;
adjusting the oversubscription controller based on the provided feedback indication; and
evicting a job based on the adjusted oversubscription controller.