Resource utilization efficiency based job scheduling
Computer-implemented methods, systems and computer program products for job scheduling in a computing system are provided. The computing resources of the computing system is allocated to one or more users of the computing system using a predefined policy. The method comprises receiving, by a job scheduler associated with the computing system one or more resource utilization scores of the one or more users, each of the one or more resource utilization scores indicating resource utilization efficiency over time at a job level for the one or more users. The method comprises receiving, by the job scheduler, one or more jobs submitted by the one or more users. The method further comprises scheduling, by the job scheduler, the received one or more jobs based on the predefined policy and further on the received one or more resource utilization scores of the one or more users.
1. A computer-implemented method for job scheduling in a computing system, wherein computing resources of the computing system are allocated to one or more users of the computing system using a predefined policy, the method comprising:
receiving, by a job scheduler of the computing system, a resource utilization score of each of the one or more users, wherein
the resource utilization score of each user is calculated based on resource utilization efficiencies of successfully completed jobs of that user and respective attenuations over time,
the resource utilization score for each user indicates resource utilization efficiency over time at a job level for that user,
the resource utilization score for each user is defined as ResUtScore=Σ t−1 n A t e ResUtScore t ,
n is a number of time periods, A t is a time attenuation factor for the t th time period, and ResUtScore t is a resource utilization score for that user during the t th time period,
the t th time period occurred more recently than the (t+1) th time period, and
A t is greater than A t+1 ;
receiving, by the job scheduler, one or more jobs submitted by the one or more users; and
scheduling, by the job scheduler, the received one or more jobs based on the predefined policy and further on the received resource utilization score for each of the one or more users, wherein said scheduling comprises:
allocating the computing resources of the computing system to the one or more users based on the predefined policy and further on the received resource utilization score for each of the one or more users.
2. The computer-implemented method of claim 1 , wherein the respective resource utilization efficiencies of successfully completed jobs of the one or more users are calculated based on collected metadata for the successfully completed jobs including job run time and corresponding resource allocation information.
3. The computer-implemented method of claim 1 , further comprising:
assigning a resource utilization efficiency to a job in response to the job failing, or the job failing to run for corresponding assigned resource(s).
4. The computer-implemented method of claim 1 , wherein the scheduling of the received one or more jobs comprises:
allocating, by the job scheduler, resources for the received one or more jobs further based on the received one or more resource utilization scores of the one or more users to avoid over-allocation or starvation.
5. The computer-implemented method of claim 4 , wherein the scheduling of the received one or more jobs comprises:
adjusting, by the job scheduler, the predefined policy with the received one or more resource utilization scores.
6. A system for job scheduling in a computing system, wherein computing resources of the computing system are allocated to one or more users of the computing system using a predefined policy, the system comprising:
a computing node comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processing units of the computing node to cause the one or more processing units to:
receive a resource utilization score of each of the one or more users, wherein
the resource utilization score of each user is calculated based on resource utilization efficiencies of successfully completed jobs of that user and respective attenuations over time,
the resource utilization score for each user indicates resource utilization efficiency over time at a job level for that user,
the resource utilization score for each user is defined as ResUtScore=Σ t−1 n A t e ResUtScore t ,
n is a number of time periods, A t is a time attenuation factor for the t th time period, and ResUtScore t is a resource utilization score for that user during the t th time period,
the t th time period occurred more recently than the (t+1) th time period, and
A t is greater than A t+1 ;
receive one or more jobs submitted by the one or more users; and
schedule the received one or more jobs based on the predefined policy and further on the received resource utilization score for each of the one or more users, wherein said scheduling comprises:
allocating the computing resources of the computing system to the one or more users based on the predefined policy and further on the received resource utilization score for each of the one or more users.
7. The system of claim 6 , wherein the respective resource utilization efficiencies of successfully completed jobs of the one or more users are calculated based on collected metadata for the successfully completed jobs including job run time and corresponding resource allocation information.
8. The system of claim 6 , further comprising:
assigning a resource utilization efficiency to a job in response to the job failing, or the job failing to run for corresponding assigned resource(s).
9. The system of claim 6 , wherein the scheduling of the received one or more jobs comprises:
allocating resources for the received one or more jobs further based on the received one or more resource utilization scores of the one or more users to avoid over-allocation or starvation.
10. The system of claim 9 , wherein the scheduling of the received one or more jobs comprises:
adjusting the predefined policy with the received one or more resource utilization scores.
11. A computer program product for job scheduling in a computing system, wherein computing resources of the computing system are allocated to one or more users of the computing system using a predefined policy, the computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith and the program instructions comprising:
program codes configured to:
receive, by a job scheduler of the computing system, a resource utilization score of each of the one or more users, wherein
the resource utilization score of each user is calculated based on resource utilization efficiencies of successfully completed jobs of that user and respective attenuations over time,
the resource utilization score for each user indicates resource utilization efficiency over time at a job level for that user,
the resource utilization score for each user is defined as ResUtScore=Σ t−1 n A t e ResUtScore t ,
n is a number of time periods, A t is a time attenuation factor for the t th time period, and ResUtScore t is a resource utilization score for that user during the t th time period,
the t th time period occurred more recently than the (t+1) th time period, and
A t is greater than A t+1 ;
receive, by the job scheduler, one or more jobs submitted by the one or more users; and
schedule, by the job scheduler, the received one or more jobs based on the predefined policy and further on the received resource utilization score for each of the one or more users, wherein said scheduling comprises:
allocating the computing resources of the computing system to the one or more users based on the predefined policy and further on the received resource utilization score for each of the one or more users.
12. The computer program product of claim 11 , wherein the respective resource utilization efficiencies of successfully completed jobs of the one or more users are calculated based on collected metadata for the successfully completed jobs including job run time and corresponding resource allocation information.
13. The computer program product of claim 11 , further comprising:
assigning a resource utilization efficiency to a job in response to the job failing, or the job failing to run for corresponding assigned resource(s).
14. The computer program product of claim 11 , wherein the program instructions comprising:
program codes configured to allocate resources for the received one or more jobs further based on the received one or more resource utilization scores of the one or more users to avoid over-allocation or starvation.