Distributed system resource allocation method, apparatus, and system
Embodiments of the present disclosure can provide distributed system resource allocation methods and apparatuses. The method comprises: receiving a resource preemption request sent by a resource scheduling server, the resource preemption request comprising job execution information corresponding to a first job management server; determining, according to the job execution information corresponding to the first job management server and comprised in the resource preemption request, resources to be returned by a second job management server and a resource return deadline; and returning, according to and the resource return deadline and a current job execution progress of the second job management server, the resources to be returned to the resource scheduling server before expiration of the resource return deadline.
1. A distributed system resource allocation method for managing resources of a computer system, the method comprising:
receiving, by a resource scheduling server, a resource application request from a first job management server, the first job management server executing a high priority job, and the resource application request including an output deadline and an execution time of the high priority job;
identifying, by the resource scheduling server, a second job management server executing a low priority job of a plurality of job management servers;
receiving, by the second job management server, a resource preemption request from the resource scheduling server when a resource preemption condition is met, the resource preemption request including the output deadline and the execution time of the high priority job executing on the first job management server;
calculating, by the second job management server, a resource obtaining deadline of the high priority job according to a difference between the output deadline and the execution time of the high priority job;
when a sum of the current time and the remaining time for completion of executing the low priority job on the second job management server is greater than or equal to the resource obtaining deadline, the second job management server performs the steps of (i) backing up the low priority job, (ii) recording a backup position, and (iii) allocating resources to the resource scheduling server that were being utilized by the low priority job after the backup is completed, wherein the second job management server continues to execute the low priority job as long as a sum of a time to perform steps (i) and (ii) and the current time is less than the resource obtaining deadline;
allocating, by the resource scheduling server, the resources that were allocated to the resource scheduling server from the second job management server to the first job management server; and
executing, by the first job management server, the high priority job utilizing the resources allocated from the resource scheduling server.
2. The method of claim 1 , further comprising:
receiving, by the second job management server, resources re-allocated by the resource scheduling server;
acquiring the backup of the low priority job by the second job management server; and
resuming, by the second job management server, the execution of the low priority job from the backup position.
3. An apparatus for resource allocation for managing resources of a computer system, comprising:
a memory storing a set of instructions; and
one or more processors configured to execute the set of instructions to cause the apparatus to perform:
receiving, by a resource scheduling server, a resource application request from a first job management server, the first job management server executing a high priority job, and the resource application request including an output deadline and an execution time of the high priority job;
identifying, by the resource scheduling server, a second job management server executing a low priority job of a plurality of job management servers;
receiving, by the second job management server, a resource preemption request from the resource scheduling server when a resource preemption condition is met, the resource preemption request including the output deadline and the execution time of the high priority job executing on the first job management server;
calculating, by the second job management server, a resource obtaining deadline of the high priority job according to a difference between the output deadline and the execution time of the high priority job;
when a sum of the current time and the remaining time for completion of executing the low priority job on the second job management server is greater than or equal to the resource obtaining deadline, the second job management server performs the steps of (i) backing up the low priority job, (ii) recording a backup position, and (iii) allocating resources to the resource scheduling server that were being utilized by the low priority job after the backup is completed, wherein the second job management server continues to execute the low priority job as long as a sum of a time to perform steps (i) and (ii) and the current time is less than the resource obtaining deadline;
allocating, by the resource scheduling server, the resources that were allocated to the resource scheduling server from the second job management server to the first job management server; and
executing, by the first job management server, the high priority job utilizing the resources allocated from the resource scheduling server.
4. The apparatus of claim 3 , wherein the set of instructions that are executable by the one or more processors are further configured to perform:
receiving, by the second job management server, resources re-allocated by the resource scheduling server;
acquiring the backup of the low priority job by the second job management server; and
resuming, by the second job management server, the execution of the low priority job from the backup position.
5. A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computer to cause the computer to perform a network management method for managing resources of a computer system, the method comprising:
receiving, by a resource scheduling server, a resource application request from a first job management server, the first job management server executing a high priority job, and the resource application request including an output deadline and an execution time of the high priority job;
identifying, by the resource scheduling server, a second job management server executing a low priority job of a plurality of job management servers;
receiving, by the second job management server, a resource preemption request from the resource scheduling server when a resource preemption condition is met, the resource preemption request including the output deadline and the execution time of the high priority job executing on the first job management server;
calculating, by the second job management server, a resource obtaining deadline of the high priority job according to a difference between the output deadline and the execution time of the high priority job;
when a sum of the current time and the remaining time for completion of executing the low priority job on the second job management server is greater than or equal to the resource obtaining deadline, the second job management server performs the steps of (i) backing up the low priority job, (ii) recording a backup position, and (iii) allocating resources to the resource scheduling server that were being utilized by the low priority job after the backup is completed, wherein the second job management server continues to execute the low priority job as long as a sum of a time to perform steps (i) and (ii) and the current time is less than the resource obtaining deadline;
allocating, by the resource scheduling server, the resources that were allocated to the resource scheduling server from the second job management server to the first job management server; and
executing, by the first job management server, the high priority job utilizing the resources allocated from the resource scheduling server.
6. The non-transitory computer readable medium of claim 5 , wherein the set of instructions that are executable by the at least one processor of a computer to cause the computer to further perform:
receiving, by the second job management server, resources re-allocated by the resource scheduling server;
acquiring the backup of the low priority job by the second job management server; and
resuming, by the second job management server, the execution of the low priority job from the backup position.