Pre-emptive scheduling of workloads to enable improved sharing of resources
A pre-emptive scheduling of workloads enables improved sharing of resources of a cluster of hosts. The steps of this pre-emptive scheduling method include: adjusting priority of active workloads that are each running on one of the nodes and idle workloads that have been suspended; determining that a priority of a first workload, which is one of the idle workloads, exceeds a priority of a second workload, which is one of the active workloads and is executing on a first node of the cluster of nodes; and suspending the second workload and resuming the first workload to run on the first node.
1 . A method of scheduling a plurality of workloads for execution in a cluster of nodes, the method comprising:
at each of the nodes, adjusting priority of first workloads that are each running thereon and adjusting priority of second workloads that have been suspended, wherein the nodes include a first node that is adjusting the priority of a first subset of the first workloads that are running on the first node and is adjusting the priority of all of the second workloads, and a second node that is adjusting the priority of a second subset of the first workloads that are running on the second node and is adjusting the priority of all of the second workloads;
at a central orchestrator, receiving the priorities of the first and second workloads from each of the nodes, including the priorities of the first workloads in the first subset and the priorities of the second workloads from the first node, and the priorities of the first workloads in the second subset and the priorities of the second workloads from the second node, and determining that a priority of a suspended workload, which is one of the second workloads, exceeds a priority of an active workload, which is one of the first workloads;
determining, by the central orchestrator, that one of the nodes on which the active workload is running has sufficient resources to execute the suspended workload if the active workload running thereon is suspended; and
instructing, by the central orchestrator, the one node to suspend the active workload running thereon and resume running the suspended workload thereon.
2 . The method of claim 1 , wherein the first node is equipped with accelerator hardware and the priority of each of the first workloads running on the first node is adjusted at the first node according to usage of the accelerator hardware by the respective first workload.
3 . The method of claim 2 , wherein, at the first node, the priority of one of the first workloads running on the first node is increased if the accelerator hardware is used by the respective first workload during a time interval and is decreased if the accelerator hardware is not being used by the respective first workload during the time interval.
4 . The method of claim 3 , wherein, at the first node, the priority of each of the first workloads running on the first node is continuously decreased over time.
5 . The method of claim 4 , wherein, at each of the nodes, the priority of each of the second workloads is continuously increased over time.
6 . The method of claim 1 , further comprising:
prior to resuming the running of the suspended workload on the one node, suspending a third workload, which is one of the first workloads and is executing on the one node,
wherein the priority of the third workload is lower than the priority of the active workload.
7 . The method of claim 6 , wherein
resources of the one node freed up by suspending the active workload is less than resources required to resume the running of the suspended workload on the one node, and
resources of the one node freed up by suspending the active and third workloads are greater than or equal to the resources required to resume the running of the suspended workload on the one node.
8 . The method of claim 1 , wherein the active workload is suspended by saving an entire executing image of the second active workload in storage.
9 . The method of claim 1 , wherein the active workload is suspended by saving a first part of an executing image of the active workload in storage and retaining a second part of the executing image of the active workload in memory of the one node.
10 . The method of claim 9 , further comprising:
determining that a priority of the active workload after being suspended exceeds a priority of a third workload, which is one of the first workloads and is executing on the second node; and
resuming running of the active workload that has been suspended on the second node by restoring the first part of the executing image of the active workload from storage and migrating the second part of the executing image of the active workload from the memory of the one node.
11 . A non-transitory computer-readable medium comprising instructions that are executable in processors of a computer system that includes a cluster of nodes, to carry out a method of scheduling a plurality of workloads for execution in the cluster of nodes, the method comprising:
at each of the nodes, adjusting priority of first workloads that are each running thereon and adjusting priority of second workloads that have been suspended, wherein the nodes include a first node that is adjusting the priority of a first subset of the first workloads that are running on the first node and is adjusting the priority of all of the second workloads, and a second node that is adjusting the priority of a second subset of the first workloads that are running on the second node and is adjusting the priority of all of the second workloads;
at a central orchestrator, receiving the priorities of the first and second workloads from each of the nodes, including the priorities of the first workloads in the first subset and the priorities of the second workloads from the first node, and the priorities of the first workloads in the second subset and the priorities of the second workloads from the second node, and determining that a priority of a suspended workload, which is one of the second workloads, exceeds a priority of an active workload, which is one of the first workloads;
determining, by the central orchestrator, that one of the nodes on which the active workload is running has sufficient resources to execute the suspended workload if the active workload running thereon is suspended; and
instructing, by the central orchestrator, the one node to suspend the active workload running thereon and resume running the suspended workload thereon.
12 . The non-transitory computer-readable medium of claim 11 , wherein the first node is equipped with accelerator hardware and the priority of each of the first workloads running on the first node is adjusted at the first node according to usage of the accelerator hardware by the respective first workload.
13 . The non-transitory computer-readable medium of claim 12 , wherein, at the first node, the priority of one of the first workloads running on the first node is increased if the accelerator hardware is used by the respective first workload during a time interval and is decreased if the accelerator hardware is not being used by the respective first workload during the time interval.
14 . The non-transitory computer-readable medium of claim 13 , wherein, at the first node, the priority of each of the first workloads running on the first node is continuously decreased over time.
15 . The non-transitory computer-readable medium of claim 14 , wherein, at each of the nodes, the priority of each of the second workloads is continuously increased over time.
16 . A computer system including a cluster of nodes in which a plurality of workloads are to be executed, wherein the computer system is programmed to carry out a method of scheduling a plurality of workloads for execution therein, the method comprising:
at each of the nodes, adjusting priority of first workloads that are each running thereon and adjusting priority of second workloads that have been suspended, wherein the nodes include a first node that is adjusting the priority of a first subset of the first workloads that are running on the first node and is adjusting the priority of all of the second workloads, and a second node that is adjusting the priority of a second subset of the first workloads that are running on the second node and is adjusting the priority of all of the second workloads;
at a central orchestrator, receiving the priorities of the first and second workloads from each of the nodes, including the priorities of the first workloads in the first subset and the priorities of the second workloads from the first node, and the priorities of the first workloads in the second subset and the priorities of the second workloads from the second node, and determining that a priority of a suspended workload, which is one of the second workloads, exceeds a priority of an active workload, which is one of the first workloads;
determining, by the central orchestrator, that one of the nodes on which the active workload is running has sufficient resources to execute the suspended workload if the active workload running thereon is suspended; and
instructing, by the central orchestrator, the one node to suspend the active workload running thereon and resume running the suspended workload thereon.
17 . The computer system of claim 16 , wherein the first node is equipped with accelerator hardware and the priority of each of the first workloads running on the first node is adjusted at the first node according to usage of the accelerator hardware by the respective first workload.
18 . The computer system of claim 17 , wherein, at the first node, the priority of one of the first workloads running on the first node is increased if the accelerator hardware is used by the respective first workload during a time interval and is decreased if the accelerator hardware is not being used by the respective first workload during the time interval.
19 . The computer system of claim 18 , wherein, at the first node, the priority of each of the first workloads running on the first node is continuously decreased over time.
20 . The computer system of claim 19 , wherein, at each of the nodes, the priority of each of the second workloads is continuously increased over time.