System and method of providing system jobs within a compute environment
The disclosure relates to systems, methods and computer-readable media for using system jobs for performing actions outside the constraints of batch compute jobs submitted to a compute environment such as a cluster or a grid. The method for modifying a compute environment from a system job disclosure associating a system job to a queuable object, triggering the system job based on an event and performing arbitrary actions on resources outside of compute nodes in the compute environment. The queuable objects include objects such as batch compute jobs or job reservations. The events that trigger the system job may be time driven, such as ten minutes prior to completion of the batch compute job, or dependent on other actions associated with other system jobs. The system jobs may be utilized also to perform rolling maintenance on a node by node basis.
1. A computer network comprising:
one or more processors;
a network manager process; and
a scheduler process executed by the one or more processors to:
receive a request to process a job in the computer network, wherein the request identifies a particular quality of service for processing the job; and
automatically create a system job based on the request, wherein the system job is executed by the one or more processors to:
identify network resources needed to provide the particular quality of service; and
communicate with the network manager process to provision the computer network to provide the identified network resources,
wherein:
the identified particular quality of service requires automatic provisioning of a node in the computer network; and
the provisioning of the computer network by the network manager process comprises said automatic provisioning.
2. The computer network of claim 1 , wherein the identified particular quality of service comprises a dedicated bandwidth, and the provisioning of the computer network comprises causing partition of at least a portion of the computer network.
3. The computer network of claim 1 , wherein the identified particular quality of service requires enablement of a special service.
4. The computer network of claim 1 , wherein the automatic creation of the system job comprises creation by at least encapsulating, in the system job, the request to process the job.
5. The computer network of claim 1 , wherein the provisioning of the computer network comprises performance of a dynamic partition to provide the identified network resources to provide the particular quality of service.
6. A method comprising:
receiving a job request to perform a job in a computer network, the computer network comprising at least one compute environment, wherein the job request specifies a quality of service for processing the job; and
automatically creating, based on the job request, a system job to configure at least a portion of the at least one compute environment to provide the specified quality of service for performance of the job,
wherein: the specified quality of service requires automatic provisioning of a node in the at least one compute environment; and the automatically creating, based on the job request, a system job to configure at least a portion of the at least one compute environment comprises the automatic provisioning of the node.
7. The method of claim 6 , wherein the specified quality of service requires use of a dedicated bandwidth.
8. The method of claim 6 , wherein:
the specified quality of service requires enablement of a special service; and
the automatically creating, based on the job request, the system job to configure the at least portion of the at least one compute environment comprises the enablement of the special service; and wherein the method further comprises using at least the special service to process the job.
9. The method of claim 6 , wherein the system job is automatically created by encapsulating the job request in the system job.
10. The method of claim 6 , further comprising causing dynamic partition of the at least portion of the at least one compute environment to provide the specified quality of service.
11. A non-transitory computer-readable storage medium storing a plurality of instructions, the plurality of instructions executed by a processor apparatus to:
receive a workload job for execution within a compute environment, the workload job comprising at least one parameter; and
automatically generate a system job based on the at least one parameter of the workload job, wherein the system job is configured to provide a guaranteed quality of service for performing the workload job in the compute environment,
wherein the guaranteed quality of service comprises automatic provisioning of a node in the compute environment; and the automatic generation of the system job based on the at least one parameter of the workload job to provide the guaranteed quality of service for the performing of the workload job in the compute environment comprises the automatic provisioning of the node.
12. The non-transitory computer-readable storage medium of claim 11 , wherein the system job comprises no workload execution tasks.
13. The non-transitory computer-readable storage medium of claim 11 , wherein the system job is configured to provide the guaranteed quality of service by at least: (i) identification of resources in the compute environment needed to guarantee the quality of service, and (ii) partition of the compute environment to provide the identified resources.
14. The non-transitory computer-readable storage medium of claim 13 , wherein the partition of the compute environment comprises at least one dynamic partition to provide the identified resources.
15. The non-transitory computer-readable storage medium of claim 11 , wherein the guaranteed quality of service comprises a dedicated bandwidth.
16. The non-transitory computer-readable storage medium of claim 11 , wherein the guaranteed quality of service comprises enablement of a special service.
17. The non-transitory computer-readable storage medium of claim 11 , wherein the system job is automatically created by encapsulating the workload job in the system job.
18. The non-transitory computer-readable storage medium of claim 11 , wherein the at least one parameter specifies the guaranteed quality of service.