System and Method for a Self-Optimizing Reservation in Time of Compute Resources

1 - 20 . (canceled)

21 . A method of dynamically modifying resources within a compute environment comprising a plurality of compute nodes under common administrative control, the method comprising:

receiving a request for allocation of resources within the compute environment, the request for allocation in support of processing one or more workload items;

monitoring one or more events after receiving the request for the allocation for resources;

based at least on the monitored one or more events, causing dynamic modification of at least the compute environment relative to a configuration of the compute environment when the request for the allocation is received, to produce a modified compute environment; and

causing processing of at least one of the one or more workload items submitted within the request for allocation using the modified compute environment.

22 . The method of claim 21 , further comprising causing dynamic modification, based at least on the monitored one or more events, of an allocation resources previously made responsive to the request.

23 . The method of claim 21 , wherein the causing dynamic modification of at least the compute environment, based at least on the monitored one or more events, comprises modification of to satisfy at least one of (i) the request for allocation of resources, or (ii) one or more preferences within the request for allocation of resources.

24 . The method of claim 21 , further comprising:

causing, responsive to the receiving the request for allocation of resources within the compute environment, reserving of at least a portion of the resources requested by the request; and

causing adjusting, based at least on the monitored one or more events, the at least portion of the resources after the reserving has been performed.

25 . The method of claim 24 , wherein the causing dynamic modification of at least the compute environment comprises causing modification of only the at least portion of the resources subject to the reserving.

26 . The method of claim 24 , further comprising, based at least on the monitored one or more events and subsequent to the reserving, causing un-reserving of the at least a portion of the resources subject to the reserving.

27 . The method of claim 21 , wherein:

the monitoring one or more events comprises receiving, at a computerized scheduler process of the compute environment, data relating to at least one of available processor capacity or available memory capacity, of one or more of the plurality of compute nodes of the compute environment; and

the receiving, at a computerized scheduler process of the compute environment, data relating to at least one of available processor capacity or available memory capacity, comprises receiving from a resource manager computerized process of the compute environment.

28 . The method of claim 21 , further comprising receiving submission of a software object configured to monitor at least one of (i) a policy relating to the compute environment, or (ii) a state of at least one aspect of one or more of the plurality of nodes.

29 . The method of claim 28 , wherein the receiving the submission of the software object comprises receiving the submission as part of the request.

30 . The method of claim 21 , wherein the receiving a request for allocation of resources within the compute environment comprises receiving a request for a reservation, the reservation comprising (i) data relating to a set of resources, and (ii) data specifying one or more jobs that can utilize the reservation.

31 . The method of claim 21 , wherein the request for allocation of resources comprises a request for one or more provisioning services for one or more of the compute nodes.

32 . The method of claim 21 , wherein the request for allocation of resources comprises a request for provisioning for one or more of the compute nodes, the provisioning being a condition precedent for processing of at least one of the one or more workload items.

33 . The method of claim 21 , wherein the request for allocation of resources comprises a request for direct volume access.

34 . The method of claim 21 , wherein the monitoring one or more events after receiving the request for the allocation for resources comprises identifying workload for processing, the identified workload having been submitted pursuant to the request for allocation.

35 . The method of claim 21 , wherein the monitoring one or more events after receiving the request for the allocation for resources comprises identifying workload for processing, the identified workload having been submitted by a user or user process different than that which submitted the request for allocation.

36 . The method of claim 21 , wherein the causing dynamic modification of at least the compute environment comprises modification to optimize at least one of (i) a processing time for the one or more workload items; or (ii) a time of completion for the one or more workload items.

37 . The method of claim 21 , wherein the causing dynamic modification of at least the compute environment comprises modification responsive to a failure of at least one compute node of the compute environment.

38 . The method of claim 21 , wherein the causing dynamic modification of at least the compute environment comprises anticipatory modification responsive to one or more projected maintenance events associated with one or more resources of the compute environment.

39 . The method of claim 21 , wherein:

the one or more workload items comprises a plurality of workload items; and

if at least one of the plurality of workload items has not commenced processing after a predetermined amount of time, then canceling any resources allocated to the at least one workload item.

40 . The method of claim 21 , wherein the one or more workload items comprises a plurality of workload items; and

the method further comprises reserving at least one resource of the compute environment responsive to the receiving the request for allocation of the resources within the compute environment; and

wherein the causing dynamic modification of at least the compute environment, based at least on the monitored one or more events, comprises:

determining that at least one of the plurality of workload items has not commenced processing after a predetermined amount of time has expired; and

based at least on the determining, canceling the reserved at least one resource.

41 . The method of claim 40 , wherein:

the reserving the at least one resource of the compute environment comprises:

identifying at least one of the plurality of compute nodes having insufficient resources for processing the at least one workload item; and

reserving at least a portion of compute resources of one of the compute nodes other than the identified at least one compute node, for processing the at least one workload item; and

the canceling the reserved at least one resource comprises canceling the reserved at least portion of the compute resources.

42 . The method of claim 40 , wherein the determining that at least one of the plurality of workload items has not commenced processing after a predetermined amount of time has expired comprises determining that the at least one of plurality of workload items has not commenced processing after expiry of a predetermined time beginning from the reserving.

43 . A non-transitory computer-readable storage apparatus comprising a storage medium storing instructions for controlling a computerized device to dynamically manage resources within a compute environment, the compute environment comprising a plurality of compute nodes under common administrative control, the instructions configured to, when executed on the computerized device:

receive a request for resources within the compute environment, the request associated with a computerized user process in data communication with the computerized device;

responsive to the request, cause processing of the request, the processing of the request comprising (i) the request to be queued, and (ii) invocation of a scheduling process wherein at least one resource of the compute environment is reserved for use by workload submitted by the computerized user process;

after the at least one resource of the compute environment is reserved, cause monitoring by the scheduling process of one or more parameters associated with the compute environment; and

based at least on the monitoring, dynamically cause the at least one resource of the compute environment to be unreserved so that the at least one resource can be allocated to one or more other requests; and

cause the request to continue to be queued for subsequent processing by the scheduling process.

44 . The non-transitory computer-readable apparatus of claim 43 , wherein the instructions are further configured to, when executed on the computerized device, cause second processing of the request, the second processing of the request comprising invocation of the scheduling process wherein at least one second resource of the compute environment is reserved for use by workload submitted by the computerized user process, the second resource comprising a node different from a node associated with the at least one resource.

45 . The non-transitory computer-readable apparatus of claim 44 , wherein the monitoring by the scheduling process of one or more parameters associated with the compute environment comprises monitoring a failure status of one or more of the plurality of compute nodes of the compute environment.

46 . The non-transitory computer-readable apparatus of claim 44 , wherein the monitoring by the scheduling process of one or more parameters associated with the compute environment comprises monitoring of one or more time parameters relating to how long the request has been queued.

47 . A non-transitory computer-readable storage apparatus comprising a storage medium storing instructions for controlling a computerized device to dynamically manage resources within a compute environment, the compute environment comprising a plurality of compute nodes under common administrative control, the instructions configured to, when executed on the computerized device:

receive a request for resources within the compute environment, the request associated with a computerized user process in data communication with the computerized device;

responsive to the request, cause processing of the request, the processing of the request comprising invocation of a computerized management process wherein at least one resource of the compute environment is reserved for use by workload submitted by the computerized user process;

after the at least one resource of the compute environment is reserved, cause monitoring of one or more parameters associated with the compute environment; and

based at least on the monitoring, dynamically cause allocation of at least one second resource of the compute environment to the workload.

48 . The non-transitory computer-readable apparatus of claim 47 , wherein:

the instructions are further configured to, when executed on the computerized device, cause processing of at least a portion of the workload by the at least one resource, the processing commencing after the reserve of the at least one resource but before the dynamic allocation of the at least one second resource;

the monitoring of one or more parameters associated with the compute environment comprises monitoring of the compute environment to identify one or more additional resources that have become available since the commencing of the processing;

the identified one or more additional resources that have become available since the commencing of the processing include the at least one second resource; and

wherein:

the request is configured to require a particular quality of service (QoS) for processing of the workload; and

the allocation of at least one second resource of the compute environment to the workload enables the processing of the workload consistent with the required QoS.

49 . A non-transitory computer-readable storage apparatus comprising a storage medium storing instructions for controlling a computerized device to manage resources within a compute environment, the compute environment comprising a plurality of compute nodes under common administrative control, the instructions configured to, when executed on the computerized device:

apply a computerized policy enforcement mechanism to at least one of the plurality of compute nodes, the computerized policy enforcement mechanism configured to enable only one or more prescribed users or computerized user processes associated therewith, to access resources of the at least one compute node;

after application of the computerized policy enforcement mechanism, receive a request for resources within the compute environment, the request submitted by a computerized user process associated with a user; and

responsive to the request:

determine that the user or computerized user process that submitted the request is one of the prescribed users or computerized user processes, respectively, specified by computerized policy enforcement mechanism; and

based at least on the determination, cause processing of the request, the processing of the request comprising invocation of a computerized management process wherein resources of the at least one of the plurality of compute nodes is reserved for use by workload submitted by the computerized user process associated with a user that submitted the request.

50 . The non-transitory computer-readable apparatus of claim 49 , wherein the instructions are further configured to, when executed on the computerized device, cause

after the resources of the at least one compute node are reserved, cause monitoring of one or more parameters associated with processing of the workload on the at least one compute node; and

based at least on the monitoring, dynamically cause allocation of at least second resources of the compute environment to the workload, the at least second resources associated with the at least one compute node which is subject to the policy enforcement mechanism.