Multilayered resource scheduling

1. A system for multilayered resource scheduling, the system comprising:

one or more processors; and

one or more non-transitory computer-readable media including one or more sets of instructions which, when executed by the one or more processors, causes:

providing a user interface operable to receive a plurality of tasks;

providing an upper level scheduler operable to sort a plurality of tasks into tasks with identical resource requirements;

providing at least one lower level scheduler communicatively coupled to the upper level scheduler, the at least one lower level scheduler being operable to:

receive resource requirements associated with a task;

ascertain compute resources sufficient to execute the task on a machine in a compute farm;

advertise the compute resources as matching the resource requirements;

receive a request to execute the task on the machine, wherein the compute resources are controlled to execute the tasks with the identical resource requirements without further scheduling operations; and

control the compute resources to execute the task;

providing at least one reporting module to provide a global view of the compute resources to the user interface; and

providing a mitigation mechanism to schedule priority tasks, the mitigation mechanism operable to instruct the upper level scheduler to stop sending the tasks with the identical resource requirements after a current task is executed to allow execution of higher priority tasks.

2. The system of claim 1 , wherein the at least one lower level scheduler is operable to receive the plurality of tasks directly without modifications of the plurality of tasks.

3. The system of claim 1 , wherein the at least one lower level scheduler is further operable to make the compute resources available in response to a demand by the upper level scheduler.

4. The system of claim 1 , wherein the upper level scheduler includes a private scheduler, the private scheduler being operable to manage tasks associated with a single entity.

5. The system of claim 1 , wherein the lower level scheduler includes a master scheduler process and a resource management process corresponding to each machine in the compute farm under control of the lower level scheduler.

6. The system of claim 5 , wherein the compute resources matching the resource requirements are advertised by the resource management process in conjunction with a proxy task.

7. The system of claim 6 , wherein the compute resources include at least one of a number of central processing units and an amount of memory.

8. The system of claim 1 , wherein the at least one lower level scheduler is operable to receive further tasks of the plurality of tasks via a proxy encapsulating the tasks with the identical resource requirements.

9. The system of claim 1 , wherein the upper level scheduler is aware of a subset of underlying compute resources, whereas the lower level scheduler is aware of full compute resources.

10. The system of claim 1 , wherein the mitigation mechanism is further operable to stop accepting the tasks with the identical resource requirements after a predetermined period of time to allow the lower level scheduler to schedule tasks associated with different resource requirements.

11. The system of claim 1 , wherein the at least one lower level scheduler is operable to ascertain the compute resources based on preexisting policies.

12. A method for multilayered resource scheduling, the method comprising:

receiving, by a user interface of a computer system, a plurality of tasks;

sorting, by an upper level scheduler of the computer system, the plurality of tasks into tasks with identical resource requirements;

receiving, by at least one lower level scheduler of the computer system, resource requirements associated with a task;

ascertaining, by the at least one lower level scheduler, compute resources sufficient to execute the task on a machine in a compute farm;

advertising, by the at least one lower level scheduler, the compute resources as matching the resource requirements;

receiving, by the at least one lower level scheduler, a request to execute the task on the machine, wherein the compute resources are controlled to execute the tasks with the identical resource requirements without further scheduling operations;

controlling, by the at least one lower level scheduler, the compute resources to execute the task;

providing, by at least one reporting module of the computer system, a global view of the compute resources to the user interface;

receiving, by the at least one lower level scheduler, further tasks of the plurality of tasks via a proxy encapsulating the tasks with the identical resource requirements; and

scheduling, by a mitigation mechanism of the computer system associated with the proxy, priority tasks, the mitigation mechanism being operable to stop accepting the tasks with the identical resource requirements after a predetermined period of time to allow the at least one lower level scheduler to schedule tasks associated with different resource requirements.

13. The method of claim 12 , further comprising providing, by the at least one lower level scheduler, the compute resources in response to a demand by the upper level scheduler.

14. The method of claim 12 , wherein the lower level scheduler includes a master scheduler process and a resource management process corresponding to each machine in the compute farm under control of the lower level scheduler.

15. The method of claim 12 , further comprising instructing, by a mitigation mechanism associated with the proxy, the upper level scheduler to stop sending the tasks with the identical resource requirements after a current task is executed to allow execution of higher priority tasks.

16. The method of claim 12 , further comprising managing, by the upper level scheduler, tasks associated with a single entity, the upper level scheduler including a private scheduler.

17. The method of claim 12 , further comprising receiving, by the at least one lower level scheduler, the plurality of tasks directly without modifications of the plurality of tasks.

18. The method of claim 12 , wherein the compute resources matching the resource requirements are advertised by a resource management process in conjunction with a proxy task.

19. A system for multilayered resource scheduling, the system comprising:

one or more processors; and

one or more non-transitory computer-readable media including one or more sets of instructions which, when executed by the one or more processors, causes:

providing an upper level scheduler operable to sort a plurality of tasks into tasks with identical resource requirements;

providing at least one lower level scheduler communicatively coupled to the upper level scheduler, the at least one lower level scheduler being operable to:

receive resource requirements associated with a task;

ascertain compute resources sufficient to execute the task on a machine in a compute farm;

advertise the compute resources as matching the resource requirements;

make the compute resources available in response to a demand by the upper level scheduler;

receive a request to execute the task on the machine, wherein the compute resources are controlled to execute the tasks with the identical resource requirements without further scheduling operations; and

control the compute resources to execute the task;

receive, further tasks of the plurality of tasks via a proxy encapsulating the tasks with the identical resource requirements;

wherein the lower level scheduler includes a master scheduler process and a resource management process corresponding to each machine in the compute farm under control of the lower level scheduler; and

wherein the compute resources matching the resource requirements are advertised by the resource management process; and

providing a mitigation mechanism associated with the proxy operable to instruct the upper level scheduler to stop sending the tasks with the identical resource requirements after a current task is executed to allow execution of higher priority tasks.

20. The system of claim 19 , wherein the one or more sets of instructions, when executed by the one or more processors, further causes:

providing a user interface to receive the plurality of tasks; and

providing at least one reporting module to provide a global view of the compute resources to the user interface.