Automated globally optimized resource allocator and scheduler
A method of efficiently executing a plurality of processes is described. The method generates, using a predictor, operating constraints for the processes. An operating constraint of the operating constraints is for each process of the processes. The method evaluates possible operating points for each process consistent with the operating constraints and according to a cost function for the processes. The method also determines an operating point for each process based on the evaluating.
1. A method of efficiently executing a plurality of processes, comprising:
generating, using a predictor, a plurality of operating constraints for the plurality of processes, an operating constraint of the plurality of operating constraints being for each process of the plurality of processes;
evaluating a plurality of possible operating points for each process consistent with the plurality of operating constraints and according to a cost function for the plurality of processes, the cost function including a user-weighted combination of computing resources utilized for the plurality of processes and a makespan for the plurality of processes; and
determining an operating point for each process based on the evaluating, the operating point including an order of each process of the plurality of processes;
wherein the plurality of processes are executed on the computing resources using the operating point including the order for each process of the plurality of processes.
2. The method of claim 1 , wherein the operating constraint for a process includes a runtime for the process versus a portion of available computing resources utilized for the process.
3. The method of claim 2 , wherein the operating point for the process includes a particular runtime and particular computing resources used for the runtime consistent with the operating constraint for the process.
4. The method of claim 1 , wherein the operating point for each process determines an order of the plurality of processes, an optimized makespan, and an optimized computing resources used for executing the plurality of processes.
5. The method of claim 1 , wherein the evaluating further includes:
searching a space including the plurality of possible operating points for each process.
6. The method of claim 1 , wherein the generating the plurality of operating constraints further includes:
reading at least one log file corresponding to at least one of the plurality of processes; and
generating the plurality of operating constraints based on the at least one log file.
7. A system for efficiently executing a plurality of processes, comprising:
a processor configured to:
generate, using a predictor, a plurality of operating constraints for the plurality of processes, an operating constraint of the plurality of operating constraints being for each process of the plurality of processes;
evaluate a plurality of possible operating points for each process consistent with the plurality of operating constraints and according to a cost function for the plurality of processes, the cost function including a user-weighted combination of computing resources utilized for the plurality of processes and a makespan for the plurality of processes; and
determine an operating point for each process based on the evaluation, the operating point including an order of each process of the plurality of processes, the plurality of processes being executed on the computing resources using the operating point including the order for each process of the plurality of processes; and
a memory coupled to the processor and configured to provide the processor with instructions.
8. The system of claim 7 , wherein the operating constraint for a process includes a runtime for the process versus a portion of available computing resources utilized for the process.
9. The system of claim 8 , wherein the operating point for the process includes a particular runtime and particular computing resources used for the runtime consistent with the operating constraint for the process.
10. The system of claim 7 , wherein to evaluate, the processor is further configured to:
search a space including the plurality of possible operating points for each process.
11. A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for efficiently executing a plurality of processes, comprising instructions for:
generating, using a predictor, a plurality of operating constraints for the plurality of processes, an operating constraint of the plurality of operating constraints being for each process of the plurality of processes;
evaluating a plurality of possible operating points for each process consistent with the plurality of operating constraints and according to a cost function for the plurality of processes, the cost function including a user-weighted combination of computing resources utilized for the plurality of processes and a makespan for the plurality of processes; and
determining an operating point for each process based on the evaluating, the operating point including an order of each process of the plurality of processes, the plurality of processes being executed on the computing resources using the operating point including the order for each process of the plurality of processes.
12. The computer program product of claim 11 , wherein the operating constraint for a process includes a runtime for the process versus a portion of available computing resources utilized for the process.
13. The computer program product of claim 11 , wherein the cost function includes a user-weighted combination of the computing resources utilized for the plurality of processes and a makespan for the plurality of processes.