Optimized job scheduling and execution in a distributed computing grid
Disclosed example methods for network switch control include accessing link characteristics for a plurality of network switches. The disclosed example methods also include comparing the link characteristics for the plurality of network switches with resource characteristics associated with a data processing function to determine a first one of the network switches on which to execute the data processing function. The disclosed example methods also include scheduling data processing function to be executed by the first one of the network switches.
1. A method for network switch control, the method comprising:
accessing, with a processor, link characteristics for a plurality of network switches;
comparing the link characteristics for the plurality of network switches with resource characteristics associated with a data processing function to determine a first one of the network switches on which to execute the data processing function, the link characteristics including a bandwidth of the link; and
scheduling the data processing function to be executed by the first one of the network switches.
2. The method of claim 1 , wherein the link characteristics include link utilization information associated with the plurality of network switches.
3. The method of claim 1 , further including:
monitoring status of the plurality of network switches to determine the link characteristics for the plurality of network switches; and
storing the link characteristics for the plurality of network switches in memory for subsequent access and comparison to the resource characteristics associated with the data processing function.
4. The method of claim 1 , further including reserving link capacity for the data processing function prior to scheduling the data processing function for execution by the first one of the network switches.
5. The method of claim 1 , further including monitoring execution of the data processing function by the first one of the network switches.
6. The method of claim 1 , wherein comparing the link characteristics for the plurality of network switches with the resource characteristics associated with the data processing function includes:
identifying respective ones of the network switches having respective link characteristics satisfying the resource characteristics associated with the data processing function;
determining respective costs for executing the data processing function on the respective ones of the network switches based on respective cost information associated with the respective ones of the network switches; and
comparing the respective costs to select the first one of the network switches.
7. A machine readable storage device comprising instructions which, when executed, cause a machine to perform operations comprising:
accessing link characteristics for a plurality of network switches;
comparing the link characteristics for the plurality of network switches with resource characteristics associated with a data processing function to determine a first one of the network switches on which to execute the data processing function, the link characteristics including a bandwidth of the link; and
scheduling the data processing function to be executed by the first one of the network switches.
8. The machine readable storage device of claim 7 , wherein the link characteristics include link utilization information associated with the plurality of network switches.
9. The machine readable storage device of claim 7 , wherein the operations further include:
monitoring status of the plurality of network switches to determine the link characteristics for the plurality of network switches; and
storing the link characteristics for the plurality of network switches in memory for subsequent access and comparison to the resource characteristics associated with the data processing function.
10. The machine readable storage device of claim 7 , wherein the operations further include reserving link capacity for the data processing function prior to scheduling the data processing function for execution by the first one of the network switches.
11. The machine readable storage device of claim 7 , wherein the operations further include monitoring execution of the data processing function by the first one of the network switches.
12. The machine readable storage device of claim 7 , wherein to compare the link characteristics for the plurality of network switches with the resource characteristics associated with the data processing function, the operations further include:
identifying respective ones of the network switches having respective link characteristics satisfying the resource characteristics associated with the data processing function;
determining respective costs for executing the data processing function on the respective ones of the network switches based on respective cost information associated with the respective ones of the network switches; and
comparing the respective costs to select the first one of the network switches.
13. A system for network switch control, the system comprising:
memory including computer readable instructions; and
a processor to execute the computer readable instructions to perform operations including:
accessing link characteristics for a plurality of network switches, the link characteristics including a bandwidth of the link;
comparing the link characteristics for the plurality of network switches with resource characteristics associated with a data processing function to determine a first one of the network switches on which to execute the data processing function; and
scheduling the data processing function to be executed by the first one of the network switches.
14. The system of claim 13 , wherein the link characteristics include link utilization information associated with the plurality of network switches.
15. The system of claim 13 , wherein the operations further include:
monitoring status of the plurality of network switches to determine the link characteristics for the plurality of network switches; and
storing the link characteristics for the plurality of network switches in memory for subsequent access and comparison to the resource characteristics associated with the data processing function.
16. The system of claim 13 , wherein the operations further include reserving link capacity for the data processing function prior to the job scheduler scheduling the data processing function for execution by the first one of the network switches.
17. The system of claim 13 , wherein the operations further include monitoring execution of the data processing function by the first one of the network switches.
18. The system of claim 13 , wherein to compare the link characteristics for the plurality of network switches with the resource characteristics associated with the data processing function, the operations further include:
identifying respective ones of the network switches having respective link characteristics satisfying the resource characteristics associated with the data processing function;
determining respective costs for executing the data processing function on the respective ones of the network switches based on the respective cost information associated with the respective ones of the network switches; and
comparing the respective costs to select the first one of the network switches.