System and method for supplementing program builds with file usage information
View Patent ↗A method for executing program builds is described comprising: analyzing file dependency information contained within a first file; scheduling jobs for a program build based on the analysis of the dependency information; executing the jobs according to the schedule; collecting file usage information from each of the jobs; and supplementing the file dependency information with the file usage information.
1. A method for executing program builds comprising:
analyzing file dependency information contained within a first file;
scheduling jobs for a program build based on the analysis of the dependency information;
executing the jobs according to the schedule;
collecting file usage information from each of the jobs; and
supplementing the file dependency information with the file usage information.
2. The method as in claim 1 further comprising:
analyzing both the file dependency information and the file usage information; and
scheduling jobs for a subsequent program build based on the analysis of the file dependency information and the file usage information.
3. The method as in claim 1 wherein scheduling jobs comprises:
determining, based on the dependency information, which jobs may be executed in parallel; and
scheduling the jobs which may be executed in parallel to execute in parallel.
4. The method as in claim 3 further comprising:
executing the jobs in parallel across a plurality of nodes.
5. The method as in claim 4 wherein collecting comprises
collecting the file usage information at each of the nodes on which the jobs are executed.
6. The method as in claim 5 further comprising:
transmitting the file usage information from each of the processing nodes to a build machine, the build machine combining and storing the file usage information received from each of the nodes.
7. The method as in claim 6 wherein the jobs are scheduled on the build machine and farmed out to the nodes.
8. The method as in claim 3 further comprising:
transmitting the files required for each of the jobs to each of the respective nodes prior to the execution of each of the jobs.
9. The method as in claim 1 wherein the first file is a Makefile.
10. The method as in claim 1 wherein supplementing the file dependency information comprises storing the file usage information as a new file.
11. The method as in claim 1 wherein supplementing the file dependency information comprises appending the file usage information to the first file.
12. The method as in claim 1 wherein the file usage information includes lengths of time it took to execute the jobs, said method further comprising:
scheduling jobs for a subsequent program build based on the amount of time it took to execute the jobs.
13. The method as in claim 12 further comprising:
determining a job schedule which will execute in the fastest time based on the amount of time it took to execute the jobs and the number of nodes available to process jobs.
14. A system for program builds comprising:
a job scheduler module to analyze file dependency information contained within a first file and to schedule jobs for a program build based on the analysis of the dependency information;
one or more build processes to execute the jobs according to the schedule; and
a usage tracking module to collect file usage information from each of the jobs and to supplement the file dependency information with the file usage information.
15. The system as in claim 14 further comprising:
a parser module to extract the file dependency information from the first file and store the extracted dependency information within a set of construction rules, the construction rules interpretable by the job scheduler.
16. The system as in claim 14 wherein the job scheduler module analyzes both the file dependency information and the file usage information and schedules jobs for a subsequent program build based on the analysis of the file dependency information and the file usage information.
17. The system as in claim 14 wherein the job scheduler module determines, based on the dependency information, which jobs may be executed in parallel and schedules the jobs which may be executed in parallel to execute in parallel.
18. The system as in claim 17 further comprising:
a plurality of nodes on which the jobs are executed in parallel according to the schedule.
19. The system as in claim 18 wherein a usage tracking module is executed on each of the plurality of nodes to collect the file usage information for jobs executed on each of the nodes.
20. The system as in claim 19 wherein each of the plurality of nodes transmits the file usage information collected from each job to a build machine on which the job scheduler module is executed, the build machine combining and storing the file usage information received from each of the nodes.
21. The system as in claim 14 further comprising a preload module to transmit the files required for each of the jobs to each of the respective nodes prior to the execution of each of the jobs.
22. The system as in claim 14 wherein the first file is a Makefile.
23. The system as in claim 14 wherein supplementing the file dependency information comprises storing the file usage information as a new file.
24. The system as in claim 14 wherein supplementing the file dependency information comprises appending the file usage information to the first file.
25. An article of manufacture having program code stored thereon which, when executed by a machine, cause the machine to perform the operations of:
analyzing file dependency information contained within a first file;
scheduling jobs for a program build based on the analysis of the dependency information;
executing the jobs according to the schedule;
collecting file usage information from each of the jobs; and
supplementing the file dependency information with the file usage information.
26. The article of manufacture as in claim 25 having program code stored thereon to cause said machine to perform the additional operations of:
analyzing both the file dependency information and the file usage information; and
scheduling jobs for a subsequent program build based on the analysis of the file dependency information and the file usage information.
27. The article of manufacture as in claim 25 having program code stored thereon to cause said machine to perform the additional operations of:
determining, based on the dependency information, which jobs may be executed in parallel; and
scheduling the jobs which may be executed in parallel to execute in parallel.
28. The article of manufacture as in claim 27 having program code stored thereon to cause said machine to perform the additional operations of:
executing the jobs in parallel across a plurality of nodes.
29. The article of manufacture as in claim 28 wherein collecting comprises
collecting the file usage information at each of the nodes on which the jobs are executed.
30. The article of manufacture as in claim 29 having program code stored thereon to cause said machine to perform the additional operations of:
transmitting the file usage information from each of the processing nodes to a build machine, the build machine combining and storing the file usage information received from each of the nodes.
31. The article of manufacture as in claim 30 wherein the jobs are scheduled on the build machine and farmed out to the nodes.
32. The article of manufacture as in claim 27 having program code stored thereon to cause said machine to perform the additional operations of:
transmitting the files required for each of the jobs to each of the respective nodes prior to the execution of each of the jobs.
33. The article of manufacture as in claim 25 wherein the first file is a Makefile.
34. The article of manufacture as in claim 25 wherein supplementing the file dependency information comprises storing the file usage information as a new file.
35. The article of manufacture as in claim 25 wherein supplementing the file dependency information comprises appending the file usage information to the first file.