Conflict detection and correction in a program build environment
View Patent ↗A method is described for executing program builds comprising: scheduling jobs for a program build based on dependencies between files used in the jobs; executing the jobs according to the schedule; collecting file usage information from each of the jobs, the file usage information indicating operations performed on the files in each of the jobs; and analyzing the file usage information to determine whether any one of the jobs resulted in a conflict.
1. A method executing program builds comprising:
scheduling jobs for a program build based on dependencies between files used in the jobs;
executing the jobs according to the schedule;
collecting file usage information from each of the jobs, the file usage information indicating operations performed on the files in each of the jobs;
analyzing the file usage information to determine whether any of the jobs resulted in a conflict;
wherein analyzing comprises comparing the file versions used by each job with the file versions that the jobs would have used in a sequential program build and wherein a conflict is defined by a job using a version of a file that it would not have used in a sequential program build;
invalidating the results of the job that generated a conflict; and
rebuilding the job using the correct file versions.
2. The method as in claim 1 wherein scheduling jobs comprises:
analyzing file dependency information contained in a Makefile; and
scheduling jobs that are independent of one another to execute in parallel.
3. The method as in claim 1 further comprising:
detecting a conflict based on the comparison between the file versions used by each job and the file versions that each job would have used in a sequential program build.
4. The method as in claim 1 further comprising:
storing multiple versions of files used at different points in time during the course of the program build; and
generating a conflict if a file version used by a job is different from a file version that the job would have used in a sequential program build.
5. The method as in claim 4 further comprising:
discarding the results of the job that generated the conflict;
identifying the correct file versions for the job; and
rebuilding the job using the correct file versions.
6. The method as in claim 5 wherein identifying the correct file versions comprises:
searching the stored versions of the file to identify a version that was the most recently modified version prior to the start time of the job.
7. The method as in claim 6 wherein the version of the file has a name object version and a content object version associated therewith.
8. The method as in claim 7 wherein searching comprises:
identifying the correct name object version associated with the file; and
using the name object version to identify the correct content object version associated with the file.
9. The method as in claim 8 wherein identifying the correct name object version comprises identifying a name object within a directory and scanning the directory for the most recently modified name object version associated with the name object and generated prior to the start time of the job, wherein the name object version points to a content object, and wherein identifying the correct content object version comprises scanning for the most recently modified content object version generated prior to the start time of the job.
10. A program building system having at least one processor comprising:
a job scheduler module scheduling jobs for a program build based on dependencies between files used in the jobs;
a usage tracking module collecting file usage information from each of the jobs, the file usage information indicating operations performed on the files in each of the jobs;
a conflict detection module analyzing the file usage information to determine whether any one of the jobs resulted in a conflict and indicating a conflict if a conflict is detected;
wherein the conflict detection module detects a conflict by comparing the file versions used by each job with the file versions that the jobs would have used in a sequential program build and wherein a conflict is defined by a job using a version of a file that it would not have used in a sequential program build; and
a conflict resolution module to discard the results of the job that generated a conflict; and reschedule the job using the correct file versions.
11. The system as in claim 10 wherein the job scheduler module analyzes file dependency information contained in a Makefile; and schedules jobs that are independent of one another to execute in parallel.
12. The system as in claim 11 wherein the conflict resolution module reschedules the job ahead of all other scheduled jobs that have not started.
13. The system as in claim 10 further comprising:
a versioning file system to store multiple versions of files used at different points in time during the course of the program build; wherein the conflict detection module generates a conflict if a version used by a job is different from a file version that the job would have used in a sequential program build.
14. The system as in claim 13 wherein the conflict resolution module identifies the correct file versions for the job via the versioning file system; and rebuilds the job using the correct file versions.
15. The system as in claim 14 wherein the conflict resolution module searches the stored versions of the file to identify a version that was the most recently modified version prior to the start time of the job.
16. The system as in claim 15 wherein the versioning file system identifies a file version using both a name object version and a content object version.
17. The system as in claim 16 wherein, to identify the correct file version, the versioning file system first identifies the correct name object version associated with the file; and then uses the name object version to identify the correct content object associated with the file.
18. The system as in claim 17 wherein identifying the correct name object version comprises identifying a name object within a directory and scanning the directory for the most recently modified name object version associated with the name object and generated prior to the start time of the job, wherein the name object version points to a content object, and wherein identifying the correct content object version comprises scanning for the most recently modified content object version generated prior to the start time of the job.
19. An article of manufacture having program code stored thereon that, when executed by a machine, cause the machine to perform the operations of:
scheduling jobs for a program build based on dependencies between files used in the jobs;
executing the jobs according to the schedule;
collecting file usage information from each of the jobs, the file usage information indicating operations performed on the files in each of the jobs;
analyzing the file usage information to determine whether any one of the jobs resulted in a conflict;
wherein analyzing comprises comparing the file versions used by each job with the file versions that the jobs would have used in a sequential program build and wherein a conflict is defined by a job using a version of a file which it would not have used in a sequential program build;
discarding the results of the job which generated a conflict; and
rebuilding the job using the correct file versions.
20. The article of manufacture as in claim 19 comprising program code which, when executed by the machine cause the machine to perform the additional operations of:
analyzing file dependency information contained in a Makefile; and
scheduling jobs which are independent of one another to execute in parallel.
21. The article of manufacture as in claim 19 comprising program code which, when executed by the machine causes the machine to perform the additional operations of:
detecting a conflict based on the comparison between the file versions used by each job and the file versions that the jobs would have used in a sequential program build.
22. The article of manufacture as in claim 19 comprising program code which, when executed by the machine causes the machine to perform the additional operations of:
storing multiple versions of files used at different points in time during the course of the program build; and
generating a conflict if a file version used by a job is different from a file version that the job would have used in a sequential program build.
23. The article of manufacture as in claim 22 comprising program code which, when executed by the machine cause the machine to perform the additional operations of:
discarding the results of the job that generated the conflict;
identifying the correct file versions for the job; and
rebuilding the job using the correct file versions.
24. The article of manufacture as in claim 23 wherein identifying comprises:
searching the stored versions of the file to identify a version that was the most recently modified version prior to the start time of the job.
25. The article of manufacture as in claim 24 wherein the version of the file has a name object version and a content object version associated therewith.
26. The article of manufacture as in claim 25 wherein searching comprises:
identifying the correct name object version associated with the file; and
using the name object version to identify the correct content object version associated with the file.
27. The article of manufacture as in claim 26 wherein identifying the correct name object version comprises identifying a name object within a directory and scanning the directory for the most recently modified name object version associated with the name object and generated prior to the start time of the job, wherein the name object version points to a content object, and wherein identifying the correct content object version comprises scanning for the most recently modified content object version generated prior to the start time of the job.