Delta based task analysis for CI systems
Systems and methods for a delta based task analysis in CI systems are disclosed. The systems and methods analyze a first job, which includes multiple first tasks, and a second job, which includes multiple second tasks. The systems and methods identify a set of different tasks between the first tasks and the second tasks. The systems and methods then create a third job based on the set of different tasks, which includes metadata corresponding to the first job and the second job. The systems and method execute the first job and the third job, which completes the second job based on the metadata included in the third job.
1 . A method comprising:
analyzing a first job and a second job, wherein the first job comprises a plurality of first tasks and the second job comprises a plurality of second tasks, and wherein the analyzing identifies a set of different tasks between the plurality of first tasks and the plurality of second tasks;
identifying a fork location and a join location in the first job, and inserting a fork command at the identified fork location and a join command at the identified join location;
creating, by a processing device, a third job comprising the set of different tasks and metadata corresponding to the first job and the second job, wherein the third job is a standalone executable job and is without a set of continuous common tasks between the first job and the second job, and wherein the metadata indicates an indication of the set of continuous common tasks between the first job and the second job and an indication of the set of different tasks; and
executing both the first job and the third job, wherein the executing of the first job and the third job completes the second job based on the metadata included in the third job by reusing results of the set of continuous common tasks from the first job and executing the set of different tasks in the third job to avoid re-executing the set of continuous common tasks for the second job, wherein the executing further comprises executing the first job until encountering the fork command, forking to the third job while continuing execution the first job, and joining the third job back to the first job upon encountering the join command.
2 . The method of claim 1 , further comprising:
identifying, in the first job, the fork location between the set of continuous common tasks and the set of different tasks; and
inserting the fork command at the fork location in the first job, wherein the fork command adds a branch to the first job that branches to the third job subsequent to the executing of the set of continuous common tasks.
3 . The method of claim 2 , wherein the set of continuous common tasks is a first set of continuous common tasks, the method further comprising:
identifying, during the analyzing, a second set of continuous common tasks subsequent to the set of different tasks; and
inserting, in the first job, the join command between the set of different tasks and the second set of continuous common tasks.
4 . The method of claim 2 , wherein the set of continuous common tasks are included in both the plurality of first tasks and the plurality of second tasks in a same order, and
the set of different tasks are included in the plurality of second tasks and are absent from the plurality of first tasks.
5 . The method of claim 1 , further comprising:
inserting a third task into the third job based on the one or more differences between the first job and the second job.
6 . The method of claim 1 , further comprising:
capturing one or more snapshots of the first job during the executing;
receiving a request to execute a fourth job;
determining that the fourth job comprises a set of third continuous tasks corresponding to at least one of the one or more snapshots;
responsive to the determining, modifying the fourth job by replacing the set of fourth continuous tasks with the at least one of the one or more snapshots; and
executing the modified fourth job.
7 . A system comprising:
a memory; and
a processing device operatively coupled to the memory, the processing device to:
analyze a first job and a second job, wherein the first job comprises a plurality of first tasks and the second job comprises a plurality of second tasks, and wherein the analysis identifies a set of different tasks between the plurality of first tasks and the plurality of second tasks;
identify a fork location and a join location in the first job, and inserting a fork command at the identified fork location and a join command at the identified join location;
create a third job comprising the set of different tasks and metadata corresponding to the first job and the second job, wherein the third job is a standalone executable job and is without a set of continuous common tasks between the first job and the second job, and wherein the metadata indicates an indication of the set of continuous common tasks between the first job and the second job and an indication of the set of different tasks; and
execute both the first job and the third job, wherein the executing of the first job and the third job completes the second job based on the metadata included in the third job by reusing results of the set of continuous common tasks from the first job and executing the set of different tasks in the third job to avoid re-executing the set of continuous common tasks for the second job, wherein the executing further comprises executing the first job until encountering the fork command, forking to the third job while continuing execution the first job, and joining the third job back to the first job upon encountering the join command.
8 . The system of claim 7 , wherein the processing device is to:
identify, in the first job, the fork location between the set of continuous common tasks and the set of different tasks; and
insert the fork command at the fork location in the first job, wherein the fork command adds a branch to the first job that branches to the third job subsequent to the executing of the set of continuous common tasks.
9 . The system of claim 8 , wherein the set of continuous common tasks is a first set of continuous common tasks, the processing device is to:
identify, during the analysis, a second set of continuous common tasks subsequent to the set of different tasks; and
insert, in the first job, the join command between the set of different tasks and the second set of continuous common tasks.
10 . The system of claim 8 , wherein the set of continuous common tasks are included in both the plurality of first tasks and the plurality of second tasks in a same order, and the set of different tasks are included in the plurality of second tasks and are absent from the plurality of first tasks.
11 . The system of claim 7 , wherein the processing device is to:
insert a third task into the third job based on the one or more differences between the first job and the second job.
12 . The system of claim 7 , wherein the processing device is to:
capture one or more snapshots of the first job during the executing;
receive a request to execute a fourth job;
determine that the fourth job comprises a set of third continuous tasks corresponding to at least one of the one or more snapshots;
modify the fourth job by replacing the set of fourth continuous tasks with the at least one of the one or more snapshots; and
execute the modified fourth job.
13 . A non-transitory computer readable medium, having instructions stored thereon which, when executed by a processing device, cause the processing device to:
analyze a first job and a second job, wherein the first job comprises a plurality of first tasks and the second job comprises a plurality of second tasks, and wherein the analysis identifies a set of different tasks between the plurality of first tasks and the plurality of second tasks;
identify, by the processing device, a fork location and a join location in the first job, and inserting a fork command at the identified fork location and a join command at the identified join location;
create, by the processing device, a third job comprising the set of different tasks and metadata corresponding to the first job and the second job, wherein the third job is a standalone executable job and is without a set of continuous common tasks between the first job and the second job, and wherein the metadata indicates an indication of the set of continuous common tasks between the first job and the second job and an indication of the set of different tasks; and
execute both the first job and the third job, wherein the executing of the first job and the third job completes the second job based on the metadata included in the third job by reusing results of the set of continuous common tasks from the first job and executing the set of different tasks in the third job to avoid re-executing the set of continuous common tasks for the second job, wherein the executing further comprises executing the first job until encountering the fork command, forking to the third job while continuing execution the first job, and joining the third job back to the first job upon encountering the join command.
14 . The non-transitory computer readable medium of claim 13 , wherein the processing device is to:
identify, in the first job, the fork location between the set of continuous common tasks and the set of different tasks; and
insert the fork command at the fork location in the first job, wherein the fork command adds a branch to the first job that branches to the third job subsequent to the executing of the set of continuous common tasks.
15 . The non-transitory computer readable medium of claim 14 , wherein the set of continuous common tasks is a first set of continuous common tasks, the processing device is to: and
identify a second set of continuous common tasks subsequent to the set of different tasks;
insert, in the first job, the join command between the set of different tasks and the second set of continuous common tasks.
16 . The non-transitory computer readable medium of claim 14 , wherein the set of continuous common tasks are included in both the plurality of first tasks and the plurality of second tasks in a same order, and the set of different tasks are included in the plurality of second tasks and are absent from the plurality of first tasks.
17 . The non-transitory computer readable medium of claim 13 , wherein the metadata indicates one or more similarities between the first job and the second job, and also indicates one or more differences between the first job and the second job, and wherein the processing device is to:
insert a third task into the third job based on the one or more differences between the first job and the second job.
18 . The non-transitory computer readable medium of claim 13 , wherein the processing device is to:
capture one or more snapshots of the first job during the executing;
receive a request to execute a fourth job;
determine that the fourth job comprises a set of third continuous tasks corresponding to at least one of the one or more snapshots;
modify the fourth job by replacing the set of fourth continuous tasks with the at least one of the one or more snapshots; and
execute the modified fourth job.