Method for performance analysis in a continuous integration pipeline

1. A method comprising:

executing a first set of files on a computing system, and collecting a first set of performance data based on the executing of the first set of files;

performing a software update on the first set of files to produce a second set of files;

executing the second set of files on the computing system and collecting a second set of performance data based on the executing of the second set of files;

identifying a first subset of the first set of performance data, the first subset corresponding to an execution of one or more first thread instances, the first thread instances being instantiated using the first set of files;

identifying a second subset of the second set of performance data, the second subset corresponding to an execution of one or more second thread instances, the second thread instances being instantiated using the second set of files;

calculating a score based on the first subset and the second subset, the score indicating a difference in resource consumption between the first thread instances and the second thread instances; and

generating and outputting a debugging message based on the score,

wherein the first thread instances and the second thread instances have at least one of the same entry function and the same opcode, and

wherein the first subset includes a plurality of first slices of performance data arranged in a first order, the second subset includes a plurality of second slices of performance data arranged in a second order, and the score is generated based on a difference between least one first slice and at least one second slice that have matching positions in the first order and the second order, respectively.

2. The method of claim 1 , wherein the difference includes a difference over average.

3. The method of claim 1 , wherein the debugging message is generated only when the score exceeds a threshold.

4. The method of claim 1 , further comprising:

generating a first directed graph based on the first subset, the first directed graph including a plurality of first nodes and a plurality of first edges, each first node corresponding to a different one of a plurality of synchronization objects, and each first edge corresponding to a respective first sequence or processor-executable that are executed between consecutive accesses of any two of the plurality of synchronization objects;

generating a second directed graph based on the second subset, the second directed graph including a plurality of second nodes and a plurality of second edges, each second node corresponding to a different one of the plurality of synchronization objects, and each second edge corresponding to a respective second sequence or processor-executable that are executed between consecutive accesses of any two of the plurality of synchronization objects; and

identifying a second edge in the second directed graph that does not have a matching counterpart in the first directed graph,

wherein the debugging message identifies at least a portion of a second sequence of processor-executable instructions that corresponds to the identified second edge, and

wherein the plurality of synchronization objects is defined in both the first set of files and the second set of files.

5. The method of claim 1 , wherein:

each of the first slices includes an indication of resource utilization during a respective time window in which one or more of the first thread instances are executed, and

each of the second slices includes an indication of resource utilization during a respective time window in which one or more of the second thread instances are executed.

6. The method of claim 1 , wherein each of the first slices and each of the second slices includes an indication of a utilization rate of a resource of the computing system.

7. The method of claim 1 , wherein the first thread instances and the second thread instances are instantiated using the same body of processor-executable instructions before and after the body of processor executable instructions is updated.

8. An apparatus comprising:

one or more processors; and

a memory configured to store one or more processor executable instructions, which when executed by the one or more processors cause the one or more processors to perform a method comprising the steps of:

executing a first set of files, and collecting a first set of performance data based on the executing of the first set of files;

performing a software update on the first set of files to produce a second set of files;

executing the second set of files and collecting a second set of performance data based on the executing of the second set of files;

identifying a first subset of the first set of performance data, the first subset corresponding to an execution of one or more first thread instances, the first thread instances being instantiated using the first set of files;

identifying a second subset of the second set of performance data, the second subset corresponding to an execution of one or more second thread instances, the second thread instances being instantiated using the second set of files;

calculating a score based on the first subset and the second subset, the score indicating a difference in resource consumption between the first thread instances and the second thread instances; and

generating and outputting a debugging message based on the score,

wherein the first thread instances and the second thread instances have at least one of the same entry function and the same opcode, and

wherein the first thread instances and the second thread instances are both instances of a set of one or more threads that is defined in both the first set of files and the second set of files

wherein the first subset includes a plurality of first slices of performance data arranged in a first order, the second subset includes a plurality of second slices of performance data arranged in a second order, and the score is generated based on a difference between at least one first slice and at least one second slice that have matching positions in the first order and the second order, respectively.

9. The apparatus of claim 8 , wherein the difference includes a difference over average.

10. The apparatus of claim 8 , wherein the debugging message is generated only when the score exceeds a threshold.

11. The apparatus of claim 8 , wherein the method further comprises:

generating a first directed graph based on the first subset, the first directed graph including a plurality of first nodes and a plurality of first edges, each first node corresponding to a different one of a plurality of synchronization objects, and each first edge corresponding to a respective first sequence or processor-executable that are executed between consecutive accesses of any two of the plurality of synchronization objects;

generating a second directed graph based on the second subset, the second directed graph including a plurality of second nodes and a plurality of second edges, each second node corresponding to a different one of the plurality of synchronization objects, and each second edge corresponding to a respective second sequence or processor-executable that are executed between consecutive accesses of any two of the plurality of synchronization objects; and

identifying a second edge in the second directed graph that does not have a matching counterpart in the first directed graph,

wherein the debugging message identifies at least a portion of a second sequence of processor-executable instructions that corresponds to the identified second edge, and

wherein the plurality of synchronization objects is defined in both the first set of files and the second set of files.

12. The apparatus of claim 8 , wherein:

each of the first slices includes an indication of resource utilization during a respective time window that overlaps with an execution of only one thread instance, and

each of the second slices includes an indication of resource utilization during a respective time window that overlaps with an execution of only one thread instance.

13. The apparatus of claim 8 , wherein each of the first slices and each of the second slices includes an indication of a utilization rate of a resource of the computing system.

14. The apparatus of claim 8 , wherein the first thread instances and the second thread instances are instantiated using the same body of processor-executable instructions before and after the body of processor executable instructions is updated.

15. A non-transitory computer-readable medium configured to store one or more processor-executable instructions, which when executed by at least one processor cause the at least one processor to perform a method comprising the steps of:

executing a first set of files on a computing system, and collecting a first set of performance data based on the executing of the first set of files;

performing a software update on the first set of files to produce a second set of files;

executing the second set of files on the computing system and collecting a second set of performance data based on the executing of the second set of files;

identifying a first subset of the first set of performance data, the first subset corresponding to an execution of one or more first thread instances, the first thread instances being instantiated using the first set of files;

identifying a second subset of the second set of performance data, the second subset corresponding to an execution of one or more second thread instances, the second thread instances being instantiated using the second set of files;

calculating a score based on the first subset and the second subset, the score indicating a difference in resource consumption between the first thread instances and the second thread instances; and

generating and outputting a debugging message based on the score,

wherein the first thread instances and the second thread instances have at least one of the same entry function and the same opcode, and

wherein the first subset includes only first slices that overlap with an execution of the first thread instances, and the second subset includes only second slices that overlap with an execution of the second thread instances.

16. The non-transitory computer-readable medium of claim 15 , wherein the difference includes a difference over average.

17. The non-transitory computer-readable medium of claim 15 , wherein the debugging message is generated only when the score exceeds a threshold.

18. The non-transitory computer-readable medium of claim 15 , wherein the method further comprises the steps of:

generating a first directed graph based on the first subset, the first directed graph including a plurality of first nodes and a plurality of first edges, each first node corresponding to a different one of a plurality of synchronization objects, and each first edge corresponding to a respective first sequence or processor-executable that are executed between consecutive accesses of any two of the plurality of synchronization objects;

generating a second directed graph based on the second subset, the second directed graph including a plurality of second nodes and a plurality of second edges, each second node corresponding to a different one of the plurality of synchronization objects, and each second edge corresponding to a respective second sequence or processor-executable that are executed between consecutive accesses of any two of the plurality of synchronization objects; and

identifying a second edge in the second directed graph that does not have a matching counterpart in the first directed graph,

wherein the debugging message identifies at least a portion of a second sequence of processor-executable instructions that corresponds to the identified second edge, and

wherein the plurality of synchronization objects is defined in both the first set of files and the second set of files.

19. The non-transitory computer-readable medium of claim 15 , wherein:

each of the first slices includes an indication of resource utilization during a respective time window that overlaps with an execution of only one thread instance, and

each of the second slices includes an indication of resource utilization during a respective time window that overlaps with an execution of only one thread instance.

20. A method comprising:

executing a first set of files on a computing system, and collecting a first set of performance data based on the executing of the first set of files;

performing a software update on the first set of files to produce a second set of files;

executing the second set of files on the computing system and collecting a second set of performance data based on the executing of the second set of files;

identifying a first subset of the first set of performance data, the first subset corresponding to an execution of one or more first thread instances, the first thread instances being instantiated using the first set of files;

identifying a second subset of the second set of performance data, the second subset corresponding to an execution of one or more second thread instances, the second thread instances being instantiated using the second set of files;

calculating a score based on the first subset and the second subset, the score indicating a difference in resource consumption between the first thread instances and the second thread instances; and

generating and outputting a debugging message based on the score,

wherein the first thread instances and the second thread instances have at least one of the same entry function and the same opcode,

wherein the first subset includes only first slices that overlap with an execution of the first thread instances, and the second subset includes only second slices that overlap with an execution of the second thread instances.