Method and system for performance profiling of software

1. A computer-implemented method of performance profiling software, comprising:

producing a plurality of performance profiles of a software processing unit;

performing statistical analysis on the plurality of performance profiles;

determining a plurality of confidence intervals for the plurality of performance profiles; and

determining a level of reliability metric for a performance profile from the plurality of confidence intervals.

2. The method of claim 1 , wherein the step of performing statistical analysis comprises matching the performance profiles with at least one additional performance profile.

3. The method of claim 2 , further comprising storing in the performance profiles an indication of a matching strategy used in the step of matching.

4. The method of claim 2 , wherein the step of matching comprises matching data related to at least one of processes, threads, and similar processing units, over the plurality of performance profiles.

5. The method of claim 2 , wherein the step of matching uses a matching strategy based on at least one of matching on process unit identifiers, simple matching on processing duration, matching on processing duration through clustering, and labeled matching at a predetermined level.

6. The method of claim 2 , further comprising preprocessing based on at least one of processing unit processor affinity, containment of predetermined characterized data, and processing unit relevance.

7. The method of claim 2 , further comprising storing in the performance profile a unique identifier correlated to a matching strategy used in the step of matching.

8. The method of claim 1 , further including:

determining if external noise in the performance profile does not exceed a predetermined limit, where external noise is defined as a variation in the performance profile caused by other processing units and applications; and

conducting the statistical analysis based on the external noise determining.

9. The method of claim 8 , wherein the step of determining if external noise in the performance profile does not exceed a predetermined limit comprises at least one of:

determining processing duration of a single processing unit,

determining processing duration of the performance profile,

testing for same matching strategy across the plurality of profiles,

testing for same environment across the plurality of profiles, and

testing for maximal deviation in processing duration across the plurality of profiles.

10. The method of claim 1 , wherein the step of determining a level of reliability metric further comprises:

determining an amount of processing necessary to produce the level of reliability metric having a value greater than a predetermined value.

11. The method of claim 10 , further comprising applying a Bonferroni correction.

12. A computer program product tangibly embedded in a storage unit, comprising computer program means for instructing a computer to perform the method of claim 1 .

13. A system for performance profiling software, comprising:

means for producing a plurality of performance profiles of a software processing unit;

means for performing statistical analysis on the plurality of performance profiles;

means for determining a plurality of confidence intervals for the plurality of performance profiles; and

means for determining a level of reliability metric for a performance profile from the plurality of confidence intervals.

14. The system of claim 13 , wherein the means for performing statistical analysis comprises means for matching the performance profiles with at least one additional performance profile.

15. The system of claim 14 , further comprising means for storing in the performance profiles an indication of a matching strategy used by the means for matching.

16. The system of claim 14 , wherein the means for matching comprises means for matching data related to at least one of processes, threads, and similar processing units, over the plurality of performance profiles.

17. The system of claim 14 , wherein the means for matching is arranged to use a matching strategy based on at least one of:

matching on process unit identifiers,

simple matching on processing duration,

matching on processing duration through clustering, and

labeled matching at a predetermined level.

18. The system of claim 14 , further comprising means for preprocessing based on at least one of:

processing unit processor affinity,

containment of predetermined characterized data, and

processing unit relevance.

19. The system of claim 14 , further comprising means for storing in the performance profile an identifier correlated to a matching strategy used by the means for matching.

20. The system of claim 13 , further including:

means for determining if external noise in the performance profile does not exceed a predetermined limit, where external noise is defined as a variation in the performance profile caused by other processing units and applications; and

means for conducting the statistical analysis based on the external noise determining.

21. The system of claim 20 , wherein the means for determining if external noise in the performance profile does not exceed a predetermined limit comprises means for at least one of:

determining processing duration of a single processing unit,

determining processing duration of the performance profile,

testing for same matching strategy across the plurality of profiles,

testing for same environment across the plurality of profiles, and

testing for maximal deviation in processing duration across the plurality of profiles.

22. The system of claim 13 , wherein the means for determining a level of reliability metric further comprises:

means for determining an amount of processing necessary to produce the level of reliability metric having a value greater than a predetermined value.

23. The system of claim 22 , further comprising means for applying a Bonferroni correction.