Method and system for predicting performance of software applications on prospective hardware architecture
View Patent ↗A system and method for identifying optimal system architectures for a reference application are provided. The system and method comprise executing a reference application and a plurality of test applications on a current system architecture and sampling performance data for each of the applications. The performance data is used to compute an application signature for each application. A similarity element is derived from the application signatures that illustrates the similarity between each application and every other application. Using a similarity threshold and an algorithm, a subset of test applications that are similar to the reference application are derived.
1. A computer implemented method for determining an optimal hardware architecture for a reference application, the method comprising:
collecting performance data, sampled at degradation events and corresponding to performance of executing applications on a current hardware architecture, the executing applications including the reference application and a plurality of test applications, wherein collecting performance data comprises sampling performance information at degradation events using a performance indicator, a degradation event type, and a plurality of hardware counters;
computing a similarity between the executing applications based on their application signatures and using the performance data; and
deriving, in a computer, using the computed similarity, a subset of test applications that are similar to the reference application based on performance, so as to select a plurality of hardware architectures, for testing the reference application, on which improved performance has been reported for the subset of test applications.
2. The computer implemented method of claim 1 , wherein performance data is sampled using a time series.
3. The computer method of claim 1 , wherein the application signatures correspond to degrees of correlation between the performance indicator and the degradation event type for each application.
4. The method of claim 1 , wherein the application signatures comprise a plurality of vectors with a dimension corresponding to a number of hardware counters.
5. The method of claim 4 , further comprising ranking a plurality of values within each of the vectors.
6. The method of claim 1 , wherein the computed similarity includes a matrix.
7. The computer implemented method of claim 1 wherein deriving a subset of test applications that are similar to the reference application based on performance comprises:
modeling, using a computer, the computed similarity as a cluster of nodes corresponding to the applications;
partitioning, using a computer, the cluster of nodes using a Minimum Spanning Tree algorithm; and
selecting, using a computer, a sub-cluster containing a node corresponding to the reference application.
8. The computer implemented method of claim 1 , wherein the degradation event type corresponds to at least one of resource stalls and cache misses and the performance indicator corresponds to at least one of cycles per instruction (CPI), latency per query, and throughput.
9. The computer implemented method of claim 3 , wherein the degrees of correlation between the performance indicator and the degradation event type are computed using a Pearson's correlation.
10. The computer implemented method of claim 1 , wherein the computed similarity between the executing applications is computed using a Spearman's correlation.
11. A non-transitory computer readable medium comprising a set of instructions which, when executed by a computer, cause the computer to execute steps for determining an optimal hardware architecture for a reference application, the steps comprising:
collecting, with a computer, performance data sampled at degradation events and corresponding to performance of executing applications on a current hardware architecture, the executing applications including the reference application and a plurality of test applications, wherein collecting performance data comprises sampling performance information at degradation events using a performance indicator, a degradation event type, and a plurality of hardware counters;
computing a similarity element between the executing applications based on their application signatures and using the performance data; and
deriving, in a computer, using the computed similarity, a subset of test applications that are similar to the reference application based on performance, so as to select a plurality of hardware architectures, for testing the reference application, on which improved performance has been reported for the subset of test applications.
12. The computer readable medium of claim 11 , wherein performance data is sampled using a time series.
13. The computer readable medium of claim 11 , wherein the application signatures correspond to degrees of correlation between the performance indicator and the degradation event type for each application.
14. The computer readable medium of claim 11 , wherein the application signatures comprise a plurality of vectors with a dimension corresponding to a number of hardware counters.
15. The computer readable medium of claim 14 , the steps further comprising ranking a plurality of values within each of the vectors.
16. The computer readable medium of claim 11 , wherein the computed similarity includes a matrix.
17. The computer readable medium of claim 11 wherein deriving a subset of test applications that are similar to the reference application based on performance comprises:
modeling, using a computer, the computed similarity as a cluster of nodes corresponding to the applications;
partitioning, using a computer, the cluster of nodes using a Minimum Spanning Tree algorithm; and
selecting, using a computer, a sub-cluster containing a node corresponding to the reference application.
18. The computer readable medium of claim 12 , wherein the degradation event type corresponds to at least one of resource stalls and cache misses and the performance indicator corresponds to at least one of cycles per instruction (CPI), latency per query, and throughput.
19. The computer readable medium of claim 13 , wherein the degrees of correlation between the performance indicator and the degradation event type are computed using a Pearson's correlation.
20. The computer readable medium of claim 11 , wherein the computed similarity between the executing applications is computed using a Spearman's correlation.