Customizing computer performance tests
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for testing performance of server systems. In some implementations, a method includes identifying, by one or more computers, a resource provided by a server system. The method includes generating, by the one or more computers, a performance test for the server system. The performance test includes a series of operations that includes (i) providing the resource and (ii) responding to one or more interactions with the resource. The method includes performing one or more performance testing cycles that each include: causing the server system to perform the series of operations of the performance test; and evaluating one or more performance measures that characterize the performance of the series of operations by the server system.
1. A method performed by one or more computers, the method comprising:
recording, by the one or more computers, user action sequences performed by one or more users;
determining, by the one or more computers, that the one or more users performed a particular user action sequence at a frequency greater than a threshold frequency;
in response to determining that the one or more users performed a particular user action sequence at a frequency greater than a threshold frequency, developing, by the one or more computers, a performance test for a server system based on the particular user action sequence;
running, by the one or more computers, the performance test to obtain baseline performance data indicating performance of the server system in the performance test;
based on the baseline performance data, generating, by the one or more computers, success criteria for the performance test; and
after performing an additional run of the performance test for the server system, evaluating, by the one or more computers, performance of the server system in the additional run of the performance test using the generated success criteria.
2. The method of claim 1 , comprising:
identifying a resource provided by the server system, wherein:
the particular user action sequence includes one or more interactions with the resource, and
the performance test comprises a series of operations for the server system to perform including providing the resource and responding to the one or more interactions with the resource.
3. The method of claim 2 , wherein the baseline performance data comprises one or more performance measures based on:
a time duration to complete the series of operations;
a time duration to complete one or more individual operations in the series of operations;
an amount of errors occurring during the series of operations; or
an accuracy of a result of performing the series of operations.
4. The method of claim 2 , wherein the baseline performance data comprises a baseline time duration for performing the series of operations.
5. The method of claim 4 , wherein generating the success criteria for the performance test comprises determining a threshold time duration for performing the series of operations, the threshold time duration being greater than the baseline time duration for performing the series of operations.
6. The method of claim 1 , wherein running the performance test to obtain the baseline performance data comprises:
sending, from a management system to a server system, a request for the server system to perform the performance test; and
receiving, by the management system and from the server system, the baseline performance data.
7. The method of claim 1 , wherein the recorded user action sequences include user interactions with multiple resources provided by a server system, the method comprising:
evaluating the recorded user action sequences to identify a resource of the multiple resources that satisfies one or more criteria; and
developing the performance test to test the identified resource.
8. The method of claim 1 , wherein the performance test includes a series of operations that simulate actions including:
loading a particular document;
presenting a particular portion of the particular document; and
activating one or more interactive user interface controls available for the particular portion of the particular document.
9. The method of claim 1 , wherein the particular user action sequence includes at least one of:
opening a particular document;
viewing a particular portion of the particular document,
sorting data within the particular document; or
filtering data within the particular document.
10. The method of claim 1 , wherein the threshold frequency comprises a threshold number of times that the particular user action sequence is performed within a particular time duration.
11. The method of claim 1 , wherein running the performance test to obtain baseline performance data comprises performing multiple cycles of the performance test.
12. The method of claim 11 , wherein the baseline performance data includes an average of a performance measure over the multiple cycles of the performance test.
13. The method of claim 12 , wherein the baseline performance data includes an average time to perform each step of the performance test over the multiple cycles of the performance test.
14. The method of claim 1 , wherein evaluating performance of the server system using the success criteria comprises:
repeatedly running the performance test for the server system to obtain multiple sets of performance data for the server system; and
determining, for each of the multiple sets of performance data, whether the performance data satisfies the success criteria.
15. The method of claim 1 , wherein evaluating the performance of the server system using the success criteria comprises:
determining a test frequency for running the performance test on the server system; and
running the performance test on the server system at time intervals that are based on the test frequency.
16. The method of claim 15 , comprising determining the test frequency for running the performance test on the server system based on the frequency at which the one or more users performed the particular user action sequence.
17. A system comprising:
one or more computers; and
one or more computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
recording, by the one or more computers, user action sequences performed by one or more users;
determining, by the one or more computers, that the one or more users performed a particular user action sequence at a frequency greater than a threshold frequency;
in response to determining that the one or more users performed a particular user action sequence at a frequency greater than a threshold frequency, developing, by the one or more computers, a performance test for a server system based on the particular user action sequence;
running, by the one or more computers, the performance test to obtain baseline performance data indicating performance of the server system in the performance test;
based on the baseline performance data, generating, by the one or more computers, success criteria for the performance test; and
after performing an additional run of the performance test for the server system, evaluating, by the one or more computers, performance of the server system in the additional run of the performance test using the generated success criteria.
18. The system of claim 17 , comprising:
identifying a resource provided by the server system, wherein:
the particular user action sequence includes one or more interactions with the resource, and
the performance test comprises a series of operations for the server system to perform including providing the resource and responding to the one or more interactions with the resource.
19. The system of claim 18 , wherein the baseline performance data comprises one or more performance measures based on:
a time duration to complete the series of operations;
a time duration to complete one or more individual operations in the series of operations;
an amount of errors occurring during the series of operations; or
an accuracy of a result of performing the series of operations.
20. One or more non-transitory computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:
recording, by the one or more computers, user action sequences performed by one or more users;
determining, by the one or more computers, that the one or more users performed a particular user action sequence at a frequency greater than a threshold frequency;
in response to determining that the one or more users performed a particular user action sequence at a frequency greater than a threshold frequency, developing, by the one or more computers, a performance test for a server system based on the particular user action sequence;
running, by the one or more computers, the performance test to obtain baseline performance data indicating performance of the server system in the performance test;
based on the baseline performance data, generating, by the one or more computers, success criteria for the performance test; and
after performing an additional run of the performance test for the server system, evaluating, by the one or more computers, performance of the server system in the additional run of the performance test using the generated success criteria.