Self-optimization of computing environments

1. A method performed by one or more computers, the method comprising:

determining, by the one or more computers, different combinations of settings used by one or more server environments;

tracking, by the one or more computers, load levels and amounts of resources allocated to the one or more server environments;

monitoring, by the one or more computers, results achieved by the one or more server environments when using the different combinations of settings, wherein the monitored results include at least one of throughput, task completion times, or response times;

based on the monitored results, generating, by the one or more computers, a performance measure for each of the different combinations of settings, wherein the performance measure for each combination of settings is based on the throughput, task completion times, or response times achieved using the combination of settings, and wherein the performance measures are normalized to account for differences in the tracked load levels and amounts of resources allocated when the throughput, task completion times, or response times were achieved using the respective combinations of settings;

ranking, by the one or more computers, the different combinations of settings based on the performance measures that are normalized to account for differences in the tracked load levels and amounts of resources allocated and that respectively correspond to the different combinations of settings;

selecting, by the one or more computers, an updated set of settings for a particular server environment, wherein the updated set of settings is selected from the different combinations of settings based on the ranking of the different combinations of settings; and

providing, by the one or more computers, the updated set of settings that was selected for the particular server environment.

2. The method of claim 1 , wherein determining different combinations of settings used by one or more server environments comprises determining, for each of multiple server environments, a combination of settings used by the server environment.

3. The method of claim 1 , wherein the one or more server environments comprise multiple server environments, and wherein the method further comprises:

determining hardware resources allocated to the multiple server environments when results are monitored; and

determining load levels present at the multiple server environments when the results are monitored;

wherein generating the performance measure for each of the different combinations of settings comprises generating, for each of the different combinations of settings, a relative performance measure that is based on (i) results achieved by a server environment using the combination of settings, (ii) the hardware resources allocated to the server environment when the results were achieved, and (iii) the load levels present at the server environment when the results were achieved, wherein the relative performance measures are generated to each indicate performance that would be achieved using a same reference level of load and a same level of resource allocation; and

wherein ranking the different combinations of settings based on the performance measures comprises ranking the different combinations of settings based on the relative performance measures that indicate performance that would be achieved using a same reference level of load and a same level of resource allocation.

4. The method of claim 1 , comprising initiating, by the one or more computers, a set of tasks for each of the one or more server environments,

wherein monitoring, by the one or more computers, the results achieved by the one or more server environments when using the different combinations of settings comprises monitoring, for each of the one or more server environments, completion times for tasks in the set of tasks.

5. The method of claim 1 , wherein providing the updated set of settings for the particular server environment comprises initiating a change for the particular server environment to use the updated set of settings.

6. The method of claim 1 , comprising periodically changing the settings for the particular server environment on an ongoing basis as additional performance measures are generated.

7. The method of claim 1 , comprising making a series of multiple incremental changes to a configuration setting of the particular server environment, each of the incremental changes moving the configuration setting closer to a corresponding setting in the updated set of settings.

8. The method of claim 1 , comprising repeatedly performing, by the one or more server environments, a predetermined set of tasks,

wherein monitoring the results achieved by the one or more server environments comprises monitoring completion times for the set of tasks.

9. The method of claim 1 , comprising:

generating a copy of the particular server environment of the one or more server environments; and

testing the copy of the particular server environment when using multiple combinations of settings,

wherein determining the different combinations of settings used by the one or more server environments comprises determining multiple combinations of settings for the particular server environment, and

wherein monitoring the results achieved by the one or more server environments comprises monitoring performance of the copy of the server environment when using the multiple combinations of settings when performing a predetermined set of tasks.

10. The method of claim 1 , wherein the settings comprise one or more of caching settings, concurrency settings, or memory allocation settings.

11. The method of claim 1 , wherein monitoring the results achieved comprises monitoring completion times for generating or serving a predetermined set of multiple documents.

12. The method of claim 1 , wherein monitoring the results achieved comprises monitoring speed to process tests configured to appear as user requests to the one or more server environment.

13. The method of claim 1 , wherein generating a performance measure for each of the different combinations of settings comprises:

scaling performance levels indicated by the monitored results for different computer systems to obtain performance measures that represent levels of performance standardized to a consistent level of load and/or hardware resources.

14. The method of claim 1 , wherein:

the particular server environment is configured to use a first set of configuration settings; and

the updated set of settings is one of the different combinations of settings that is configured to provide higher performance than the first set of configuration settings.

15. The method of claim 1 , wherein:

the particular server environment is configured to use a first set of settings that corresponds to a first performance measure; and

the updated set of settings is one of the different combinations of settings and has a corresponding performance measure that indicates higher performance than the first performance measure.

16. The method of claim 1 , wherein:

the particular server environment is configured to use a first set of settings;

the updated set of settings is one of the different combinations of settings; and

the updated set of settings is selected based on the ranking indicating that the updated set of settings provides higher performance than the first set of settings.

17. A system comprising:

one or more computers; and

one or more computer-readable media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

determining, by the one or more computers, different combinations of settings used by one or more server environments;

tracking, by the one or more computers, load levels and amounts of resources allocated to the one or more server environments;

monitoring, by the one or more computers, results achieved by the one or more server environments when using the different combinations of settings, wherein the monitored results include at least one of throughput, task completion times, or response times;

based on the monitored results, generating, by the one or more computers, a performance measure for each of the different combinations of settings, wherein the performance measure for each combination of settings is based on the throughput, task completion times, or response times achieved using the combination of settings, and wherein the performance measures are normalized to account for differences in the tracked load levels and amounts of resources allocated when the throughput, task completion times, or response times were achieved using the respective combinations of settings;

ranking, by the one or more computers, the different combinations of settings based on the performance measures that are normalized to account for differences in the tracked load levels and amounts of resources allocated and that respectively correspond to the different combinations of settings;

selecting, by the one or more computers, an updated set of settings for a particular server environment, wherein the updated set of settings is selected from the different combinations of settings based on the ranking of the different combinations of settings; and

providing, by the one or more computers, the updated set of settings that was selected for the particular server environment.

18. The system of claim 17 , wherein determining different combinations of settings used by one or more server environments comprises determining, for each of multiple server environments, a combination of settings used by the server environment.

19. One or more computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

determining, by the one or more computers, different combinations of settings used by one or more server environments;

tracking, by the one or more computers, load levels and amounts of resources allocated to the one or more server environments;

monitoring, by the one or more computers, results achieved by the one or more server environments when using the different combinations of settings, wherein the monitored results include at least one of throughput, task completion times, or response times;

based on the monitored results, generating, by the one or more computers, a performance measure for each of the different combinations of settings, wherein the performance measure for each combination of settings is based on the throughput, task completion times, or response times achieved using the combination of settings, and wherein the performance measures are normalized to account for differences in the tracked load levels and amounts of resources allocated when the throughput, task completion times, or response times were achieved using the respective combinations of settings;

ranking, by the one or more computers, the different combinations of settings based on the performance measures that are normalized to account for differences in the tracked load levels and amounts of resources allocated and that respectively correspond to the different combinations of settings;

selecting, by the one or more computers, an updated set of settings for a particular server environment, wherein the updated set of settings is selected from the different combinations of settings based on the ranking of the different combinations of settings; and

providing, by the one or more computers, the updated set of settings that was selected for the particular server environment.