Identification of optimal cloud resources for executing workloads

1. A system comprising:

a processor to:

deduplicate historical client queries based on a workload selection configuration to determine a grouping of historical client queries;

generate a workload based on at least a portion of the grouping of historical client queries;

repeatedly execute a test run of the workload using resources of a cloud environment to determine whether there is a performance difference in the test run;

in response to determining that there is no performance difference, identify one or more sets of decreased resources of the cloud environment;

re-execute the test run using the one or more sets of decreased resources of the cloud environment to determine whether there is a performance difference in the test run that is attributed to the one or more sets of decreased resources of the cloud environment;

determine minimum resources of the cloud environment to repeatedly execute the test run using the minimum resources without existence of a performance difference in response to re-executing the test run using the one or more sets of decreased resources of the cloud environment; and

re-execute the test run using the minimum resources and additional resources without modifying the minimum resources to determine that the additional resources improve a performance in the test run.

2. The system of claim 1 , wherein the re-execute the test run using the one or more decreased resources of the cloud environment to determine whether there is a performance difference in the test run that is attributed to the one or more sets of decreased resources of the cloud environment comprises:

selecting a query of the test run based on an execution rate of the query associated with repeatedly executing the test run of the workload using the one or more decreased resources of the cloud environment.

3. The system of claim 1 , wherein the test run implements a feature that comprises procedural logic.

4. The system of claim 1 , wherein the processor to repeatedly execute the test run of the workload using the resources of the cloud environment to determine whether there is a performance difference in the test run comprises:

compare a baseline run of the workload and a target run of the workload to identify whether there is a performance difference in the target run, wherein the baseline run and the target run are compared based on the workload selection configuration comprising one or more parameters input by a user to filter out unwanted historical client queries.

5. The system of claim 1 , wherein the processor to:

prune the grouping of historical client queries to eliminate queries that consume more than a threshold amount of resources to execute; and

select a subset of the pruned grouping of historical client queries to determine a diverse sample by applying an algorithm to map the pruned grouping of historical client queries to a high dimensional vector space.

6. The system of claim 1 , wherein the test run implements a feature that modifies a function of a database.

7. The system of claim 1 , wherein the decreased resources of the cloud environment have a lower concurrency than the resources of the cloud environment.

8. The system of claim 1 , wherein the processor to execute the test run on a dedicated virtual warehouse of a cloud database service that is dedicated for performance of feature testing runs.

9. The system of claim 1 , wherein the processor further to determine a version of database data that existed when a historical client query was run, and wherein the processor to execute on the version of the database data that existed when the historical client query of the one or more historical client queries was run.

10. The system of claim 4 , wherein the test run implements a feature, and wherein parameters for the baseline run and parameters for the target run are identical except for a presence of the feature in the target run.

11. The system of claim 4 , wherein the processor is further to obfuscate results of the baseline run and the target run such that a user does not have visibility into any database data associated with the workload.

12. The system of claim 4 , wherein the processor to identify one or more errors in response to comparing the baseline run and the target run, the one or more errors comprising:

a wrong result to one or more historical client queries executed in the target run;

an internal error in the target run; or

a user error in the target run; and

wherein to execute the target run, the processor is further to identify whether any of the one or more errors still exists; and

wherein to flag the performance difference as a false positive, the processor is further to flag any of the one or more errors as a false positive in response to identifying that any of the one or more errors no longer exists when the target run is rerun.

13. The system of claim 12 , wherein the processor is further to generate a report comprising data about the target run, wherein the report does not comprise an indication of any performance difference or error that was flagged as a false positive.

14. A method, the method comprising:

deduplicating historical client queries based on a workload selection configuration to determine a grouping of historical client queries;

generating a workload based on at least a portion of the grouping of historical client queries;

repeatedly executing a test run of the workload using resources of a cloud environment to determine whether there is a performance difference in the test run;

in response to determining that there is no performance difference, identifying one or more sets of decreased resources of the cloud environment;

re-executing the test run using the one or more sets of decreased resources of the cloud environment to determine whether there is a performance difference in the test run that is attributed to the one or more sets of decreased resources of the cloud environment;

determining minimum resources of the cloud environment to repeatedly execute the test run using the minimum resources without existence of a performance difference in response to re-executing the test rune using the one or more sets of decreased resources of the cloud environment; and

re-executing the test run using the minimum resources and additional resources without modifying the minimum resources to determine that the additional resources improve a performance in the test run.

15. The method of claim 14 , wherein the re-executing the test run using the one or more decreased resources of the cloud environment to determine whether there is a performance difference in the test run that is attributed to the one or more sets of decreased resources of the cloud environment comprises:

selecting a query of the test run based on an execution rate of the query associated with repeatedly executing the test run of the workload using the one or more decreased resources of the cloud environment.

16. The method of claim 14 , wherein the test run implements a feature that comprises procedural logic.

17. The method of claim 14 , wherein repeatedly executing the test run of the workload using resources of the cloud environment to determine whether there is the performance difference in the test run comprises:

comparing a baseline run of the workload and a target run of the workload to identify whether there is a performance difference in the target run, wherein the baseline run and the target run are compared based on the workload selection configuration comprising one or more parameters input by a user to filter out unwanted historical client queries.

18. The method of claim 14 , further comprising:

pruning the grouping of historical client queries to eliminate queries that consume more than a threshold amount of resources to execute; and

selecting a subset of the pruned grouping of historical client queries to determine a diverse sample by applying an algorithm to map the pruned grouping of historical client queries to a high dimensional vector space.

19. The method of claim 14 , wherein the test run implements a feature that modifies a function of a database.

20. The method of claim 14 , wherein the test run implements a feature, and wherein parameters for the baseline run and parameters for the target run are identical except for a presence of the feature in the target run.

21. The method of claim 14 , wherein the decreased resources of the cloud environment have a lower concurrency than the resources of the cloud environment.

22. The method of claim 14 , further comprising:

executing the test run on a dedicated virtual warehouse of a cloud database service that is dedicated for performance of feature testing runs.

23. The method of claim 14 , further comprising identifying one or more errors in response to comparing the baseline run and the target run, the one or more errors comprising:

a wrong result to one or more historical client queries executed in the target run;

an internal error in the target run; or

a user error in the target run; and

wherein to execute the target run, the processor is further to identify whether any of the one or more errors still exists; and

wherein to flag the performance difference as a false positive, the processor is further to flag any of the one or more errors as a false positive in response to identifying that any of the one or more errors no longer exists when the target run is rerun.

24. The method of claim 23 , further comprising:

generating a report comprising data about the target run, wherein the report does not comprise an indication of any performance difference or error that was flagged as a false positive.

25. A non-transitory computer readable storage media, programmable to execute instructions that, when executed, cause a processor to:

deduplicate historical client queries based on a workload selection configuration to determine a grouping of historical client queries;

generate a workload based on at least a portion of the grouping of historical client queries;

repeatedly execute a test run of the workload using resources of a cloud environment to determine whether there is a performance difference in the test run;

in response to determining that there is no performance difference, identify one or more sets of decreased resources of the cloud environment;

re-execute the test run using the one or more sets of decreased resources of the cloud environment to determine whether there is a performance difference in the test run that is attributed to the one or more sets of decreased resources of the cloud environment;

determine minimum resources of the cloud environment to repeatedly execute the test run using the minimum resources without existence of a performance difference in response to re-executing the test run using the one or more sets of decreased resources of the cloud environment; and

re-execute the test run using the minimum resources and additional resources without modifying the minimum resources to determine that the additional resources improve a performance in the test run.

26. The non-transitory computer readable storage media of claim 25 , wherein the processor to re-execute the test run using the scaled resources of the cloud environment to identify whether the performance difference still exists comprises:

select a query of the test run based on an execution rate of the query associated with repeatedly executing the test run of the workload using the one or more decreased resources of the cloud environment.

27. The non-transitory computer readable storage media of claim 25 , wherein test run implements a feature that comprises procedural logic.

28. The non-transitory computer readable storage media of claim 25 , wherein the processor to repeatedly execute the test run of the workload using the resources of the cloud environment to determine whether there is a performance difference in the test run comprises:

compare a baseline run of the workload and a target run of the workload to identify whether there is a performance difference in the target run, wherein the baseline run and the target run are compared based on the workload selection configuration comprising one or more parameters input by a user to filter out unwanted historical client queries.

29. The non-transitory computer readable storage media of claim 25 , the processor to:

prune the grouping of historical client queries to eliminate queries that consume more than a threshold amount of resources to execute; and

select a subset of the pruned grouping of historical client queries to determine a diverse sample by applying an algorithm to map the pruned grouping of historical client queries to a high dimensional vector space.

30. The non-transitory computer readable storage media of claim 25 , the processor to:

execute the target run on a dedicated virtual warehouse of a cloud database service that is dedicated for performance of feature testing runs.