Optimized test case selection for quality assurance testing of video games

1. A test case selection system, comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to:

identify a plurality of test cases for testing a software application;

determine, based at least in part on a plurality of input data associated with the software application and a test case selection model defined by one or more weights associated with the input data, a first subset of the plurality of test cases for testing the software application;

select, by random selection, a second subset of the plurality of test cases for testing the software application;

receive a first set of outcomes associated with the first subset of the plurality of test cases;

receive a second set of outcomes associated with the second subset of the plurality of test cases; and

generate, based at least in part on the first set of outcomes and the second set of outcomes, a comparison between the test case selection model and the random selection of test cases.

2. The test case selection system of claim 1 , wherein the test case selection model is a regression model, and wherein the weights are determined by a machine learning algorithm.

3. The test case selection system of claim 1 , wherein to generate the comparison between the test case selection model and the random selection of test cases, the computer-executable instructions further cause the one or more processors to:

determine a first probability of detecting a bug from testing the software application using the first subset of the plurality of test cases; and

determine a second probability of detecting a bug from testing the software application using the second subset of the plurality of test cases.

4. The test case selection system of claim 1 , wherein the computer-executable instructions further cause the one or more processors to:

determine, based at least in part on a regression model, the first set of outcomes, and the second set of outcomes, a second set of one or more weights; and

update the test case selection model with the second set of one or more weights as an updated test case selection model.

5. The test case selection system of claim 4 , wherein the computer-executable instructions further cause the one or more processors to:

determine, based at least in part on the updated test case selection model, a third subset of the plurality of test cases for testing the software application; and

receive a third set of outcomes associated with the third subset of the plurality of test cases; and

generate, based at least in part on the second set of outcomes and the third set of outcomes, a comparison between the test case selection model and the updated test case selection model.

6. The test case selection system of claim 1 , wherein the computer-executable instructions further cause the one or more processors to:

identify a particular test case from the first subset of the plurality of test cases that is to be allocated to an automated testing system; and

send an indication of the particular test case to the automated testing system, wherein a particular outcome associated with the particular test case is received from the automated testing system.

7. The test case selection system of claim 6 , wherein the computer-executable instructions further cause the one or more processors to:

identify a testing parameter value associated with the particular test case;

determine, based at least in part on the first set of outcomes, an updated testing parameter value; and

update the particular test case with the updated testing parameter value.

8. The test case selection system of claim 1 , wherein to receive the first set of outcomes associated with the first subset of the plurality of test cases, the computer-executable instructions further cause the one or more processors to:

receive indications of whether a bug was detected from testing the software application using individual ones of the first subset of the plurality of test cases.

9. A test case selection method, comprising:

determining, based at least in part on a plurality of input data associated with a software application and a test case selection model defined by first set of weights associated with the input data, a first subset of test cases selected from a plurality of test cases for testing the software application;

sending an indication that the first subset of test cases are to be used to test the software application;

receiving a first set of outcomes associated with the first subset of test cases, the first set of outcomes indicating a number of bugs detected as a result of the software application being tested with the first subset of test cases;

determining, based at least in part on a regression model and the first set of outcomes, a second set of weights;

updating the test case selection model with the second set of weights as an updated test case selection model;

determining, by random selection, a second subset of test cases selected from the plurality of test cases for testing the software application;

sending an indication that the second subset of test cases are to be used to test the software application;

receiving a second set of outcomes associated with the second subset of test cases; and

generating, based at least in part on the first set of outcomes and the second set of outcomes, a comparison between the test case selection model and the random selection of test cases.

10. The test case selection method of claim 9 , further comprising:

determining, based at least in part on the first set of outcomes, a first probability of detecting a bug from testing the software application using the first subset of test cases; and

determining, based at least in part on the second set of outcomes, a second probability of detecting a bug from testing the software application using the second subset of test cases, wherein the comparison between the test case selection model and the random selection of test cases includes the first probability and the second probability.

11. The test case selection method of claim 10 , further comprising:

determining, based at least in part on the updated test case selection model, a third subset of test cases selected from the plurality of test cases for testing the software application; and

receiving a third set of outcomes associated with the third subset of test cases; and

generating, based at least in part on the second set of outcomes and the third set of outcomes, a comparison between the test case selection model and the updated test case selection model.

12. The test case selection method of claim 11 , further comprising:

determining, based at least in part on the regression model and the third set of outcomes, a third set of weights; and

updating the updated test case selection model with the third set of weights as a second updated test case selection model.

13. The test case selection method of claim 9 , wherein the first set of outcomes indicates which of the first subset of test cases resulted in a bug detection when used to test the software application.

14. A system, comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to:

receive a first set of outcomes associated with a first subset of test cases selected from a plurality of test cases to test a software application, the first set of outcomes indicating a number of bugs detected as a result of the software application being tested with the first subset of test cases;

determine, based at least in part on the first set of outcomes, a set of weights associated with a test case selection model; and

deploy the test case selection model by updating a prior set of weights with the set of weights;

select, by random selection, a second subset of the plurality of test cases for testing the software application;

receive a second set of outcomes associated with the second subset of the plurality of test cases; and

generate, based at least in part on the first set of outcomes and the second set of outcomes, a comparison between the prior set of weights and the random selection of test cases.

15. The system of claim 14 , wherein the computer-executable instructions further cause the one or more processors to:

determine, based at least in part on the test case selection model, a second subset of test cases selected from the plurality of test cases;

instruct testing of the software application using the second subset of test cases;

receive a second set of outcomes associated with the second subset of test cases; and

generate, based at least in part on the first set of outcomes and the second set of outcomes, a comparison between the set of weights and the prior set of weights.

16. The system of claim 15 , wherein the computer-executable instructions further cause the one or more processors to:

determining, based at least in part on the second set of outcomes, a second set of weights associated with a second test case selection model; and

deploy the second test case selection model by updating the set of weights with the second set of weights.

17. The system of claim 14 , wherein the set of weights is based at least in part on the second set of outcomes.

18. The system of claim 14 , wherein the first set of outcomes indicates which of the first subset of test cases resulted in a bug detection when used to test the application software.

19. The method of claim 9 , further comprising:

identifying a particular test case from the first subset of the plurality of test cases that is to be allocated to an automated testing system; and

sending an indication of the particular test case to the automated testing system, wherein a particular outcome associated with the particular test case is received from the automated testing system.

20. The system of claim 14 , wherein the computer-executable instructions further cause the one or more processors to:

identify a particular test case from the plurality of test cases that is to be allocated to an automated testing system; and

send an indication of the particular test case to the automated testing system, wherein a particular outcome associated with the particular test case is received from the automated testing system.