Risk-based test plan construction

1. A method comprising:

determining, by a computing device, a plurality of test cases to test an application;

determining, by the computing device, a set of attributes assigned to each test case in the plurality of test cases, wherein the set of attributes are determined without execution of each test case;

calculating, by the computing device, a test case risk score for each test case in the plurality of test cases based on the set of attributes associated with each respective test case, wherein the test case risk score quantifies a risk in not executing each respective test case;

selecting, by the computing device, a first subset of the plurality of test cases based on at least a portion of the calculated risk scores; and

outputting, by the computing device, a test plan including the first subset of the plurality of test cases along with a test plan risk score that quantifies the risk in not executing a second subset of the plurality of test cases not included in the first subset of the plurality of test cases, wherein the test plan does not include the second subset of the plurality of test cases.

2. The method of claim 1 , wherein the set of attributes comprise test case related attributes that characterize the test case.

3. The method of claim 2 , wherein test case related attributes comprise a test case dependency attribute describing a number of test cases that are dependent on a test case, a risk of failure attribute describing a risk of failure to a functionality of the software application, a use case popularity attribute describing how popular a use case being tested by the test case is, a test case override attribute indicating whether or not the test case should always be included in the subset of the plurality of test cases, and a test case execution time indicating an amount of resources needed to execute the test case.

4. The method of claim 1 , wherein the set of attributes comprise feature related attributes that characterize a feature of the application being tested by a respective test case.

5. The method of claim 4 , wherein the feature related attributes comprise a feature popularity attribute indicating a popularity of use for a feature being tested by a test case, and an age of the feature indicating an age of the feature being tested by the test case.

6. The method of claim 1 , further comprising weighting attributes in the set of attributes with different weights for each respective test case.

7. The method of claim 6 , wherein calculating the test case risk score comprises:

applying each attribute's respective weight to a value for each attribute for a test case to determine an attribute score for each attribute; and

using each of the attribute scores to determine the test case risk score for the test case.

8. The method of claim 1 , wherein each test case is associated with a time estimating resources used to execute the test case.

9. The method of claim 8 , wherein selecting the first subset of the plurality of test cases comprises:

determining a cumulative time for testing the application; and

selecting the first subset of the plurality of test cases based on the cumulative time and respective test case risk score associated with the first subset of the plurality of test cases.

10. The method of claim 9 , wherein selecting the first subset of the plurality of test cases comprises:

selecting test cases with a highest test case risk score until the cumulative time for testing is reached based on the time associated with each selected test case.

11. The method of claim 1 , wherein selecting the first subset of the plurality of test cases comprises:

determining an acceptable test case risk score for testing the application; and

selecting the first subset of the plurality of test cases based on the acceptable test case risk score.

12. The method of claim 1 , wherein outputting comprises outputting the lowest test case risk score associated with one of the first subset of the plurality of test cases as the test plan risk score.

13. The method of claim 1 , wherein calculating the test case risk score comprises:

determining an age attribute for a test case; and

when the age attribute indicates the test case is new or has not been tested before, assigning a test case risk score such that the test case is included in the first subset of the plurality of test cases.

14. The method of claim 1 , wherein calculating the test case risk score comprises:

determining an override attribute for a test case;

when the override attribute is set to a first value, assigning a test case risk score such that the test case is included in the first subset of the plurality of the test cases.

15. A non-transitory computer-readable storage medium containing instructions, that when executed, control a computer system to be configured for:

determining a plurality of test cases to test an application;

determining a set of attributes assigned to each test case in the plurality of test cases, wherein the set of attributes are determined without execution of each test case;

calculating a test case risk score for each test case in the plurality of test cases based on the set of attributes associated with each respective test case, wherein the test case risk score quantifies a risk in not executing each respective test case;

selecting a first subset of the plurality of test cases based on at least a portion of the calculated risk scores; and

outputting a test plan including the first subset of the plurality of test cases along with a test plan risk score that quantifies the risk in not executing a second subset of the plurality of test cases not included in the first subset of the plurality of test cases, wherein the test plan does not include the second subset of the plurality of test cases.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the set of attributes comprise feature related attributes that characterize a feature of the application being tested by a respective test case.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the set of attributes comprise feature related attributes that characterize a feature of the application being tested by a respective test case.

18. The non-transitory computer-readable storage medium of claim 15 , wherein calculating the test case risk score comprises:

determining an age attribute for a test case; and

when the age attribute indicates the test case is new or has not been tested before, assigning a test case risk score such that the test case is included in the first subset of the plurality of test cases.

19. The non-transitory computer-readable storage medium of claim 15 , wherein calculating the test case risk score comprises:

determining an override attribute for a test case;

when the override attribute is set to a first value, assigning a test case risk score such that the test case is included in the first subset of the plurality of test cases.

20. An apparatus comprising:

one or more computer processors; and

a non-transitory computer-readable storage medium comprising instructions, that when executed, control the one or more computer processors to be configured for:

determining a plurality of test cases to test an application;

determining a set of attributes assigned to each test case in the plurality of test cases, wherein the set of attributes are determined without execution of each test case;

calculating a test case risk score for each test case in the plurality of test cases based on the set of attributes associated with each respective test case, wherein the test case risk score quantifies a risk in not executing each respective test case;

selecting a first subset of the plurality of test cases based on at least a portion of the calculated risk scores; and

outputting a test plan including the first subset of the plurality of test cases along with a test plan risk score that quantifies the risk in not executing a second subset of the plurality of test cases not included in the first subset of the plurality of test cases, wherein the test plan does not include the second subset of the plurality of test cases.