Method and system for software application testing recommendations
A method for application testing recommendation includes deploying a software application to first testers having first system configurations, and receiving testing coverage data describing (i) a first subset of features of the software application being tested by the first testers and (ii) information describing the first system configurations. The method further includes deploying the software application to a marketplace, and receiving market coverage data describing (i) a second subset of features being used by market users and (ii) second system configurations of the market users. The method further includes comparing the market coverage data and the testing coverage data to identify a coverage discrepancy, selecting second testers to test the software application based on the coverage discrepancy, and deploying the software application to the second testers.
1. A computer-implemented method comprising:
receiving, through a software developer user interface, data associated with a software application to test;
determining that a criteria is satisfied by a difference between (i) testing coverage data that describes an amount of feature testing of the software application and system configurations of different systems executing the application software and (ii) market coverage data that describes a second amount of feature usage of the software application and a second set of system configurations of different systems executing the application software; and
in response to determining that the criteria is satisfied by the difference between (i) the testing coverage data that describes the amount of feature testing of the software application and the system configurations of different systems executing the application software and (ii) the market coverage data that describes the second amount of feature usage of the software application and the second set of system configurations of different systems executing the application software, providing, on the software developer user interface, one or more controls that are each associated with a respective recommended additional user, from among a set of additional users, to test the software application;
receiving, through the software developer user interface, a selection of a particular control that is associated with a particular recommended additional user; and
providing the software application to the particular recommended additional user to test in response to receiving the selection of the particular control.
2. The computer-implemented method of claim 1 , wherein the difference between (i) testing coverage data that describes an amount of feature testing of the software application and system configurations of different systems executing the application software and (ii) market coverage data that describes a second amount of feature usage of the software application and a second set of system configurations of different systems executing the application software, reflects a coverage discrepancy that indicates a portion of the market coverage data that is not in the testing coverage data.
3. The computer-implemented method of claim 2 , wherein the coverage discrepancy exists in response to determining that a portion of the market coverage data that is not in the testing coverage data satisfies a predetermined threshold.
4. The computer-implemented method of claim 2 , wherein the selection of the particular control that is associated with the particular recommended tester is in response to obtaining the particular recommended tester based on a type of difference in the coverage discrepancy.
5. The computer-implemented method of claim 2 , further comprising:
identifying defects during the test of an execution of the software application by the respective recommended tester; and
providing a recommendation to correct the coverage discrepancy based on the identified defects during the test of the execution of the software application.
6. The computer-implemented method of claim 2 , wherein the comparison of (i) the testing coverage data that describes the amount of feature testing on the different types of the system configures of the software application, and (ii) the market coverage data that describes the amount of the feature usage on the different types of the system configurations of the software application further comprises:
determining analytic data types associated with the testing coverage data and the market coverage data; and
determining a mapping to a numerical data format in response to determining at least one of the analytic data types is of a subjective data format.
7. The computer-implemented method of claim 1 , further comprising:
receiving real time analytics during the test of the execution of the software application by the particular recommended tester, wherein the real time analytics describe time-based data, geographic based data, defect based data, and input/output data associated with the execution of the software application.
8. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving, through a software developer user interface, data associated with a software application to test;
determining that a criteria is satisfied by a difference between (i) testing coverage data that describes an amount of feature testing of the software, application and system configurations of different systems executing the application software and (ii) market coverage data that describes a second amount of feature usage of the software application and a second set of system configurations of different systems executing the application software; and
in response to determining that the criteria is satisfied by the difference between (i) the testing coverage data that describes the amount of feature testing of the software application and the system configurations of different systems executing the application software and (ii) the market coverage data that describes the second amount of feature usage of the software application and the second set of system configurations of different systems executing the application software, providing, on the software developer user interface, one or more controls that are each associated with a respective recommended additional user, from among a set of additional users, to test the software application;
receiving, through the software developer user interface, a selection of a particular control that is associated with a particular recommended additional user; and
providing the software application to the particular recommended additional user to test in response to receiving the selection of the particular control.
9. The system of claim 8 , wherein the difference between (i) testing coverage data that describes an amount of feature testing of the software application and system configurations of different systems executing the application software and (ii) market coverage data that describes a second amount of feature usage of the software application and a second set of system configurations of different systems executing the application software, reflects a coverage discrepancy that indicates a portion of the market coverage data that is not in the testing coverage data.
10. The system of claim 9 , wherein the coverage discrepancy exists in response to determining that a portion of the market coverage data that is not in the testing coverage data satisfies a predetermined threshold.
11. The system of claim 9 , wherein the selection of the particular control that is associated with the particular recommended tester is in response to obtaining the particular recommended tester based on a type of difference in the coverage discrepancy.
12. The system of claim 9 , wherein the operations further comprise:
identifying defects during the test of an execution of the software application by the respective recommended tester; and
providing a recommendation to correct the coverage discrepancy based on the identified defects during the test of the execution of the software application.
13. The system of claim 9 , wherein the comparison of (i) the testing coverage data that describes the amount of feature testing on the different types of the system configures of the software application, and (ii) the market coverage data that describes the amount of the feature usage on the different types of the system configurations of the software application the operations further comprise:
determining analytic data types associated with the testing coverage data and the market coverage data; and
determining a mapping to a numerical data format in response to determining at least one of the analytic data types is of a subjective data format.
14. The system of claim 8 , wherein the operations further comprise:
receiving real time analytics during the test of the execution of the software application by the particular recommended tester, wherein the real time analytics describe time-based data, geographic based data, defect based data, and input/output data associated with the execution of the software application.
15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving, through a software developer user interface, data associated with a software application to test;
based on a comparison of (i) testing coverage data that describes an amount of feature testing on different types of system configurations of the software application, and (ii) market coverage data that describes an amount of feature usage on different types of system configurations of the software application, providing, on the software developer user interface, one or more controls that are each associated with a respective recommended tester, from among a set of testers, to test the software application;
receiving, through the software developer user interface, a selection of a particular control that is associated with a particular recommended tester;
providing the software application to the particular recommended tester to test in response to receiving the selection of the particular control; and
receiving real time analytics during the test of an execution of the software application by the particular recommended tester, wherein the real time analytics describe time-based data, geographic based data, defect based data, and input/output data associated with the execution of the software application.
16. The computer-readable medium of claim 15 , wherein the comparison of (i) testing the coverage data that describes the amount of feature testing on the different types of the system configures of the software application, and (ii) the market coverage data that describes the amount of the feature usage on the different types of the system configurations of the software application, reflects a coverage discrepancy that indicates a portion of the market coverage data that is not in the testing coverage data.
17. The computer-readable medium of claim 16 , wherein the coverage discrepancy exists in response to determining that a portion of the market coverage data that is not in the testing coverage data satisfies a predetermined threshold.
18. The computer-readable medium of claim 16 , wherein the selection of the particular control that is associated with the particular recommended tester is in response to obtaining the particular recommended tester based on a type of difference in the coverage discrepancy.
19. The computer-readable medium of claim 16 , wherein the operations further comprise:
identifying defects during the test of an execution of the software application by the respective recommended tester; and
providing a recommendation to correct the coverage discrepancy based on the identified defects during the test of the execution of the software application.
20. The computer-readable medium of claim 16 , wherein the comparison of (i) the testing coverage data that describes the amount of feature testing on the different types of the system configures of the software application, and (ii) the market coverage data that describes the amount of the feature usage on the different types of the system configurations of the software application the operations further comprise:
determining analytic data types associated with the testing coverage data and the market coverage data; and
determining a mapping to a numerical data format in response to determining at least one of the analytic data types is of a subjective data format.