AI software testing system and method
A system for performing software testing uses machine learning to extract features from a user interface of an app, classify screen types and screen elements of the user interface, and implement flows of test sequences to test the app. Training is performed to train the system to learn common application states of an application graph and to navigate through an application. In some implementations, the training includes Q-learning to learn how to navigate to a selected screen state. In some implementations, there is reuse of classifiers cross-application and cross platform.
1. An artificial intelligence software testing system to test software applications having a sequence of screens with each screen having a set of screen elements, comprising:
a set of intelligent machine learning bots trained to:
crawl through a software application;
identify screen types and screen elements of the screens using a set of trained classifiers; and
apply test cases to the software application;
wherein the software application has an associated logical state graph associated with potential user interactions with a graphical user interface and the set of intelligent machine learning bots are trained to determine application states and sequences of states associated with the logical state graph.
2. The system of claim 1 , wherein applying test cases comprises:
identify test cases based on the identified screen types and screen elements;
apply the identified test cases to the software application; and
report test results for the software application.
3. The system of claim 1 , wherein the set of intelligent machine learning bots are trained to:
identify test cases based on the identified screen types, screen elements, and associated application states and sequences of states of the logical state graph.
4. The system of claim 1 , wherein the set of classifiers are trained to:
analyze a visual appearance of screens using at least one classifier trained to analyze a visual appearance of graphical user interfaces;
identify screen types and screen elements based at least in part on a visual appearance.
5. The system of claim 1 , wherein the set of classifiers are trained to:
determine a screen type based at least in part on a visual appearance of the screen based on an image classification.
6. The system of claim 1 , wherein the set of classifiers are trained to analyze elements, screens, and flows of applications.
7. The system of claim 1 , wherein the set of intelligent bots are further trained to report performance of the software application.
8. The system of claim 1 , wherein the system is configured to test software apps for different platforms by applying a conversion table to adjust the testing for differences in software application appearance and formatting on different platforms, devices, screen sizes, and screen densities.
9. The system of claim 1 , wherein the set of classifiers are trained to analyze images on the screens to identify displayable user interface elements of the software application.
10. The system of claim 1 , further comprising generating training data for the system based on an initial set of labelled training data for a set of training apps and providing ongoing training data based on feedback from ongoing testing of software apps.
11. The system of claim 1 , wherein the system is trained to:
identify a set of interactable user interface screen elements; and
determine navigational paths between at least two different screen types.
12. An artificial intelligence software testing system to test software applications having a sequence of screens with each screen having a set of screen elements, comprising:
a set of intelligent machine learning bots trained to:
crawl through a software application;
identify screen types and screen elements of the screens using a set of trained classifiers; and
apply test cases to the software application;
wherein the set of classifiers are trained to recognize screens and screen elements common to a class of software applications having common screen states in a nodal state graph.
13. The system of claim 12 , wherein the nodal graph includes at least one of search screen node, a shopping cart screen node, a sign-in screen node, a sign-out screen node, a product screen node and a checkout screen node.
14. An artificial intelligence software testing system to test software applications having a sequence of screens with each screen having a set of screen elements, comprising:
a set of intelligent machine learning bots trained to:
crawl through a software application;
identify screen types and screen elements of the screens using a set of trained classifiers; and
apply test cases to the software application;
wherein the software application is represented by an abstract node graph and the set of intelligent machine learning bots includes a set of trained machine learning engine each trained for a subgraph of the node graph.
15. A computer-implemented method to test software applications having a sequence of screens with each screen having a set of screen elements, using a set of intelligent machine learning bots trained to perform a method comprising:
crawling through a software application;
identifying screen types and screen elements of the screens using a set of trained classifiers; and
applying test cases to the software application;
wherein applying test cases includes identifying test cases based on the identified screen types and screen elements, applying the identified test cases to the software application, and reporting test results for the software application.
16. The method of claim 15 , wherein the method comprises the set of classifiers:
analyzing a visual appearance of screen using at least one classifier trained to analyze a visual appearance of graphical user interfaces;
identifying screen types and screen elements based at least in part on a visual appearance.
17. The method of claim 15 , comprising:
determining a screen type based at least in part on a visual appearance of the screen based on an image classification.
18. The method of claim 15 , wherein the set of classifiers are trained to analyze elements, screens, and flows of applications.
19. The method of claim 15 , wherein the set of intelligent bots are further trained to report performance of the software application.
20. The method of claim 15 , further comprising testing software apps for different platforms by applying a conversion table to adjust the testing for differences in software application appearance and formatting on different platforms, devices, screen sizes, and screen densities.
21. The method of claim 15 , wherein the set of classifiers are trained to analyze images on the screens to identify displayable user interface elements of the software application.
22. The method of claim 15 , further comprising generating training data for the system based on an initial set of labelled training data for a set of training apps and providing ongoing training data based on feedback from ongoing testing of software apps.
23. The method of claim 15 , comprising training the system to:
identify a set of interactable user interface screen elements of an app; and
determine navigational paths between at least two different screen types.
24. A computer-implemented method to test software applications having a sequence of screens with each screen having a set of screen elements, using a set of intelligent machine learning bots trained to perform a method comprising:
crawling through a software application;
identifying screen types and screen elements of the screens using a set of trained classifiers; and
applying test cases to the software application;
wherein the software application has an associated logical state graph associated with potential user interactions with a graphical user interface and the method further comprises determining application states and sequences of states associated with the logical state graph.
25. The method of claim 24 , wherein the method further comprises:
identifying test cases based on the identified screen types, screen elements, and associated application states and sequences of states of the logical state graph.
26. A computer-implemented method to test software applications having a sequence of screens with each screen having a set of screen elements, using a set of intelligent machine learning bots trained to perform a method comprising:
crawling through a software application;
identifying screen types and screen elements of the screens using a set of trained classifiers, including recognizing screens and screen elements common to a class of software applications having common screen states in a nodal state graph; and
applying test cases to the software application.
27. The method of claim 26 , wherein the nodal graph includes at least one of search screen node, a shopping cart screen node, a sign-in screen node, a sign-out screen node, a product screen node and a checkout screen node.
28. A computer-implemented method to test software applications having a sequence of screens with each screen having a set of screen elements, using a set of intelligent machine learning bots trained to perform a method comprising:
crawling through a software application;
identifying screen types and screen elements of the screens using a set of trained classifiers; and
applying test cases to the software application;
wherein the method includes representing the software application by an abstract node graph and the set of intelligent machine learning bots includes a set of trained machine learning engines each trained for a subgraph of the node graph.