Logistics of stress testing
A method of validating the end to end performance of a mobile device including performance of applications, related services, and the mobile device by automating the conditions of the End User environment. The method is accomplished by leveraging stress models and applying these to automate the stressing of applications, their related services, while the device performance is being monitored by varying the emulated radio network and packet data network conditions.
1. A method of testing a mobile device, said method comprising:
receiving user input to configure an automated workflow for testing a mobile device; and
executing said automated workflow, wherein said executing said automated workflow comprises:
generating a test sequence for simulating a stress environment according to a stress model;
generating a play sequence associated with said test sequence;
executing said play sequence on said mobile device; and
analyzing data transmitted from said mobile device that results from said executing said play sequence.
2. The method of claim 1 , wherein said stress model is one of a Forward Stress Testing Model, a Reverse Stress Testing Model, and a Field Trial Stress Model.
3. The method of claim 1 , wherein said stress environment comprises simulated end user events.
4. The method of claim 1 , wherein said executing said automated workflow further comprises: selecting a test type; and editing said test sequence.
5. The method of claim 1 , wherein said executing said automated workflow further comprises:
extracting requirements from a test plan; and
generating a test case according to said requirements.
6. The method of claim 5 , wherein said executing said automated workflow further comprises:
recording said test case as a profile;
loading previously recorded profiles; and
generating said stress environment according to said recorded profiles.
7. The method of claim 1 , wherein said executing said automated workflow further comprises repeating said executing said play sequence until a predetermined stress goal is reached.
8. The method of claim 1 , wherein said analyzing data comprises: performing data mining and data collection; and monitoring proof of correction.
9. The method of claim 1 , wherein said data comprises a key performance indicator (KPI) of a battery, a processor or memory of said mobile device.
10. The method of claim 1 further comprising rendering a graphics user interface (GUI) for display on a display device, wherein said GUI comprises a script editor, and wherein said user input is received through said GUI.
11. The method of claim 1 , wherein said user input comprises: selection of said mobile device, an application installed in said mobile device: an emulated radio network condition; and connectivity to an external network.
12. The method of claim 1 , wherein said executing said automated workflow further comprises generating a report based on said analyzing said data.
13. The method of claim 1 , wherein said executing said automated workflow further comprises selecting a device driver according to a product model of said mobile device.
14. A system comprising:
a processor;
memory coupled to said processor and comprising instructions that, when executed by said processor, implement:
a work flow engine configured to generate a test case according to a test plan for testing a mobile device;
a playback engine configured to:
generate a test sequence for simulating a stress environment according to a stress mode; and
generate a play sequence associated with said test sequence;
execute said play sequence on said mobile device; and
an analytics engine configured to analyze data transmitted from said mobile device resulting from executing said play sequence.
15. The system of claim 14 , wherein said workflow engine is further configured to extract requirements from a test plan for generating said test case.
16. The system of claim 14 , wherein said workflow engine is further configured to: record generated test cases as profiles; and scan said profiles.
17. The system of claim 16 , wherein said playback engine is further configured to: load said profiles; and generate said stress environment according to said profiles.
18. The system of claim 14 , wherein said analytics engine is further configured to: collect said data; perform data mining; and report said data.
19. The system of claim 14 , wherein said playback engine is further configured to access a database to select a tool and a device driver to execute a command or function on said mobile device, and wherein said mobile device is coupled to said system remotely through a communication network.
20. The system of claim 14 , wherein said instructions further implement a graphics user interface (GUI) configured to receive user input related to a structured or non-structured script execution.