Cloud assisted behavioral automated testing
A computer readable storage medium, system and method for improving automated testing systems to include a first and second behavioral data. The first behavioral data is collected periodically and the second behavioral data is collected in real time. The receipt of the first behavioral data and a second behavioral data are followed by the receipt of a system configuration template. A test case is updated based on the first and second behavioral data, and an automated test environment is reconfigured based on the first behavioral data, second behavioral data, and the system configuration template. The test executes in the automated test environment producing a test result.
1. At least one computer readable storage device excluding propagating signals comprising instructions that, when executed, cause a programmable device to at least:
obtain first behavioral data and second behavioral data, the first behavioral data collected periodically and the second behavioral data collected in real time, the first and second behavioral data based on how an application was used on a first set of computing devices;
access a system configuration template, the system configuration template based at least in part on at least one user persona;
update a test case based on the first behavioral data and the second behavioral data;
alter an execution behavior of the application of first and second portions on a target set of computing devices by reconfiguring an automated test environment corresponding to the first behavioral data and the second behavioral data associated with the first set of computing devices, the reconfiguring to cause the first portion of the target set of computing devices to execute a first version of the application, and to cause the second portion of the target set of computing devices to execute a second version of the application; and
execute the updated test case in the reconfigured automated test environment to produce a test result.
2. The at least one computer readable storage device of claim 1 , wherein the instructions, when executed, cause the programmable device to at least:
obtain a pre-existing test case from a repository;
examine content of the pre-existing test case;
compare the content against the first behavioral data and the second behavioral data; and
create a new test case when the first behavioral data and second behavioral data are not referenced in the content.
3. The at least one computer readable storage device of claim 1 , wherein the instructions, when executed, cause the programmable device to at least:
determine input data based on the updated test case;
determine an input pattern based on the system configuration template; and
provide the input data to the automated test environment based on the input pattern.
4. The at least one computer readable storage device of claim 3 , wherein the instructions, when executed, cause the programmable device to at least:
evaluate a usage pattern associated with the system configuration template; and
convert the usage pattern into a set of time periods associated with a set of data rates.
5. The at least one computer readable storage device of claim 1 , wherein the system configuration template is based in part on a behavioral pattern according to the at least one user persona.
6. The at least one computer readable storage device of claim 5 , wherein the at least one user persona is a consumer role.
7. The at least one computer readable storage device of claim 1 , wherein the first behavioral data includes application data, usage data, and threat data.
8. The at least one computer readable storage device of claim 1 , wherein the instructions, when executed, cause the programmable device to-at least update an installed software stack in the automated test environment, the installed software stack determined at least in part by the first behavioral data.
9. A system to test an application on a target set of computing devices, the system comprising:
behavioral logic aggregators to aggregate data from behavioral data collector services;
system configuration templates, the system configuration templates based on at least one user persona and behavioral information collected from the behavioral data collector services;
one or more processors associated with one of more of the behavioral logic aggregators; and
memory including instructions that, when executed, cause at least one of the one or more processors to at least:
obtain first behavioral data and second behavioral data from one or more of the behavioral logic aggregators, the first behavioral data collected periodically and the second behavioral data collected in real time, the first and second behavioral data based on how the application was used on a first set of computing devices;
access a first one of the system configuration templates;
update a test case based on the first behavioral data and the second behavioral data;
alter an execution behavior of the application of first and second portions on a target set of computing devices by reconfiguring at least one automated test system corresponding to the first behavioral data and the second behavioral data associated with the first set of computing devices, the reconfiguring to cause the first portion of the target set of computing devices to execute a first version of the application, and to cause the second portion of the target set of computing devices to execute a second version of the application; and
execute the updated test case in the at least one automated test system to produce a test result.
10. The system of claim 9 , wherein the one or more processors is to:
access a pre-existing test case from a repository;
examine content of the pre-existing test case;
compare the content against the first behavioral data and the second behavioral data; and
create a new test case when the first behavioral data and the second behavioral data are not referenced in the content.
11. The system of claim 9 , wherein the at least one of the processors is to:
determine input data based on the updated test case;
determine an input pattern based on the system configuration template; and
provide the input data to the at least one automated test system based on the input pattern.
12. The system of claim 11 , wherein the at least one of the processors is to:
evaluate a usage pattern associated with the system configuration template; and
convert the usage pattern into a set of time periods associated with a set of data rates.
13. The system of claim 9 , wherein the system configuration template is based in part on a behavioral pattern according to the at least one user persona.
14. The system of claim 13 , wherein the at least one user persona is a consumer role.
15. The system of claim 9 , wherein the first behavioral data includes application data, usage data, and threat data.
16. The system of claim 9 , wherein the at least one of the processors is to update an installed software stack in the automated test system, the installed software stack determined at least in part by the first behavioral data.
17. A method comprising:
obtaining first behavioral data and second behavioral data, the first behavioral data collected periodically and the second behavioral data collected in real time, the first and second behavioral data based on how an application has been used on a first set of computing devices;
accessing a system configuration template based at least in part on at least one user persona;
updating a test case based on the first behavioral data and the second behavioral data;
altering an execution behavior of the application of first and second portions on a target set of computing devices by reconfiguring an automated test environment corresponding to the first behavioral data and the second behavioral data associated with the first set of computing devices, the reconfiguring to cause the first portion of the target set of computing devices to execute a first version of the application, and to cause the second portion of the target set of computing devices to execute a second version of the application; and
executing the updated test case in the reconfigured automated test environment to produce a test result.
18. The method of claim 17 , further including:
obtaining a pre-existing test case from a repository;
examining content of the pre-existing test case;
comparing the content against the first behavioral data and the second behavioral data; and
creating a new test case when the first behavioral data and the second behavioral data are not referenced in the content.
19. The method of claim 17 , further including:
determining input data based on the updated test case;
determining an input pattern based on the system configuration template; and
providing the input data to the automated test environment based on the input pattern.
20. The method of claim 17 , further including updating an installed software stack in the automated test environment, the installed software stack determined at least in part by the first behavioral data.