Techniques for distributed testing
View Patent ↗Techniques for distributed testing are provided. Resources are identified for performing tests over a network. The tests and policies are sent to the resources and a proxy. The proxy delivers data for the tests to the resources and enforces the policies during the tests. The proxy also gathers statistics and results from the resources, which are executing the tests, and the proxy reports the statistics and results to one or more third-party services for subsequent manipulation and analysis.
1. A computer-implemented method that is adapted to executed by a processor to perform the method, comprising:
using policies to identify a set of resources for performing one or more tests, wherein the resources are distributed over a network, each resource is a client machine that has a different configuration with a different operating system from that of remaining resources, each test is a script with a sequence of events defined to process in a predefined order and customized via the policies for each of the client machines;
making a proxy available to the resources, wherein the proxy delivers data to the one or more resources for use within the one or more tests, the proxy distributes the data on an as-requested basis to each of the one or more resources to ensure network bandwidth is not adversely impacted on the network;
controlling and monitoring the one or more tests from the proxy, the proxy is an active participant in each of the one or more tests; and
capturing results from the one or more tests and offloading the results and statistics for the one or more tests to a third-party service that collects, reports, and manages the statistics.
2. The method of claim 1 , wherein using further includes assigning tokens to the tests for verification of the tests, wherein the tokens include signatures for creators of the tests and security roles that are to be assigned to the tokens possessed by the tests.
3. The method of claim 2 , wherein assigning further includes, using by the set of resources the tokens to validate the one or more tests for execution when received by the set of resources.
4. The method of claim 1 , wherein using further includes compiling and configuring the tests as an execution plan for execution on the set of resources.
5. The method of claim 4 further comprising, saving the execution plan to replay as a repeatable process on demand.
6. The method of claim 5 , wherein using further includes saving identity information for the set of resources for authentication of the set of resources and their subsequent execution of the tests.
7. The method of claim 1 , wherein capturing further includes, aggregating and summarizing the statistics, by the third-part service to present reports and samples of the statistics to requestors.
8. A computer-implemented method that is adapted to executed by a processor to perform the method, comprising, comprising:
receiving an execution plan and policies for tests that are to be executed on multiple clients of a network, wherein at least one of the clients is a virtual machine;
enforcing the policies for the execution plan on each of the multiple clients, and wherein processing restrictions for each test are enforced on each of the clients;
delivering data requested by each of the multiple clients when each of the multiple clients processes the tests in accordance with their execution plans;
monitoring the tests to gather statistics and results from the multiple clients, wherein at least some of the statistics include details from each test on each client for: memory usage, processing load, bandwidth utilization, page faults, time of execution, resources consulted, and ports used; and
reporting the results and the statistics to one or more third party services.
9. The method of claim 8 , wherein receiving further includes recognizing the execution plan as a series of tests that the multiple clients are to perform and delivered to the multiple clients by a master server.
10. The method of claim 8 , wherein enforcing further includes gathering information for connections and actions taken by the multiple clients when the multiple clients execute the tests.
11. The method of claim 8 , wherein delivering data further includes acquiring the data before the multiple clients begin processing the tests, and wherein the data is acquired from a master server that is initiating the tests.
12. The method of claim 8 , wherein monitoring further includes receiving reporting information for the statistics and the results from the tests that execute on the multiple clients.
13. The method of claim 8 , wherein monitoring further includes tracking connections that the multiple clients have to an application server that the multiple clients are supplying the results to and gathering the statistics in response to tracking those connections.
14. The method of claim 8 , wherein reporting further includes providing the results of the tests to a master server that initiated the tests with the multiple clients and providing the statistics to a stator service that summarizes the statistics and provides reports and sampling of the statistics to the master server.
15. A machine-implemented system adapted to be executed on one or more processors, comprising:
a master server implemented in a non-transitory computer-readable storage medium and that processes on a first server; and
a proxy implemented in a non-transitory computer-readable storage medium and that processes a second server;
wherein the master server identifies clients to perform tests and communicates the tests and policies for running the tests to the clients and the proxy, at least one of the clients is a virtual machine, wherein the proxy enforces the policies during the tests and monitors the clients during the tests to gather statistics and results that are then reported back to the master server, and wherein at least some of the statistics include details from each test on each client for: memory usage, processing load, bandwidth utilization, page faults, time of execution, resources consulted, and ports used; and the proxy enforces processing restrictions for each test on each client.
16. The system of claim 15 , wherein the master server includes a user interface that permits a user to define characteristics for desired clients that the master server uses to locate the clients for the tests.
17. The system of claim 15 , wherein the master server organizes the tests as an execution plan that is saved and capable of being replayed on demand with one or more of the clients.
18. The system of claim 15 , wherein the master server provides authentication tokens with the tests that the clients can use to authenticate the tests for execution on the clients.
19. The system of claim 15 , wherein the proxy monitors connections that the clients make to an application server that the clients communicate with in response to the tests to acquire the statistics and the results.
20. The system of claim 15 , wherein the proxy supplies the statistics to a third-party stator service that compiles and aggregates the statistics and reports a sampling of the statistics back to the master server.
21. A machine-implemented system adapted to be executed by one or more processors, comprising:
an identity service implemented in a non-transitory computer-readable storage medium and processed by a processor of a network; and
a master server implemented in a non-transitory computer-readable storage medium and processed by another processor for the network;
wherein the identity service validates clients for running tests that are defined in execution plans by the master server, the clients acquire test data from a proxy and the proxy monitors the tests that the clients execute to gather statistics and results that are reported to a stator service, the stator service summarizes and samples the statistics and reports them back to the master server, at least one client is a virtual machine, and the proxy enforces processing restrictions on each client for each test, and at least some of the statistics include details from each test on each client for: memory usage, processing load, bandwidth utilization, page faults, time of execution, resources consulted, and ports used.
22. The system of claim 21 , wherein the master server uses the identity service to acquire authentication tokens that accompany the tests sent to the clients, the authentication tokens are verified by the clients via the identity service.
23. The system of claim 22 , wherein the authentication tokens include security roles that the clients are to enforce when executing the tests.
24. The system of claim 21 , wherein some of the clients interact with the proxy and other of the clients interact with another different proxy, the different proxy performing the same features for the clients that use the different proxy as those services that the proxy performs for the clients that it services.