Generic test framework for service interfaces
A generic framework for testing service interfaces where attributes for service access are abstracted from test execution attributes. An eXtensible Markup Language (XML) Service Attribute (SA) file may be used to define a server tag and one or more services tags. The server tag specifies protocol, address, and other access information for a host machine that provides the service. A service type and configuration parameter defaults may also be included with the server tag. Each service tag defines a name of a corresponding service, and a request-response configuration for the corresponding service. With this information in hand, a test executor can then issue a request message to the corresponding service indicated in the service tag, using the protocol and address information and request configuration, and match the response received against the response configuration. A separate Test Execution (TE) file may specify attributes of the test including hardware parameters, sequential/parallel execution, or the like.
1. A method for providing a generic framework for executing a test of an interface to a service, wherein service access (SA) attributes are stored in a service access (SA) interface configuration file, and from test execution (TE) attributes stored in a separate TE configuration file such that the SA attributes are independently stored from the TE attributes, the method comprising:
interpreting the service access (SA) interface configuration file having multiple tags including at least one server tag and one or more service tag for further:
determining, from the at least one service tag:
protocol and address information for a server that provides the interface to the service;
a service type;
configuration parameter defaults; and
determining, from each service tag:
a name of a corresponding service;
a request configuration for issuing a command to the corresponding service;
a response configuration for an expected response to the command to the corresponding service;
wherein the request configuration and response configuration are arranged in pairs and associated with the name of a corresponding service in each service tag;
interpreting the test execution (TE) configuration file to determine execution attributes of running the test of the service interface, wherein the TE configuration file includes one or more test execution attributes selected from the group consisting of
running the test asynchronously or synchronously,
number of iterations of the test, or
running the test multiple times sequentially or in parallel; and
executing the test against the corresponding service by:
issuing a request message to the corresponding service interface indicated in at least one service tag, using the name of the corresponding service, the protocol and address information and the request configuration; and
receiving a response message from the service;
comparing the response message to the corresponding expected response in the response configuration; and
further controlling the step of executing the test according to the attributes in the TE configuration file.
2. The method of claim 1 wherein the address information in the SA file includes a uniform resource identifier and port number.
3. The method of claim 1 wherein the at least one service tag additionally indicates a message format used to access the server.
4. The method of claim 1 wherein the at least one service tag additionally indicates parameters to be used in place of parameter defaults for either the request or response configuration.
5. The method of claim 1 wherein the response configuration additionally specifies a status code indicating a server condition.
6. The method of claim 5 wherein the at least one service tag additionally indicates a condition or a dependency, and the step of comparing the response message to the response configuration depends on the condition or dependency.
7. The method of claim 6 wherein the condition or dependency is defined by data or state information shared between two services.
8. The method of claim 1 wherein the test configuration file further specifies aspects of executing the test including one or more of:
server hardware parameters, or
logging output from the tests.
9. The method of claim 1 wherein the service is a cloud service.
10. The method of claim 1 wherein the step of running the test executes as a cloud service.
11. The method of claim 1 wherein the at least one service tag additionally comprises a depends-on parameter that specifies another service from which an output must be received before executing the test of the corresponding service.
12. The method of claim 1 wherein the at least one service tag additionally comprises a depends-on parameter that specifies a token exchange with another service before executing the test of the corresponding service.
13. The method of claim 1 wherein the at least one service tag additionally comprises an execute tag that controls whether the corresponding service is tested on a given iteration of the test.