Automatically running tests against WEB APIs based on specifications

1. A computing device comprising:

a processor;

a network interface coupled to the processor to enable communication over a network;

a storage device coupled to the processor;

a compliance engine software stored in the storage device, wherein an execution of the software by the processor configures the computing device to perform acts comprising:

receiving an electronic specification defining an application program interface (API), over the network;

identifying service calls to be tested based on the specification;

creating a test case for each of the identified service calls;

creating a sequence for the created test cases, for an efficient correlation between the specification and the API, wherein the sequence for the created test cases comprises:

automatically identifying one or more Hypertext Transfer Protocol (HTTP) invocations comprising at least one of: POST, GET, PUT, PATCH, or DELETE invocation;

automatically ordering any identified POST invocation before any GET, PUT, PATCH, and DELETE invocation; and

automatically ordering any DELETE invocation after any POST, GET, PUT, and PATCH invocation;

generating a test plan based on the created sequence;

executing the generated test plan; and

upon identifying an error in response to the executed test plan, generating a notification indicating that the specification is not an accurate representation of the API.

2. The computing device of claim 1 , wherein identifying service calls to be tested comprises parsing a text of the specification to identify Web service calls and one or more input parameters associated with each Web service call.

3. The computing device of claim 1 , wherein creating a test case for each of the identified service calls comprises identifying one or more threshold criteria for an error.

4. The computing device of claim 3 , wherein the one or more threshold criteria comprises at least one of: (i) receiving a Hypertext Transfer Protocol (HTTP) error code over the network, or (ii) a discrepancy between an expected data and a received data, in response to the execution of the test case.

5. The computing device of claim 1 , wherein generating the test plan comprises creating an executable source code.

6. The computing device of claim 1 , wherein generating the test plan comprises translating the sequence of the created test cases in a format that a Web service testing application understands.

7. The computing device of claim 1 , wherein executing the generated test plan comprises, for each test case:

sending a Hypertext Transfer Protocol (HTTP) requests to the API;

receiving a response to the HTTP request; and

storing the response in the storage device.

8. The computing device of claim 1 , wherein execution of the compliance engine by the processor further configures the computing device to perform an act comprising: upon identifying no errors in response to the executed test plan, certifying that the specification is an accurate representation of the API.

9. The computing device of claim 1 , wherein execution of the compliance engine by the processor further configures the computing device to perform acts comprising, upon identifying an error in response to the executed test plan:

identifying a portion of the specification related to the error; and

indicating the portion of the specification in the generated notification.

10. The computing device of claim 9 , wherein execution of the compliance engine by the processor further configures the computing device to perform acts comprising, upon identifying an error in response to the executed test plan, sending the notification to a developer of the API.

11. A non-transitory computer readable storage medium tangibly embodying a computer readable program code having computer readable instructions that, when executed, causes a computer device to carry out a method of determining whether a specification is an accurate representation of an application program interface (API), the method comprising:

receiving the specification electronically, over a network;

identifying service calls to be tested based on the specification;

creating a test case for each of the identified service calls;

creating a sequence for the created test cases, for an efficient correlation between the specification and the API, wherein the sequence for the created test cases comprises:

automatically identifying one or more Hypertext Transfer Protocol (HTTP) invocations comprising at least one of: POST, GET, PUT, PATCH, or DELETE invocation;

automatically ordering any identified POST invocation before any GET, PUT, PATCH, and DELETE invocation; and

automatically ordering any DELETE invocation after any POST, GET, PUT, and PATCH invocation;

generating a test plan based on the created sequence;

executing the generated test plan; and

upon identifying an error in response to the executed test plan, generating a notification indicating that the specification is not an accurate representation of the API.

12. The non-transitory computer readable storage medium of claim 11 , wherein identifying service calls to be tested comprises parsing a text of the specification to identify Web service calls and one or more input parameters associated with each Web service call.

13. The non-transitory computer readable storage medium of claim 11 , wherein creating a test case for each of the identified service calls comprises identifying one or more threshold criteria for an error.

14. The non-transitory computer readable storage medium of claim 13 , wherein the one or more threshold criteria comprises at least one of: (i) receiving a Hypertext Transfer Protocol (HTTP) error code over the network, or (ii) a discrepancy between an expected data and a received data, in response to the execution of the test case.

15. The non-transitory computer readable storage medium of claim 11 , wherein generating the test plan comprises creating an executable source code.

16. The non-transitory computer readable storage medium of claim 11 , wherein executing the generated test plan comprises, for each test case:

sending a Hypertext Transfer Protocol (HTTP) requests to the API;

receiving a response to the HTTP request; and

storing the response in the storage device.

17. The non-transitory computer readable storage medium of claim 11 , further comprising, upon identifying no errors in response to the executed test plan, certifying that the specification is an accurate representation of the API.

18. The non-transitory computer readable storage medium of claim 11 , further comprising, upon identifying an error in response to the executed test plan:

identifying a portion of the specification related to the error; and

indicating the portion of the specification in the generated notification.