Automatic non-code test suite generation from API specification
Disclosed herein are system, method, and computer program product embodiments for automatic non-code test suite generation of an application programming language (API) specification. An embodiment operates by receiving a specification of an API, wherein the API comprises a plurality of endpoints. The embodiment generates, using a parser, an abstraction model corresponding to the specification of the API, wherein the abstraction model comprises a plurality of entities corresponding to the plurality of endpoints. The embodiment identifies, based on the abstraction model, an operation that is applicable to an entity of the plurality of entities. The embodiment then generates a functional test based on a use case corresponding to the entity and the operation.
1. A method, comprising:
generating, by one or more computing devices, using a parser and a modeling language, a domain-specific abstraction model corresponding to a specification of an application programming interface (API), the domain-specific abstraction model defining one or more entities each corresponding to an endpoint of the API and each comprising a path and a set of operations available at the endpoint;
identifying, by the one or more computing devices, based on the domain-specific abstraction model, use cases for each endpoint and each operation in the set of operations at each endpoint, each use case comprising an operation in the set of operations available at the endpoint, a request format, a required parameter, an optional parameter, one or more expected response headers, and an expected response value; and
in response to a build in a development pipeline, the build including an update made to the API by a developer in the development pipeline, generating, by the one or more computing devices, a functional test that for each of the use cases: transmits the operation to the endpoint via the path in the request format with the required parameter and verifies that a received response code matches the expected response value and that one or more headers in a response payload from the API match the one or more expected headers.
2. The method of claim 1 , wherein the generating the domain-specific abstraction model is based on an open API framework.
3. The method of claim 1 , wherein the specification is defined in a RESTful API Modeling Language (RAML) file, an Open API Specification (OAS), a GraphQL, or a gRPC Remote Procedure Call format.
4. The method of claim 1 , wherein the operation is one of a fetch, a create, a list, a delete, or an update.
5. The method of claim 1 , wherein the operation performs a happy path test.
6. The method of claim 1 , wherein the use cases test one or more security attributes.
7. The method of claim 1 , wherein the set of operations available at the endpoint comprises a get, a put, a delete, and a post.
8. A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
generate, using a parser and a modeling language, a domain-specific abstraction model corresponding to a specification of an application programming interface (API), the domain-specific abstraction model defining one or more entities each corresponding to an endpoint of the API and each comprising a path and a set of operations available at the endpoint;
identify, based on the domain-specific abstraction model, use cases for each endpoint and each operation in the set of operations at each endpoint, each use case comprising an operation in the set of operations available at the endpoint, a request format, a required parameter, an optional parameter, one or more expected response headers, and an expected response value; and
in response to a build in a development pipeline, the build including an update made to the API by a developer in the development pipeline, generate a functional test that for each use case of the use cases: transmits the operation to the endpoint via the path in the request format with the required parameter and verifies that a received response code matches the expected response value and that one or more headers in a response payload from the API match the one or more expected headers.
9. The system of claim 8 , wherein the generating the domain-specific abstraction model is based on an open API framework.
10. The system of claim 8 , wherein the specification is defined in a RESTful API Modeling Language (RAML) file, an Open API Specification (OAS), a GraphQL, or a gRPC Remote Procedure Call format.
11. The system of claim 8 , wherein the operation is one of a fetch, a create, a list, a delete, or an update.
12. The system of claim 8 , wherein the operation performs a happy path test.
13. The system of claim 8 , wherein the use cases test one or more security attributes.
14. The system of claim 8 , wherein the set of operations available at the endpoint comprises a get, a put, a delete, and a post.
15. A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:
generating, using a parser and a modeling language, a domain-specific abstraction model corresponding to a specification of an application programming interface (API), the domain-specific abstraction model defining one or more entities each corresponding to an endpoint of the API and each comprising a path and a set of operations available at the endpoint;
identifying based on the domain-specific abstraction model, use cases for each endpoint and each operation in the set of operations at each endpoint, each use case comprising an operation in the set of operations available at the endpoint, a request format, a required parameter, an optional parameter, one or more expected response headers, and an expected response value; and
in response to a build in a development pipeline, the build including an update made to the API by a developer in the development pipeline, generating a functional test that for each of the use cases: transmits the operation to the endpoint in the request format with the required parameter and verifies that a received response code matches the expected response value and that one or more headers in a response payload from the API by match the one or more expected headers.
16. The non-transitory computer-readable device of claim 15 , wherein the generating the domain-specific abstraction model is based on an open API framework.
17. The non-transitory computer-readable device of claim 15 , wherein the specification is defined in a RESTful API Modeling Language (RAML) file, an Open API Specification (OAS), a GraphQL, or a gRPC Remote Procedure Call format.
18. The non-transitory computer-readable device of claim 15 , wherein the operation is one of a fetch, a create, a list, a delete, or an update.
19. The non-transitory computer-readable device of claim 15 , wherein the operation performs a happy path test.
20. The non-transitory computer-readable device of claim 15 , wherein the set of operations available at the endpoint comprises a get, a put, a delete, and a post.