Automated generation of load tests for API testing
Disclosed herein are system, method, and computer-readable medium embodiments for providing the ability to automate the process of generating load tests used for benchmarking APIs. Rather than having to manually generate load tests for a web service API, a test developer can interact with a test service through a web browser and provide the service an API specification and testing parameters. The test service can analyze the API specification, automatically identified endpoints of the API, and generate load tests according to the expected input/output structures of the endpoints. The automatic load test generation can proceed by referring to a library of test instructions and extracting the portions of the test instructions that work for the identified endpoints.
1. A computer-implemented method comprising:
receiving, by one or more computing devices, a specification of an application programming interface (API) and parameters for one or more load tests, wherein:
the parameters comprise a number of parallel requests to the API and a duration for the one or more load tests, and
the one or more computing devices comprise a library comprising test instructions;
analyzing, by the one or more computing devices, the specification of the API and the parameters;
generating, by the one or more computing devices, a load test by importing at least a portion of the test instructions from the library based on the analyzing; and
executing, by the one or more computing devices, the load test on an endpoint of the API for the duration.
2. The computer-implemented method of claim 1 , wherein the generating comprises generating the load test based on factors determined by the analyzing, the factors comprising:
one or more endpoints of the API;
transformations applied to the one or more endpoints;
potential inputs and outputs at the one or more endpoints; and/or
potential transport responses at the one or more endpoints.
3. The computer-implemented method of claim 1 , further comprising generating, by the one or more computing devices, a report based on the results of the executed load test.
4. The computer-implemented method of claim 1 , further comprising publishing, by the one or more computing devices, the generated load test on an API monitoring service.
5. The computer-implemented method of claim 1 , further comprising scheduling, by an API monitoring scheduler, the load test based on a time schedule.
6. The computer-implemented method of claim 5 , further comprising executing the scheduled load test based on the time schedule.
7. The computer-implemented method of claim 6 , further comprising logging information relating to the executing the scheduled load test.
8. The computer-implemented method of claim 6 , further comprising sending an alert indicating a completion of the executing the scheduled load test.
9. A non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:
receiving a specification of an application programming interface (API) and parameters for one or more load tests, wherein:
the parameters comprise a number of parallel requests to the API and a duration for the one or more load tests, and
the one or more computing devices comprise a library comprising test instructions;
analyzing the specification of the API and the parameters;
generating a load test by importing at least a portion of the test instructions from the library based on the analyzing; and
executing the load test on an endpoint of the API for the duration.
10. The non-transitory computer-readable medium of claim 9 , wherein the generating comprises generating the load test based on factors determined by the analyzing, the factors comprising:
one or more endpoints of the API;
transformations applied to the one or more endpoints;
potential inputs and outputs at the one or more endpoints; and/or
potential transport responses at the one or more endpoints.
11. The non-transitory computer-readable medium of claim 9 , wherein the operations further comprise generating a report based on the results of the executed load test.
12. The non-transitory computer-readable medium of claim 9 , wherein the operations further comprise publishing the generated load test on an API monitoring service.
13. The non-transitory computer-readable medium of claim 9 , wherein the operations further comprise scheduling, by an API monitoring scheduler, the load test based on a time schedule.
14. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise executing the scheduled load test based on the time schedule.
15. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise logging information relating to the executing the scheduled load test.
16. The non-transitory computer-readable medium of claim 13 , wherein the operations further comprise sending an alert indicating a completion of the executing the scheduled load test.
17. A system comprising:
one or more computing devices; and
a non-transitory computer-readable medium having instructions stored thereon that, when executed by the one or more computing devices, cause the one or more computing devices to perform operations comprising:
receiving a specification of an application programming interface (API) and parameters for one or more load tests, wherein:
the parameters comprise a number of parallel requests to the API and a duration for the one or more load tests, and
the one or more computing devices comprise a library comprising test instructions;
analyzing the specification of the API and the parameters;
generating a load test by importing at least a portion of the test instructions from the library based on the analyzing; and
executing the load test on an endpoint of the API for the duration.
18. The system of claim 17 , wherein the generating comprises generating the load test based on factors determined by the analyzing, the factors comprising:
one or more endpoints of the API;
transformations applied to the one or more endpoints;
potential inputs and outputs at the one or more endpoints; and/or
potential transport responses at the one or more endpoints.
19. The system of claim 17 , wherein the operations further comprise a report based on the results of the executed load test.
20. The system of claim 17 , wherein the operations further comprise scheduling, by an API monitoring scheduler, the load test based on a time schedule.