Large language model-based test-case refinement for application programming interfaces

1 . A method of using a computing device associated with a large language model to improve testing of an application programming interface in order to minimize errors associated with the application programming interface, the method comprising:

accessing by a computing device an application programming interface specification associated with an application programming interface, the application programming interface specification an electronically available document which describes at least in-part in natural language how to build and utilize functionality associated with the application programming interface specification, the application programming interface specification containing natural language descriptions, source code, and test cases to be used for testing of the application programming interface;

accessing by the computing device from the application programming interface specification one or more test cases previously generated for testing the application programming interface which when utilized to test the application programming interface resulted in one or more errors;

accessing by the computing device the one or more errors generated when testing the application programming interface using the one or more test cases previously generated;

generating automatically by the computing device a scenario specific prompt for input into a large language model, the scenario specific prompt to provide instructions for the large language model to correctly generate test cases and update the application programming interface specification;

inputting automatically by the computing device via the scenario specific prompt the application programming interface specification, the one or more test cases previously generated, and the one or more errors into the large language model for refinement;

receiving a modified set of the one or more test cases, modified by the large language model, the modified set of the one or more test cases generated by the large language model in response to the automatically generated prompt to improve testing of the application programming interface; and

receiving from the large language model a modified application programming interface specification, modified by the large language model to improve testing of the application programming interface.

2 . The method of claim 1 , wherein the modified set of the one or more test cases are automatically utilized for testing of the application programming interface by an interactive development environment.

3 . The method of claim 1 , wherein the one or more errors are HTTP status codes.

4 . The method of claim 1 , wherein the application programming interface specification is in a format associated with OpenAPI.

5 . The method of claim 1 , wherein the application programming interface specification is incomplete or incorrect.

6 . The method of claim 1 , wherein the one or more test cases previously generated for testing of the application programming interface exist in a JSON file format which describes one or more parameters, one or more data values for each parameter, and a sequence of operations.

7 . The method of claim 1 , wherein the modified set of the one or more test cases are output in a JSON file format.

8 . The method of claim 1 , wherein the modified set of the one or more test cases and the modified application programming interface specification are utilized by a debugger in further testing of application programming interface.

9 . A computer system associated with a large language model to improve testing of an application programming interface in order to minimize errors associated with the application programming interface, the computer system comprising:

one or more computer processors;

one or more computer-readable storage media;

program instructions stored on the computer-readable storage media for execution by at least one of the one or more processors, the program instructions comprising:

program instructions to access an application programming interface specification associated with an application programming interface, the application programming interface specification an electronically available document which describes at least in-part in natural language how to build and utilize functionality associated with the application programming interface specification, the application programming interface specification containing natural language descriptions, source code, and test cases to be used for testing of the application programming interface;

program instructions to access from the application programming interface specification one or more test cases previously generated for testing the application programming interface which when utilized to test the application programming interface resulted in one or more errors;

program instructions to access the one or more errors generated when testing the application programming interface using the one or more test cases previously generated;

program instructions to generate automatically a scenario specific prompt for input into a large language model, the scenario specific prompt to provide instructions for the large language model to correctly generate test cases and update the application programming interface specification;

program instructions to input automatically by the computing device via the scenario specific prompt the application programming interface specification, the one or more test cases previously generated, and the one or more errors into the large language model for refinement;

program instructions to receive a modified set of the one or more test cases, modified by the large language model, the modified set of the one or more test cases generated by the large language model in response to the automatically generated prompt to improve testing of the application programming interface; and

program instructions to receive from the large language model a modified application programming interface specification, modified by the large language model to improve testing of the application programming interface.

10 . The computer system of claim 9 , wherein the modified set of the one or more test cases are output and automatically utilized for testing of the application programming interface by an interactive development environment.

11 . The computer system of claim 9 , wherein the one or more errors are HTTP status codes.

12 . The computer system of claim 9 , wherein the application programming interface specification is in a format associated with OpenAPI.

13 . The computer system of claim 9 , wherein the one or more test cases previously generated for testing of the application programming interface exist in a JSON file format which describes one or more parameters, one or more data values for each parameter, and a sequence of operations.

14 . A computer program product associated with a large language model to improve testing of an application programming interface in order to minimize errors associated with the application programming interface, the computer program product comprising:

one or more non-transitory computer-readable storage media and program instructions stored on the one or more non-transitory computer-readable storage media capable of performing a method, the method comprising:

accessing by a computing device an application programming interface specification associated with an application programming interface, the application programming interface specification an electronically available document which describes at least in-part in natural language how to build and utilize functionality associated with the application programming interface specification, the application programming interface specification containing natural language descriptions, source code, and test cases to be used for testing of the application programming interface;

accessing by the computing device from the application programming interface specification one or more test cases previously generated for testing the application programming interface which when utilized to test the application programming interface resulted in one or more errors;

accessing by the computing device the one or more errors generated when testing the application programming interface using the one or more test cases previously generated;

generating automatically by the computing device a scenario specific prompt for input into a large language model, the scenario specific prompt to provide instructions for the large language model to correctly generate test cases and update the application programming interface specification;

inputting automatically by the computing device via the scenario specific prompt the application programming interface specification, the one or more test cases previously generated, and the one or more errors into the large language model;

receiving a modified set of the one or more test cases, modified by the large language model, the modified set of the one or more test cases generated by the large language model in response to the automatically generated prompt to improve testing of the application programming interface; and

receiving from the large language model a modified application programming interface specification, modified by the large language model to improve testing of the application programming interface.

15 . The computer program product of claim 14 , wherein the one or more errors are HTTP status codes.

16 . The computer program product of claim 14 , wherein the application programming interface specification is in a format associated with OpenAPI.