Regression testing of computer systems using recorded prior computer system communications
A technique includes accessing, by at least one hardware processor, a recorded request and a recorded response associated with an integration test involving a first computer system and a second computer system. The recorded request was previously issued by the first computer system to the second computer system to cause the second computer system to provide the recorded response. The technique includes, in a virtualized integration test involving the second computer system and initiated using the recorded request, comparing, by the hardware processor(s), the recorded response to a request produced by the second computer system in the virtualized integration test. The technique includes identifying, by the hardware processor(s), an action taken by the second computer system as being likely to be associated with a regression based on the comparison.
1. A method comprising:
causing a test application to be executed for an actual integration test involving a server computer system and a client computer system, wherein the execution of the test application generates an external application programming interface (API) call from the client computer system and an external API response from the server computer system;
recording, by a processor of the server computer system, integration test data of the actual integration test, including recording the external API call and the external API response;
executing, by the processor, a virtualized integration test of the server computer system without involving any client computer system, including: executing the recorded external API call associated with the actual integration test on the test application, and generating a mocked external API response in response to the execution of the recorded external API call;
determining, by the processor, whether the recorded external API response associated with the actual integration test matches the mocked external API response associated with the virtualized integration test; and
in response to a determination that the recorded external API response does not match the mocked external API response, identifying, by the processor, an action to be taken by the server computer system as being likely to be associated with a regression.
2. The method of claim 1 , wherein:
the server computer system provides a service consumed by the client computer system; and
identifying the action comprises generating a notification that identifies a response provided by the service as likely to be associated with the regression.
3. The method of claim 1 , the method further comprising:
in response to the external API call from the client computer system, generating an internal API call for accessing data of the server computer system and an internal API response containing the data of the server computer system.
4. The method of claim 3 , further comprising:
creating a mapping from the internal API call to the internal API response.
5. The method of claim 1 , further comprising:
creating a mapping from the recorded external API call to the recorded external API response.
6. The method of claim 1 , further comprising:
generating a comparison result representing whether the recorded external API response matches the mocked external API response.
7. The method of claim 1 , wherein identifying the action as likely being associated with the regression comprises generating a notification of the action likely being associated with the regression.
8. The method of claim 1 , further comprising:
in response to the external API call from the cheat computer system, generating an internal request for internal data in the server computer system and an internal response with the internal data.
9. The method of claim 8 , wherein recording the integration test data of the actual integration test further includes: recording the internal request and the internal response.
10. The method of claim 1 , further comprising:
generating a report that identifies the mocked external API response that does not match the recorded external API response.
11. A non-transitory machine readable storage medium storing instructions that, when executed by a machine, cause the machine to:
cause a test application to be executed for an actual integration test involving a server computer system and a client computer system, Wherein the execution of the test application generates an external application programming interface (API) call from the client computer system and an external API response from the server computer system;
record integration test data of the actual integration test, including the external API call and the external API response;
execute a virtualized integration test of the server computer system without involving any client computer system, including: executing the recorded external API call associated with the actual integration test, and generating a mocked external API response in response to the execution of the recorded external API call;
determine whether the recorded external API response associated with the actual integration test matches the mocked external API response associated with the virtualized integration test; and
in response to a determination that the recorded external API response does not match the mocked external API response, identify an action to be taken by the server computer system as being likely to be associated with a regression.
12. The non-transitory machine readable storage medium of claim 11 , wherein the instructions, when executed, cause the machine to:
in response to the external API call from the client computer system, generate an internal request for internal data within the server computer system and an internal response with the internal data; and
record the internal request and the internal response as part of the integration test data of the actual integration test.
13. The non-transitory machine readable storage medium of claim 11 , wherein the instructions, when executed by the machine, further cause the machine to:
create a mapping from the recorded external API call to the recorded external API response.
14. The non-transitory machine readable storage medium of claim 11 , wherein the instructions, when executed by the machine, further cause the machine to:
use a given action performed by the client computer system as an index to a recorded internal communication, wherein the mocked external API response comprises an API call of the recorded internal communication.
15. An apparatus comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to:
cause a test application to be executed for an actual integration test involving a server computer system and a client computer system, wherein the execution of the test application generates an external application programming interface (API) call from the client computer system and an external API response from the server computer system;
record integration test data of the actual integration test, including the external API call and the external API response;
execute a virtualized integration test of the server computer system without involving any client computer system, including: executing the recorded external API call associated with the actual integration test, and generating a mocked external API response in response to the execution of the recorded external API call;
determine whether the recorded external API response associated with the actual integration test matches the mocked external API response associated with the virtualized integration test; and
in response to a determination that the recorded external API response does not match the mocked external API response, identify an action taken by the server computer system as being likely to be associated with a regression.
16. The apparatus of claim 15 , wherein the instructions, when executed, further cause the at least one processor to:
create a mapping from the recorded external API call to the recorded external API response.
17. The apparatus of claim 15 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to:
provide another API call to initiate another virtualized integration test of the server computer system; and
mock a response of the server computer system to said another API call based on an API response recorded during said another virtualized integration test of the server computer system.
18. The apparatus of claim 15 , wherein:
the server computer system provides a service to the client computer system;
the virtualized integration test comprises a test conducted without the server computer system being connected to the client computer system; and
the actual integration test of the server computer system conducted prior to the virtualized integration test was performed with the client computer system being connected to the server computer system.
19. The apparatus of claim 15 , wherein, to determine whether the recorded external API response matches the mocked external API response, the instructions, when executed by the at least one processor, cause the at least one processor to compare the mocked external API response to the recorded external API response to determine whether the mocked external API response is an exact match to the recorded external API response.
20. The apparatus of claim 15 , wherein the instructions, when executed by the at least one processor, cause the at least one processor to:
in response to the external API call from the client computer system, generate an internal request for internal data in the server computer system and an internal response with the internal data; and
record the internal request and the internal response as part of the integration test data of the actual integration test.