Test case generator built into data-integration workflow editor
Embodiments presented herein provide techniques for generating test cases for transformation modules that are part of a larger data-integration workflow for an extract, transfer, and load (ETL) project. A test case generated according to the present disclosure can be executed independently of the application that generated it and independently of data sources and destinations referenced in the data-integration workflow. To achieve this independence, the test case can include code that simulates data sources and destinations that interact with the transformation module in the data-integration workflow. Furthermore, the test case can be a unit test based on a unit testing framework and can be compatible with a selected software framework.
1. A method for generating a test case for a transformation module of a data-integration workflow, the method comprising:
receiving a request to create the test case for the transformation module of the data-integration workflow that is compatible with a software framework;
identifying a first network path from an input data source to the transformation module in the data-integration workflow;
identifying a second network path from the transformation module to an output data source in the data-integration workflow;
generating code that defines the test case for the transformation module;
executing the test case in the software framework by:
sending input data from a simulated input data source of the test case to the transformation module without accessing the input data source; and
receiving output data at a simulated output data source of the test case from the transformation module without accessing the output data source;
determining the transformation module has been changed to generate a changed module; and
re-executing the test case in the software framework, wherein re-execution of the test case verifies functionality of the changed module.
2. The method of claim 1 , further comprising:
displaying a graphical model of the data-integration workflow in a graphical user interface (GUI), wherein at least one icon of the graphical model represents the transformation module; and
receiving the request via the graphical user interface, wherein selection of the at least one icon identifies the transformation module.
3. The method of claim 1 , further comprising:
receiving test input data for the test case; and
receiving target output data for the test case, wherein executing the test case comprises:
executing the transformation module using the test input data,
receiving the actual output data from the transformation module, and
comparing the actual output data to the target output data.
4. The method of claim 3 , further comprising:
determining, based on the comparison of the actual output data to the target output data, whether the transformation module passed or failed a test defined by the test case, the test input data, and the target output data; and
storing an indication of the determination.
5. The method of claim 3 , wherein executing the test case includes simulating the input data source to provide the test input data to the transformation module.
6. The method of claim 1 , wherein the test case is associated with a version of the data-integration workflow.
7. A non-transitory computer-readable storage medium containing instructions that, when executed by one or more processors, perform an operation for generating a test case for a transformation module of a data-integration workflow, the operation comprising:
receiving a request to create the test case for the transformation module of the data-integration workflow that is compatible with a software framework;
identifying a first network path from an input data source to the transformation module in the data-integration workflow;
identifying a second network path from the transformation module to an output data source in the data-integration workflow;
generating code that defines the test case for the transformation module;
executing the test case in the software framework by:
sending input data from a simulated input data source of the test case to the transformation module without accessing the input data source; and
receiving output data at a simulated output data source of the test case from the transformation module without accessing the output data source;
determining the transformation module has been changed to generate a changed module; and
re-executing the test case in the software framework, wherein re-execution of the test case verifies functionality of the changed module.
8. The non-transitory computer-readable storage medium of claim 7 , wherein the operation further comprises:
displaying a graphical model of the data-integration workflow in a graphical user interface (GUI), wherein at least one icon of the graphical model represents the transformation module; and
receiving the request via the graphical user interface, wherein selection of the at least one icon identifies the transformation module.
9. The non-transitory computer-readable storage medium of claim 7 , wherein the operation further comprises:
receiving test input data for the test case; and
receiving target output data for the test case; and, wherein executing the test case comprises:
executing the transformation module using the test input data,
receiving the actual output data from the transformation module, and
comparing the actual output data to the target output data.
10. The non-transitory computer-readable storage medium of claim 9 , wherein the operation further comprises:
determining, based on the comparison of the actual output data to the target output data, whether the transformation module passed or failed a test defined by the test case, the test input data, and the target output data; and
storing an indication of the determination.
11. The non-transitory computer-readable storage medium of claim 9 , wherein executing the test case includes simulating the input data source to provide the test input data to the transformation module.
12. The non-transitory computer-readable storage medium of claim 7 , wherein the test case is associated with a version of the data-integration workflow.
13. A system for generating a test case for a transformation module of a data-integration workflow, comprising:
one or more processors; and
a memory storing one or more applications that, when executed on the one or more processors, perform an operation, the operation comprising:
receiving a request to create the test case for the transformation module of the data-integration workflow that is compatible with a software framework;
identifying a first network path from an input data source to the transformation module in the data-integration workflow;
identifying a second network path from the transformation module to an output data source in the data-integration workflow;
generating code that defines the test case for the transformation module;
executing the test case in the software framework by:
sending input data from a simulated input data source of the test case to the transformation module without accessing the input data source; and
receiving output data at a simulated output data source of the test case from the transformation module without accessing the output data source;
determining the transformation module has been changed to generate a changed module; and
re-executing the test case in the software framework, wherein re-execution of the test case verifies functionality of the changed module.
14. The system of claim 13 , wherein the operation further comprises:
displaying a graphical model of the data-integration workflow in a graphical user interface (GUI), wherein at least one icon of the graphical model represents the transformation module; and
receiving the request via the graphical user interface, wherein selection of the at least one icon identifies the transformation module.
15. The system of claim 13 , wherein the operation further comprises:
receiving test input data for the test case; and
receiving target output data for the test case, wherein executing the test case comprises:
executing the transformation module using the test input data,
receiving the actual output data from the transformation module, and
comparing the actual output data to the target output data.
16. The system of claim 15 , wherein the operation further comprises:
determining, based on the comparison of the actual output data to the target output data, whether the transformation module passed or failed a test defined by the test case, the test input data, and the target output data; and
storing an indication of the determination.
17. The system of claim 15 , wherein executing the test case includes simulating the input data source to provide the test input data to the transformation module.