Recording, replaying and modifying an unstructured information management architecture (UIMA) pipeline
The technique herein substantially improves productivity of Annotator developers by providing methods and systems to develop and test Annotators without having to run a full pipeline every time changes are made to a particular Annotator. To this end, preferably a running pipeline is instrumented to enable automated recording of static configuration and dynamically-generated event data as the pipeline is executed. Based on these data, a reusable data model is generated that captures code and other dependencies in the pipeline (e.g., configuration parameters, intermediary CASes, program flow, annotations, and the like). The data model is then used to facilitate testing of Annotators without using the full pipeline (or even major sub-pipelines therein).
1. A method of testing elements of a pipeline software system comprising a set of distinct elements organized end-to-end from an initial element to a final element, comprising:
as an end-to-end run of a pipeline represented by the pipeline software system executes, receiving information about a state of the pipeline software system, wherein the pipeline software system comprises an Unstructured Information Management Architecture (UIMA) pipeline and the information includes one or more intermediary Common Analysis Structure (CAS) datasets;
using the information received to generate a reusable data model representative of conditions existing along the set of distinct elements of the pipeline software system; and
in association with a development action on a particular element positioned along the pipeline between the initial and final elements, and based on the reusable data model, identifying the conditions upstream of the particular element, providing the identified conditions as input to the particular element, and replaying execution of the pipeline software system without requiring initialization and re-execution of the end-to-end run of the pipeline from the initial element to the final element;
wherein the method is carried out in software executing in a hardware element.
2. The method as described in claim 1 wherein the information also includes event data generated by at least one UIMA Annotator in the UIMA pipeline.
3. The method as described in claim 2 further including instrumenting the UIMA Annotator to generate the event data.
4. The method as described in claim 1 wherein the development action is one of: executing or debugging an element in a standalone manner, running a test to determine how deletion of an element affects the pipeline, running a test to determine how adding an element affects the pipeline, running a test to determine how a conditional execution of an element affects the pipeline, and building a code dependency tree.
5. The method as described in claim 1 further including receiving additional information about the state of the pipeline software system from one or more additional runs of the pipeline software system.
6. An apparatus to test elements of a pipeline software system comprising a set of distinct elements organized end-to-end from an initial element to a final element, comprising:
a processor; and
computer memory holding computer program instructions executed by the processor, the computer program instructions comprising:
code, configured as an end-to-end run of a pipeline represented by the pipeline software system executes, to receive information about a state of the pipeline software system, wherein the pipeline software system comprises an Unstructured Information Management Architecture (UIMA) pipeline and the information includes one or more intermediary Common Analysis Structure (CAS) datasets;
code, configured to use the information received to generate a reusable data model representative of conditions existing along the set of distinct elements of the pipeline software system; and
code, configured in association with a development action on a particular element positioned along the pipeline between the initial and final elements, and based on the reusable data model, to identify the conditions upstream of the particular element, to provide the identified conditions as input to the particular element, and to replay execution of the pipeline software system without requiring initialization and re-execution of the end-to-end run of the pipeline from the initial element to the final element.
7. The apparatus as described in claim 6 wherein the information also includes event data generated by at least one UIMA Annotator in the UIMA pipeline.
8. The apparatus as described in claim 7 further including code to instrument the UIMA Annotator to generate the event data.
9. The apparatus as described in claim 6 further including a user interface through the development action is performed, the development action being one of: executing or debugging an element in a standalone manner, running a test to determine how deletion of an element affects the pipeline, running a test to determine how adding an element affects the pipeline, running a test to determine how a conditional execution of an element affects the pipeline, and building a code dependency tree.
10. The apparatus as described in claim 6 wherein the code configured to receive the information receives additional information about the state of the pipeline software system from one or more additional runs of the pipeline software system.
11. A computer program product in a non-transitory computer readable storage medium for use in a computing entity, the computer program product holding computer program instructions which, when executed, test elements of a pipeline software system comprising a set of distinct elements organized end-to-end from an initial element to a final element, the computer program instructions comprising:
code, configured as an end-to-end run of a pipeline represented by the pipeline software system executes, to receive information about a state of the pipeline software system, wherein the pipeline software system comprises an Unstructured Information Management Architecture (UIMA) pipeline and the information includes one or more intermediary Common Analysis Structure (CAS) datasets;
code, configured to use the information received to generate a reusable data model representative of conditions existing along the set of distinct elements of the pipeline software system; and
code, configured in association with a development action on a particular element positioned along the pipeline between the initial and final elements, and based on the reusable data model, to identify the conditions upstream of the particular element, to provide the identified conditions as input to the particular element, and to replay execution of the pipeline software system without requiring initialization and re-execution of the end-to-end run of the pipeline from the initial element to the final element.
12. The computer program product as described in claim 11 wherein the information also includes event data generated by at least one UIMA Annotator in the UIMA pipeline.
13. The computer program product as described in claim 11 further including code to instrument the UIMA Annotator to generate the event data.
14. The computer program product as described in claim 11 further including a user interface through the development action is performed, the development action being one of: executing or debugging an element in a standalone manner, running a test to determine how deletion of an element affects the pipeline, running a test to determine how adding an element affects the pipeline, running a test to determine how a conditional execution of an element affects the pipeline, and building a code dependency tree.
15. The apparatus as described in claim 11 wherein the code configured to receive the information receives additional information about the state of the pipeline software system from one or more additional runs of the pipeline software system.