Analytics workflow integrated with logic control
A workflow includes a sequence of tools and a logic controller. A data file is input into the workflow and processed by the tools. A log file and a metadata file are generated for a tool. The log file includes data describing events that occurred during the operation of the tool in the processing of the data file. The metadata file includes data generated from the performance of the data analytics function of the tool. The logic controller uses the log file to determine whether to trigger a subsequent tool. Responsive to a determination of trigging the subsequent tool, the metadata file is sent to the subsequent tool and processed by a subset of the sequence of tools to generate a result item. The subset starts with the subsequent tool. The result item represents an outcome of the processing of the data file by the workflow.
1. A computer-implemented method, comprising:
accessing an analytics workflow including a parsing tool, a filter tool, an error container including one or more error tools, and a no error container including a plurality of tools;
generating a log file and a metadata file by inputting a data file to the parsing tool, the log file including data describing events occurring during the parsing tool processing the data file using one or more data analytics functions, the metadata file including analytics data generated from the parsing tool processing the data file using the one or more data analytics functions and representing an outcome of the one or more data analytics functions;
determining, by inputting the log file to the filter tool, whether the log file includes at least one entry indicating an error occurred during the parsing tool processing the data file; and
responsive to determining that the log file includes at least one entry indicating the error:
generating a message that describes the error that occurred during the parsing tool processing the data file, wherein generating the message is performed without processing the metadata file using the plurality of tools of the no error container; and
displaying the message in a user interface; or
responsive to determining that the log file does not include at least one entry indicating the error:
generating additional analytics data by inputting the metadata file to at least one of the plurality of tools of the no error container, wherein generating the additional analytics data is performed without processing the log file using the one or more error tools of the error container; and
presenting the additional analytics data in the user interface.
2. The computer-implemented method of claim 1 , wherein the no error container is implemented by a processor of a computing device, wherein generating the message is performed without providing power to the processor implementing the no error container.
3. The computer-implemented method of claim 1 , wherein the error container is implemented by a processor of a computing device, wherein generating the additional analytics data is performed without providing power to the processor implementing the error container.
4. The computer-implemented method of claim 1 , wherein generating the additional analytics data comprises generating at least one of a chart or a graph that depicts analytics derived from processing the metadata file using the at least one of the plurality of tools of the no error container.
5. The computer-implemented method of claim 1 , wherein determining whether the log file includes at least one entry indicating the error is performed while the parsing tool is processing the data file, the computer-implemented method further comprising pausing processing of the data file by the parsing tool until the error is resolved.
6. The computer-implemented method of claim 5 , further comprising:
receiving input at the user interface displaying the message, the input indicating that the error is resolved; and
resuming processing of the data file by the parsing tool responsive to receiving the input.
7. The computer-implemented method of claim 1 , wherein generating the additional analytics data by inputting the metadata file to the at least one of the plurality of tools of the no error container is performed in parallel while the parsing tool is processing the data file.
8. A system comprising:
one or more processors; and
a computer-readable storage medium storing instructions that are executable by the one or more processors to perform operations comprising:
accessing an analytics workflow including a parsing tool, a filter tool, an error container including one or more error tools, and a no error container including a plurality of tools;
generating a log file and a metadata file by inputting a data file to the parsing tool, the log file including data describing events occurring during the parsing tool processing the data file using one or more data analytics functions, the metadata file including analytics data generated from the parsing tool processing the data file using the one or more data analytics functions and representing an outcome of the one or more data analytics functions;
determining, by inputting the log file to the filter tool, whether the log file includes at least one entry indicating an error occurred during the parsing tool processing the data file; and
responsive to determining that the log file includes at least one entry indicating the error:
generating a message that describes the error that occurred during the parsing tool processing the data file, wherein generating the message is performed without processing the metadata file using the plurality of tools of the no error container; and
displaying the message in a user interface; or
responsive to determining that the log file does not include at least one entry indicating the error:
generating additional analytics data by inputting the metadata file to at least one of the plurality of tools of the no error container, wherein generating the additional analytics data is performed without processing the log file using the one or more error tools of the error container; and
presenting the additional analytics data in the user interface.
9. The system of claim 8 , wherein the no error container is implemented by a processor of a computing device, wherein generating the message is performed without providing power to the processor implementing the no error container.
10. The system of claim 8 , wherein the error container is implemented by a processor of a computing device, wherein generating the additional analytics data is performed without providing power to the processor implementing the error container.
11. The system of claim 8 , wherein generating the additional analytics data comprises generating at least one of a chart or a graph that depicts analytics derived from processing the metadata file using the at least one of the plurality of tools of the no error container.
12. The system of claim 8 , wherein determining whether the log file includes at least one entry indicating the error is performed while the parsing tool is processing the data file, the operations further comprising pausing processing of the data file by the parsing tool until the error is resolved.
13. The system of claim 12 , the operations further comprising:
receiving input at the user interface displaying the message, the input indicating that the error is resolved; and
resuming processing of the data file by the parsing tool responsive to receiving the input.
14. The system of claim 8 , wherein generating the additional analytics data by inputting the metadata file to the at least one of the plurality of tools of the no error container is performed in parallel while the parsing tool is processing the data file.
15. A non-transitory computer-readable storage medium storing instructions that are executable by at least one processing device to perform operations comprising:
accessing an analytics workflow including a parsing tool, a filter tool, an error container including one or more error tools, and a no error container including a plurality of tools;
generating a log file and a metadata file by inputting a data file to the parsing tool, the log file including data describing events occurring during the parsing tool processing the data file using one or more data analytics functions, the metadata file including analytics data generated from the parsing tool processing the data file using the one or more data analytics functions and representing an outcome of the one or more data analytics functions;
determining, by inputting the log file to the filter tool, whether the log file includes at least one entry indicating an error occurred during the parsing tool processing the data file; and
responsive to determining that the log file includes at least one entry indicating the error:
generating a message that describes the error that occurred during the parsing tool processing the data file, wherein generating the message is performed without processing the metadata file using the plurality of tools of the no error container; and
displaying the message in a user interface; or
responsive to determining that the log file does not include at least one entry indicating the error:
generating additional analytics data by inputting the metadata file to at least one of the plurality of tools of the no error container, wherein generating the additional analytics data is performed without processing the log file using the one or more error tools of the error container; and
presenting the additional analytics data in the user interface.
16. The non-transitory computer-readable storage medium of claim 15 , wherein the no error container is implemented by a processor of a computing device, wherein generating the message is performed without providing power to the processor implementing the no error container.
17. The non-transitory computer-readable storage medium of claim 15 , wherein the error container is implemented by a processor of a computing device, wherein generating the additional analytics data is performed without providing power to the processor implementing the error container.
18. The non-transitory computer-readable storage medium of claim 15 , wherein generating the additional analytics data comprises generating at least one of a chart or a graph that depicts analytics derived from processing the metadata file using the at least one of the plurality of tools of the no error container.
19. The non-transitory computer-readable storage medium of claim 15 , wherein determining whether the log file includes at least one entry indicating the error is performed while the parsing tool is processing the data file, the operations further comprising pausing processing of the data file by the parsing tool until the error is resolved.
20. The non-transitory computer-readable storage medium of claim 15 , wherein generating the additional analytics data by inputting the metadata file to the at least one of the plurality of tools of the no error container is performed in parallel while the parsing tool is processing the data file.