Generating a logical representation from a physical flow
View Patent ↗Described herein are techniques for processing analytic flows. A physical flow can be parsed into multiple elements. A logical representation can be generated based on the physical flow using a dictionary. An engine-specific logical representation or second physical flow can then be generated based on the logical representation using the dictionary.
1. A method comprising, by a processing system:
parsing a physical hybrid flow having a first portion specific to and executable by a first execution engine and a second portion specific to and executable by a second execution engine into multiple elements;
generating a logical representation from the physical hybrid flow using a dictionary by converting the multiple elements into logical elements represented in a logical language, the dictionary comprising a first mapping between the logical language and a first programming language associated with the first execution engine and a second mapping between the logical language and a second programming language associated with the second execution engine, wherein the logical representation is engine-agnostic;
generating a different logical representation from the logical representation, wherein the different logical representation is specific to a third execution engine; and
generating code from the different logical representation, wherein the code is executable by the third execution engine.
2. The method of claim 1 , wherein the first physical flow comprises at least one of a script written in a programming language or flow metadata generated by a flow-design tool.
3. The method of claim 1 , wherein the dictionary comprises an entry for an operator in the logical language, the entry including a name of the operator in the logical language, a designation of the operator in a supported programming language, and a template structure for conversion between the logical language and the supported programming language.
4. The method of claim 3 , wherein each entry further comprises a link to a cost module to compute a cost of the respective operator for the supported programming language.
5. The method of claim 3 , wherein the dictionary further comprises engine-specific mappings providing additional details for implementing operators in specific engines.
6. A system comprising:
a flow parser to parse a physical hybrid flow having a first portion specific to and executable by a first execution engine and a second portion specific to and executable by a second execution engine into multiple elements;
an engine-agnostic generator to generate an engine-agnostic logical representation from the physical hybrid flow using a dictionary by converting the multiple elements into logical element represented in a logical language, the dictionary comprising a first mapping between the logical language and a first programming language associated with the first execution engine and a second mapping between the logical language and a second programming language associated with the second execution engine;
an engine-specific generator to generate an engine-specific logical representation from the engine-agnostic logical representation, the engine-specific logical representation specific to a third execution engine; and
a code generator to generate executable code from the engine-specific logical representation, wherein the code is executable by the third execution engine.
7. The system of claim 6 , further comprising a dictionary updater to update the dictionary by:
determining whether the dictionary supports a selected language;
if supported,
locking the dictionary for the selected language,
retrieving an existing dictionary template for the selected language,
receiving input updating the existing dictionary template,
merging the updated dictionary template with the dictionary, and
unlocking the dictionary for the selected language; and
if not supported,
generating a new dictionary template for the selected language,
receiving input filling in the new dictionary template, and
merging the new dictionary template with the dictionary.
8. The system of claim 6 , further comprising a dictionary updater to update the dictionary.
9. The system of claim 8 , wherein the dictionary updater is to update the dictionary by:
determining whether the dictionary supports a selected language;
if supported,
locking the dictionary for the selected language,
merging an update to the dictionary with the dictionary,
unlocking the dictionary.
10. The system of claim 9 , wherein the dictionary is to merge the update to the dictionary with the dictionary by:
receiving input updating the existing dictionary template,
merging the updated dictionary template with the dictionary.
11. The system of claim 9 , wherein the dictionary is to update the dictionary by further:
if not supported,
generating a new dictionary template for the selected language,
receiving input filling in the new dictionary template, and
merging the new dictionary template with the dictionary.
12. A non-transitory computer-readable storage medium storing instructions for execution by a computer, the instructions when executed causing the computer to:
parse a physical hybrid flow comprising a first portion written for a first execution engine and a second portion written for a second execution engine into multiple elements;
generate an engine-agnostic logical representation from the physical hybrid flow using a dictionary by converting the multiple elements into logical elements represented in a logical language, the dictionary comprising a first mapping between the logical language and a first programming language associated with the first execution engine and a second mapping between the logical language and a second programming language associated with the second execution engine;
generate an engine-specific logical representation from the engine-agnostic logical representation, wherein the engine-specific logical representation is specific to a third execution engine; and
generate executable code from the engine-specific logical representation, wherein the executable code is executable by the third execution engine.
13. The computer-readable medium of claim 12 , wherein the dictionary comprises entries for operators, each entry including:
a name of the respective operator in the logical language,
a name of the respective operator in a first programming language associated with the first execution engine,
a name of the respective operator in a second programming language associated with the second execution engine,
a template structure for conversion between the logical language and the first programming language, and
a template structure for conversion between the logical language and the second programming language.