Data extraction using object relationship templates
Methods, systems, apparatus, and machine-readable media facilitate a system for data extraction using object relationship templates. In an enterprise cloud computing environment, a system for data extraction using object relationship templates is implemented to dynamically generate data extraction flows from one or more enterprise data sources quickly and accurately in response to changing business needs. Object relationship templates representing all of a portion of a data extraction flow are created and stored for reuse either alone or in combination with other data extraction flows and other templates to create new data extraction flows. Corresponding methods, systems, apparatus, and machine-readable media for data extraction using object relationship templates can be implemented in servers supporting the enterprise cloud computing environment.
1. A computer-implemented method to present a graphical depiction of a definition for extracting data from a data store, the method comprising: in a device having a display interface, the device in communication with a data source:
displaying, on the display interface, a first visual representation of a first data extraction flow, the first data extraction flow defining a first object-relationship template comprising a first set of object-relationships used to create a first dataset from the data source;
searching, with one or more hardware processors, an object relationship template repository maintained in a database for one or more object-relationship templates that reference the first object-relationship template to generate, with the one or more hardware processors, a list of object-relationship templates that reference the first object relationship template;
displaying, on the display interface, for an object appearing in the first visual representation of the data extraction flow, the list of object-relationship templates that reference the first object-relationship;
in response to user input, selecting, with one or more hardware processors, at least a second object-relationship template from the list of object-relationship templates maintained in the database referencing the first object-relationship template;
combining, with the one or more hardware processors, the first object-relationship template with the at least the second object-relationship template to form an updated visual representation of a second data extraction flow, different from the first data extraction flow, the second data extraction flow comprising the first set of object-relationships combined with the object-relationships from the at least the selected second object-relationship template;
displaying, on the display interface, the updated visual representation of the second data extraction flow;
saying, in response to the user input, the updated visual representation of the second data extraction flow as a new object relationship template; and
storing the new object relationship template in a repository, wherein the new object-relationship template is accessible based on any one of the objects referenced in the template.
2. The method of claim 1 further comprising:
creating, in response to user input, a second dataset extracted from the data store using the second data extraction flow.
3. The method of claim 1 , further comprising:
saving, in response to user input, the updated visual representation of the second data extraction flow as an object relationship template; and
storing the object relationship template in a repository, wherein the object-relationship template is accessible based on any one of the objects referenced in the template.
4. The method of claim 1 , wherein generating an updated visual representation of the data extraction flow includes:
generating any associated Java Script Object Notation (JSON) for the object-relationships from the at least selected second object-relationship template; and
appending the JSON to a dataflow definition file representing the data extraction flow.
5. A system for extracting data from a data source, the system comprising:
a repository for storing reusable object-relationship templates, an object-relationship template containing a set of object relationships used for building a dataflow definition;
a user device having a display;
a processor in communication with the repository and the user device, the processor having instructions for building the dataflow definition, the instructions comprising:
displaying, on the display, a first visual representation of a first data extraction flow, the first data extraction flow defining a first object-relationship template comprising a first set of object-relationships used to create a first dataset from the data source;
searching, with one or more hardware processors, an object relationship template repository maintained in a database for one or more object-relationship templates that reference the first object-relationship template to generate, with the one or more hardware processors, a list of object-relationship templates that reference the first object relationship template;
displaying, on the display, for an object appearing in the first visual representation of the data extraction flow, the list of object-relationship templates that reference the first object-relationship template;
in response to user input, selecting, with one or more hardware processors, at least a second object-relationship template from the list of object-relationship templates maintained in the database referencing the first object-relationship template;
combining, with the one or more hardware processors, the first object-relationship template with the at least the second object-relationship template to form an updated visual representation of a second data, extraction flow, different from the first data extraction flow, the second data extraction flow comprising the first set of object-relationships combined with the object-relationships from the at least the selected second object-relationship template;
displaying, on the display, the updated visual representation of the second data extraction flow;
saying, in response to the user input, the updated visual representation of the second data extraction flow as a new object relationship template; and
storing the new object relationship template in a repository, Wherein the new object-relationship template is accessible based on any one of the objects referenced in the template.
6. The system of claim 5 , further comprising:
a data store in communication with the processor;
wherein the processor having instructions for creating a dataset in accordance with the dataflow definition, the instructions comprising:
extracting data from the data store in accordance with the dataflow definition; and
creating, in response to user input, a second dataset from the extracted data, the second dataset containing data for the second set of object-relationships displayed in the visual representation.
7. The system of claim 5 , wherein the processor having instructions for managing the storage and selection of reusable object relationship templates, the instructions comprising:
saving, in response to user input, the visual representation of the second data extraction flow as an object relationship template; and
storing the second object relationship template in a repository, wherein the second object-relationship template is accessible based on any one of the objects referenced in the template.
8. The system of claim 5 , wherein the instructions for building the dataflow definition from the updated set of object relationships displayed in the visual representation further comprising:
generating any associated Java Script Object Notation (JSON) for the object-relationships contained in the at least selected second object-relationship templates; and
appending the JSON to a dataflow definition file corresponding to the updated visual representation of the updated set of object relationships.
9. An article of manufacture comprising a computer readable storage medium having content stored thereon, Which when executed, cause a machine to perform operations including:
in a device having a display interface, the device in communication with a data store:
displaying, on the display interface, a first visual representation of a first data extraction flow, the first data extraction flow defining a first object-relationship template comprising a first set of object-relationships used to create a first dataset from the data source;
searching, with one or more hardware processors, an object relationship template repository maintained in a database for one or more object-relationship templates that reference the first object-relationship template to generate, with the one or more hardware processors, a list of object-relationship templates that reference the first object relationship template;
displaying, on the display interface, for an object appearing in the first visual representation of the data extraction flow, the list of object-relationship templates that reference the first object-relationship template;
in response to user input, selecting, with one or more hardware processors, at least a second object-relationship template from the list of object-relationship templates maintained in the database referencing the first object-relationship template;
combining, with the one or more hardware processors, the first object-relationship template with the at least Hall the second object-relationship template to form an updated visual representation of a second data extraction flow, different from the first data extraction flow, the second data extraction flow comprising the first set of object-relationships combined with the object-relationships from the at least the selected second object-relationship template;
displaying, on the display interface, the updated visual representation of the second data extraction flow;
saving, in response to the user input, the updated visual representation of the second data extraction flow as a new object relationship template; and
storing the new object relationship template in a repository, Wherein the new object-relationship template is accessible based on any one of the objects referenced in the template.
10. The article of manufacture of claim 9 , wherein the machine is to perform operations further including:
creating, in response to user input, a dataset extracted from the data store using the second data extraction flow.
11. The article of manufacture of claim 9 , wherein the machine is to perform operations further including:
saving, in response to user input, the visual representation of the second data extraction flow as an object relationship template; and
storing the second object relationship template in a repository, wherein the second object-relationship template is accessible based on any one of the objects referenced in the template.
12. The article of manufacture of claim 9 , wherein the operation of updating the visual representation of the data extraction flow includes:
generating any associated Java Script Object Notation (JSON) for the object-relationships from the at least selected second object-relationship template; and
appending the JSON to a dataflow definition file representing the data extraction flow.