Techniques for generic data extraction
View Patent ↗Techniques are presented for generic data extraction. Metadata defines a data extraction element with reference to an identifier, a data source, and one or more rules. The metadata may be processed to populate the data extraction element when a reference is made within a data template to the identifier. The identifier may be used to import data to a template or to export data to a different template or service.
1. A method implemented in a computer-readable medium and to process on a computer for performing the method, comprising:
receiving an identifier for a data extraction element;
acquiring data sources for populating the data extraction element, the data sources are different from one another;
establishing one or more rules; and
associating the identifier, the data sources, and the one or more rules as metadata for defining the data extraction element, the metadata is a schema that includes the identifier for referencing the data extraction element, an identifier for the data sources;
using the rules for acquiring, filtering, calculating, and organizing data that populates the data extraction element within that data extraction element, and each of the rules include a data store query for acquiring the data, a calculation for performing the calculating against the data, a sorting algorithm for organizing the data, a permissible range of data values for the data, a default data value for the data values, a type of sorting to perform on the data values, a data attribute, and a formula for scoring the data within the data extraction element, the type of sorting defines an ascending order or descending order;
processing the metadata when references are made to the data extraction element within data templates;
indexing the identifier within a database along with descriptive information;
sharing the data extraction element via the indexed identifier along with the descriptive information within the database;
making the data extraction element is accessible via the identifier from the database or the data templates as a cohesive unit that is independent of the data sources and the their structures; and
establishing at least one rule to define a join attribute, the join attribute determines whether a particular record that has no value for a given field defined within the data extraction element is to be included or excluded from the populated data extraction element.
2. The method of claim 1 further comprising, interactively processing the method via a graphical user interface (GUI) tool.
3. The method of claim 1 , wherein acquiring further includes receiving a mapping that maps one or more fields of the data sources with one or more additional fields to be associated with the data extraction element via specific schema elements of the schema.
4. The method of claim 1 further comprising:
generating a search query that conforms to the one or more rules; and
populating the data extraction element using additional ones of the one or more rules and an answer set returned in response to executing the search query.
5. The method of claim 1 further comprising: embedding the identifier in the data templates.
6. The method of claim 4 further comprising: performing additional calculations against the answer set before populating the data extraction element.