Adaptive engine for processing geographic data
View Patent ↗The subject disclosure pertains to systems and methods for processing input data provided in multiple formats to generate output data in a standardized, common format. To facilitate processing of the input data, a set of rules can be defined that describe processing for various types and formats of input data. The rules can be maintained separately from the processing component, such that the rules can be updated without necessitating modification of the processing component. Subsets of the rules can be retrieved and utilized to process specific input data. Errors within the input data can be identified and automatically corrected based upon previous input data sets. In addition, rules can be automatically generated based on previous input data and user feedback.
1. A system for transforming geographical data from various vendor formats to a standardized format, comprising:
a first memory for storing first data from a first data source, the first data having a first format, and for storing second data from a second data source, the second data source having a second format;
a second memory for storing a first rule for transforming said first data from the first data source into said common output data and for storing a second rule for transforming said second data from the second data source into said common output data; and
a processor coupled to the second memory for selecting data to be transformed into said output data, for identifying a source of the data to be transformed, and for transforming the data according to said first rule if the identified source of data comprises the first data source and for transforming the data into said output data according to said second rule if the identified source of data comprises the second data source.
2. The system of claim 1 , the processor is further for generating executable code from the rule associated with the selected data from the second memory; and for
executing the executable code over the selected data.
3. The system of claim 1 , the processor further for utilizing a machine learning system to identify possible errors in the data stored in the first memory based at least in part upon previously stored data.
4. The system of claim 1 , the processor further for generating a new rule using an inference based at least in part on previously stored data.
5. The system of claim 1 , further comprising:
a transformation user interface that allows a user to select one of said rules the second memory.
6. The system of claim 1 , further comprising:
a rule user interface that provides a user with the ability to update the second memory.
7. The system of claim 1 , wherein each of said rules stored in said second memory is specified using a natural language.
8. A method for transforming geographical data from various vendor formats to a standardized format, comprising:
storing a first set of data records arranged in a first format, the second set of data records comprising geographical data;
storing a second set of data records arranged in a second format, the second set of data records comprising geographical data;
storing a first set of instructions associated with the first format;
storing a second set of instructions associated with the second format; and
converting said first set of data records into output data, said output data having said standardized format, using at least one instruction from the first set of instructions the at least one instruction defining an expression to execute on the first set of data records.
9. The method of claim 8 , wherein the step of converting further comprises:
executing at least one instruction from the second set of instructions, the at least one instruction defining an expression to execute on the second set of data records.
10. The method of claim 9 , wherein the at least one instruction from the second set of instructions has an associated priority and the at least one instruction from the second set of instructions is executed in order based at least in part upon the priority.
11. The method of claim 8 , further comprising:
detecting a possible error in the input data based at least in part on at least one of said instructions.
12. The method of claim 8 , further comprising:
detecting a possible error in the first set of data records using an inference based at least in part on previously stored data.
13. The method of claim 12 , further comprising:
generating a confidence level for the possible error; and
selecting an error handling action based upon a comparison of the confidence level to a predefined threshold.
14. The method of claim 8 , further comprising:
generating an additional instruction using an inference based at least in part on previously stored data; and
maintaining the additional instruction with the first set of instructions.
15. The method of claim 8 , wherein the at least one instruction from the first set of instructions is selected using an inference based at least in part on previously stored data.
16. A system for processing geographical data from multiple vendors in a variety of formats to generate standardized output comprising:
means for storing a first set of data records arranged in a first format and for storing a second set of data records arranged in a second format, the first and second set of data records comprising geographical data;
means for storing a first set of instructions associated with the first format and for storing a second set of instructions associated with the second format; and
means for converting said first set of data records into output data, said output data having said standardized format, using at least one instruction from the first set of instructions, the at least one instruction defining an expression to execute on the first set of data records.
17. The system of claim 16 , further comprising:
means for generating an additional instruction utilizing a machine learning system; and
storing the additional instruction in the means for storing a first set of instructions.
18. The system of claim 16 , further comprising:
means for detecting a possible error in the first set of data records;
means for assigning a probability to the possible error; and
means for performing error handling based at least in part on the probability of the possible error.
19. The system of claim 16 , further comprising:
means for selecting the at least one instruction from the instruction data store.