Data export/import from multiple data source to a destination data repository using corresponding data exporters and an importer
View Patent ↗Embodiments of the present invention provide methods and systems for exporting data from a number of data sources using a number of corresponding data exporters, and importing the exported data into a destination data repository using a data importer. In various embodiments, the data exporters may be automatically generated using a data exporter generator adapted to generate the data exporters in view of data schemas of the data sources.
1. A method, comprising:
reading a data schema using a data exporter generator operated by one or more computing devices, wherein the data schema provides a data description that describes data that is associated with the data schema and that is stored in the data source; and
automatically generating a data exporter by the data exporter generator, wherein the data exporter is generated based at least in part on the data description provided by the data schema, and wherein the data exporter is adapted to export the data associated with the data schema and provide the exported data to a data importer associated with a destination data repository,
wherein the data importer is adapted to receive the exported data, selectively process the received data, and write the selectively processed data into the destination data repository,
wherein the destination data repository is a common medical service provider demographic database and the data source comprises a plurality of heterogeneous medical service provider demographic data sources, and
wherein the data exporter generator is adapted to generate a plurality of different data exporters that includes the data exporter, and wherein the plurality of different data exporters are adapted to export and facilitate importing of providers' places of service address and contracting address and to export and facilitate importing of demographic data from different regions.
2. The method of claim 1 , wherein the automatically generating the data exporter comprises the data exporter generator automatically generating a read interface for the data exporter to read the data associated with the data schema out of the data source.
3. The method of claim 2 , wherein the automatically generating the read interface comprises the data exporter generator automatically adapting a generic read interface based at least in part on the data description provided by the data schema, wherein the data description comprises at least one data format associated with the data described by the data schema.
4. The method of claim 2 , wherein the automatically generating the data exporter further comprises the data exporter generator automatically generating a validation block for the data exporter to validate the data read out of the data source by the data exporter in order to generate validated data.
5. The method of claim 4 , wherein the automatically generating the validation block comprises the data exporter generator automatically modifying a generic validation block based at least in part on the data description provided by the data schema, wherein the data description includes a description of valid data values for the data.
6. The method of claim 5 , wherein the automatically modifying the generic validation block comprises the data exporter generator automatically modifying the generic validation block based at least in part on the data description in order to invoke one or more supplemental validation routines for providing supplemental validation, wherein the data description includes information associated with the one or more supplemental validation routines.
7. The method of claim 4 , wherein the automatically generating the data exporter further comprises the data exporter generator automatically generating a transformation block and a set of transformation rules, wherein the data exporter is adapted to transform the validated data based at least in part on the set of transformation rules to generate transformed data.
8. The method of claim 7 , wherein the automatically generating the data exporter further comprises the data exporter generator automatically generating a write block, wherein the data exporter is adapted to invoke the data importer and provide the transformed data to the data importer for processing and storage into the destination data repository using the write block.
9. The method of claim 2 , wherein the data source comprises single value data and one or more lists having variable lengths, wherein each list comprises a plurality of fields.
10. The method claim 1 , wherein each of the plurality of different data exporters is generated based at least in part on a respective corresponding data schema that describes respective data stored in a respective one of the plurality of data sources.