System and method for providing data migration services
View Patent ↗Embodiments of the invention provide techniques for migrating data which allows transforming a data source of an existing system into a migrated data source complying with data requirements of a new system, typically in response to an upgrade of an application configured to access the data source. The system and method for data migration services is reusable (i.e., it may be used for different levels of data migration, e.g., in system upgrades to different versions), idempotent (i.e., it may be used multiple times on the same level of data migration, e.g., it may be applied multiple time in the same system upgrade of the same data without causing any disruption or different result when applied multiple times), and metadata driven (i.e., it does not require significant changes to be used for data migration of different systems).
1. A method for migrating a version of a data source from an initial version to a target version required by a target system, the method comprising:
retrieving a set of target metadata, wherein the data source comprises a relational database configured according to a source schema, and wherein the target metadata specifies a target schema of the relational database required by the target system;
retrieving a plurality of migration rules applied to migrate the source schema from the source schema to the target schema;
determining an ordered priority for applying the plurality of migration rules; and
applying the plurality of migration rules according to the determined priority, wherein applying the plurality of migration rules comprises:
for each of the plurality of migration rules:
identifying at least one element of the initial version of the data source configured according to the source schema and subject to a respective migration rule,
analyzing, based on the respective migration rule, the target metadata and the identified source schema element to determine at least one difference between the identified source schema element and the target schema as specified by the target metadata, wherein the target metadata describes a structure of the data source required by the target schema, relative to the respective migration rule, and
transforming the source schema element using the determined at least one difference, wherein the transformed source schema element conforms to the structure of the target schema, as specified by the target metadata.
2. The method of claim 1 , wherein the data source comprises a file system, and wherein the target metadata specifies a structure of the file system required by the target system.
3. The method of claim 1 , wherein at least one migration rule specifies to create all new tables of the target database schema not present in the initial version of the data source configured according to the source schema.
4. The method of claim 1 , wherein at least one migration rules specifies to add one or more columns to a table present in the initial version of the data source.
5. The method of claim 1 , wherein at least one migration rule specifies to add one or more class definitions to the initial version of the data source.
6. The method of claim 1 , wherein at least one migration rule specifies to add one or more rows to an existing table in the initial version of the data source.
7. The method of claim 1 , wherein the target system is a system selected from one of a database management system, a filing system, and a content management system.
8. A computer-readable storage medium containing a program which, when executed, performs an operation for migrating a version of a data source from an initial version to a target version required by a target system, the operation comprising:
retrieving a set of target metadata, wherein the data source comprises a relational database configured according to a source schema, and wherein the target metadata specifies a target schema of the relational database required by the target system;
retrieving a plurality of migration rules applied to migrate the source schema from the source schema to the target schema;
determining an ordered priority for applying the plurality of migration rules; and applying the plurality of migration rules according to the determined priority, wherein applying the plurality of migration rules comprises; and
for each of the plurality of migration rules:
identifying at least one element of the initial version of the data source configured according to the source schema and subject to a respective migration rule,
analyzing, based on the respective migration rule, the target metadata and the identified source schema element to determine at least one difference between the identified source schema element and the target schema as specified by the target metadata, wherein the target metadata describes a structure of the data source required by the target schema, relative to the respective migration rule, and
transforming the source schema element using the determined at least one difference, wherein the transformed source schema element conforms to the structure of the target schema, as specified by the target metadata.
9. The computer-readable storage medium of claim 8 , wherein the data source comprises a file system, and wherein the target metadata specifies a structure of the file system required by the target system.
10. The computer-readable storage medium of claim 8 , wherein at least one migration rule specifies to create all new tables of the target database schema not present in the initial version of the data source configured according to the source schema.
11. The computer-readable storage medium of claim 8 , wherein at least one migration rules specifies to add one or more columns to a table present in the initial version of the data source.
12. The computer-readable storage medium of claim 8 , wherein at least one migration rule specifies to add one or more class definitions to the initial version of the data source.
13. The computer-readable storage medium of claim 8 , wherein at least one migration rule specifies to add one or more rows to an existing table in the initial version of the data source.
14. The computer-readable storage medium of claim 8 , wherein the target system is a system selected from one of a database management system, a filing system, and a content management system.
15. A system for providing a data migration service, the system comprising:
a processor;
a data source;
target metadata describing data requirements of a target computing system; and
a memory storing a universal data migration module (UDMM) application in communication with the data source and the target metadata, wherein the UDMM, when executed by the processor, performs an operation for migrating a version of a data source from an initial version to a target version required by a target system by performing the steps of:
retrieving a set of target metadata, wherein the data source comprises a relational database configured according to a source schema, and wherein the target metadata specifies a target schema of the relational database required by the target system;
retrieving a plurality of migration rules applied to migrate the source schema from the source schema to the target schema;
determining an ordered priority for applying the plurality of migration rules; and applying the plurality of migration rules according to the determined priority,
wherein applying the plurality of migration rules comprises:
for each of the plurality of migration rules:
identifying at least one element of the initial version of the data source configured according to the source schema and subject to a respective migration rule,
analyzing, based on the respective migration rule, the target metadata and the identified source schema element to determine at least one difference between the identified source schema element and the target schema as specified by the target metadata, wherein the target metadata describes a structure of the data source required by the target schema, relative to the respective migration rule, and
transforming the source schema element using the determined at least one difference, wherein the transformed source schema element conforms to the structure of the target schema, as specified by the target metadata.
16. The system of claim 15 , wherein the data source comprises a file system, and wherein the target metadata specifies a structure of the file system required by the target system.
17. The system of claim 15 , wherein at least one migration rule specifies to create all new tables of the target database schema not present in the initial version of the data source configured according to the source schema.
18. The system of claim 15 , wherein at least one migration rules specifies to add one or more columns to a table present in the initial version of the data source.
19. The system of claim 15 , wherein at least one migration rule specifies to add one or more class definitions to the initial version of the data source.
20. The system of claim 15 , wherein at least one migration rule specifies to add one or more rows to an existing table in the initial version of the data source.
21. The system of claim 15 , wherein the target system is a system selected from one of a database management system, a filing system, and a content management system.