Linking of parent-child data records in a legacy software modernization system
A system and method for linking parent and child data records during data migration is disclosed. One embodiment comprises sequentially receiving a plurality of data records from an arbitrary source of records without running a query on the source of records, generating a destination store of records that contains the plurality of data records, and locating any records in the destination store of records that has a dangling key. In this way, the embodiment may correct dangling keys to generate a destination store of records with referential integrity.
1. A method comprising:
at a conversion engine executed on a computing device:
sequentially receiving a plurality of legacy data records from a legacy source data file stored in an arbitrary source without running a query on the arbitrary source, the plurality of legacy data records including parent records and child records;
concurrently migrating the parent records and corresponding child records while loading additional legacy data records, wherein the parent records and corresponding child records are migrated, independently and in parallel, through a data migration workbench transformer of the conversion engine, such that during the loading of the plurality of data records:
assigning a surrogate key to at least one parent record of the parent records;
assigning a corresponding foreign key to the corresponding child record linked to the at least one parent record;
generating a destination store of records that contains the plurality of data records;
locating at least one record in the destination store of records that has a dangling key; and
correcting dangling keys to generate a destination store of records with referential integrity, wherein correcting dangling keys includes at least one of deleting the record with the dangling key or changing the dangling key in the record to create a valid foreign key.
2. The method of claim 1 , further comprising logging the records with dangling keys prior to correcting the dangling keys.
3. The method of claim 2 , further comprising generating a report containing logged records.
4. The method of claim 1 , wherein the arbitrary source is a database.
5. The method of claim 1 , wherein the arbitrary source is a flat file.
6. The method of claim 1 , wherein the arbitrary source is a network stream.
7. The method of claim 1 , wherein generating a destination store of records that contains the plurality of legacy data records further comprises generating a surrogate key for each natural key in the plurality of legacy data records.
8. A non-transitory computer-readable medium comprising computer-executable instructions being executable by a modernization system to:
at a conversion engine executed on a computing device:
sequentially receive a plurality of legacy data records from a legacy source data file stored in an arbitrary source without running a query on the arbitrary source, the plurality of legacy data records including parent records and child records;
concurrently migrate the parent records and corresponding child records while loading additional legacy data records, wherein the parent records and corresponding child records are migrated, independently and in parallel, through a data migration workbench transformer of the conversion engine, such that during the loading of the plurality of data records:
assign a surrogate key to at least one parent record of the parent records;
assign a corresponding foreign key to the corresponding child record linked to the at least one parent record;
generate a destination store of records that contains the plurality of data records;
locate at least one record in the destination store of records that has a dangling key; and
correct dangling keys to generate a destination store of records with referential integrity, wherein correcting dangling keys includes at least one of deleting the record with the dangling key or changing the dangling key in the record to create a valid foreign key.
9. The non-transitory computer-readable medium of claim 8 , further comprising instructions executable to log the records with dangling keys prior to correcting the dangling keys.
10. The non-transitory computer-readable medium of claim 9 , further comprising instructions executable to generate a report containing logged records.
11. The non-transitory computer-readable medium of claim 8 , wherein the arbitrary source is a database.
12. The non-transitory computer-readable medium of claim 8 , wherein the arbitrary source is a flat file.
13. The non-transitory computer-readable medium of claim 8 , wherein the arbitrary source is a network stream.
14. The non-transitory computer-readable medium of claim 8 , wherein generating a destination store of records that contains the plurality of data records further comprises generating a surrogate key for each natural key in the plurality of data records.