SECURE DATA TRANSFORMATION AND MERGING SYSTEM
The disclosure describes embodiments of systems, methods, and non-transitory computer readable storage media that securely generates an unscrubbed Metro2 file. In particular, the disclosed systems can receive data from multiple data sources that relates to a user account. In some cases, the disclosed systems can merge the data related to the user account by utilizing a transformation framework to generate a data staging table. In one or more embodiments, the disclosed systems, based on the data stagging table can generate a scrubbed Metro2 file that does not include personal identification information. In some cases, the disclosed systems can validate the scrubbed Metro2 file by analyzing the data related to the user account. Subsequently, in some instances, based on validating the scrubbed Metro2 file, the disclosed systems can add the personal identification information to the scrubbed Metro2 file and generate an unscrubbed Metro2 file.
1 . A computer-implemented method comprising:
receiving, from multiple data sources, data related to a user account of an inter-network facilitation system;
generating a data staging table by merging the data related to the user account of the inter-network facilitation system from the multiple data sources utilizing a transformation framework;
generating, based on the data staging table, a scrubbed Metro2 file that comprises the data related to the user account but does not include personal identification information;
validating the scrubbed Metro2 file by analyzing the data related to the user account; and
based upon validating the scrubbed Metro2 file, generating an unscrubbed Metro2 file by adding the personal identification information to the scrubbed Metro2 file.
2 . The computer-implemented method of claim 1 wherein merging the data related to the user account of the inter-network facilitation system from the multiple data sources further comprises:
generating one or more base tables from the multiple data sources;
generating a set of data frames from the one or more base tables by performing a set of queries on the one or more base tables; and
merging the set of data frames according to a predefined order of steps.
3 . The computer-implemented method of claim 2 , further comprising:
validating the set of data frames by applying one or more validation scripts to the set of data frames.
4 . The computer-implemented method of claim 3 , wherein the one or more validation scripts comprises: a trend script, record comparison script, or logic script.
5 . The computer-implemented method of claim 1 , further comprising:
generating an intermediate instance of the scrubbed Metro2 file by converting the data staging table into a corresponding CVS file.
6 . The computer-implemented method of claim 1 , further comprising:
encrypting a final instance of the unscrubbed Metro2 file; and
sending an encrypted unscrubbed Metro2 file to a third-party system.
7 . The computer-implemented method of claim 1 , further comprising:
receiving an indication of an authorized transaction; and
based on the authorized transaction, requesting the data related to the user account of the inter-network facilitation system from the multiple data sources.
8 . The computer-implemented method of claim 7 , wherein the authorized transaction is an automatic funds transfer from a secured account to a credit account.
9 . A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computing device to:
receive, from multiple data sources, data related to a user account of an inter-network facilitation system;
generate a data staging table by merging the data related to the user account of the inter-network facilitation system from the multiple data sources utilizing a transformation framework;
generate, based on the data staging table, a scrubbed Metro2 file that comprises the data related to the user account but does not include personal identification information;
validate the scrubbed Metro2 file by analyzing the data related to the user account; and
based upon validating the scrubbed Metro2 file, generate an unscrubbed Metro2 file by adding the personal identification information to the scrubbed Metro2 file.
10 . The non-transitory computer-readable medium of claim 9 , wherein merging the data related to the user account of the inter-network facilitation system from the multiple data sources further comprises:
generating one or more base tables from the multiple data sources;
generating a set of data frames from the one or more base tables by performing a set of queries on the one or more base tables; and
merging the set of data frames according to a predefined order of steps.
11 . The non-transitory computer-readable medium of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
validate the set of data frames by applying one or more validation scripts to the set of data frames.
12 . The non-transitory computer-readable medium of claim 11 , wherein the one or more validation scripts comprises: a trend script, record comparison script, or logic script.
13 . The non-transitory computer-readable medium of claim 9 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
generate an intermediate instance of the scrubbed Metro2 file by converting the data staging table into a corresponding CVS file.
14 . The non-transitory computer-readable medium of claim 9 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
encrypting a final instance of the unscrubbed Metro2 file; and
sending an encrypted unscrubbed Metro2 file to a third-party system.
15 . The non-transitory computer-readable medium of claim 9 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
receive an indication of an authorized transaction; and
based on the authorized transaction, request the data related to the user account of the inter-network facilitation system from the multiple data sources.
16 . The non-transitory computer-readable medium of claim 15 , wherein the authorized transaction is an automatic funds transfer from a secured account to a credit account.
17 . A system comprising:
at least one processor; and
at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
receive, from multiple data sources, data related to a user account of an inter-network facilitation system;
generate a data staging table by merging the data related to the user account of the inter-network facilitation system from the multiple data sources utilizing a transformation framework;
generate, based on the data staging table, a scrubbed Metro2 file that comprises the data related to the user account but does not include personal identification information;
validate the scrubbed Metro2 file by analyzing the data related to the user account; and
based upon validating the scrubbed Metro2 file, generate an unscrubbed Metro2 file by adding the personal identification information to the scrubbed Metro2 file.
18 . The system of claim 17 , wherein merging the data related to the user account of the inter-network facilitation system from the multiple data sources further comprises:
generating one or more base tables from the multiple data sources;
generating a set of data frames from the one or more base tables by performing a set of queries on the one or more base tables; and
merging the set of data frames according to a predefined order of steps.
19 . The system of claim 18 , further comprising instructions that, when executed by the at least one processor, cause the system to:
validate the set of data frames by applying one or more validation scripts to the set of data frames.
20 . The system of claim 19 , wherein the one or more validation scripts comprises: a trend script, record comparison script, or logic script.