System and methods for dynamic visual graph structure providing multi-stream data integrity and analysis
Systems, computer program products, and methods are described herein for dynamic visual graph structure providing multi-stream data integrity and analysis. The present disclosure is configured to provide a reactive system aimed to trace the root cause of incidents and uncover potential gaps in security of an enterprise system. Maintaining accurate and meaningful information related to incidents is the key for success of data security protocols. The integrity of message data is kept intact for improved forensic investigation, as each database may keep varying information related to a single global session.
1. A system for multi-stream data integrity and analysis, the system comprising:
a processing device;
a non-transitory storage device containing instructions when executed by the processing device, causes the processing device to perform steps of:
receive global session data from a streaming or batch data source;
based on one or more data characteristics of the global session data, determine one or more impacted databases;
generate a knowledge graph metadata file for the global session data, wherein the knowledge graph metadata file comprises nodes representing data entries, relationships representing data dependencies, and one or more predictive analytics markers;
assign a global session reference number, a unique graph key, and an integrity counter to the metadata file, wherein the integrity counter triggers an automated rollback process upon detection of compromised data integrity;
generate a unique hash value based on the global session reference number, the unique graph key, and the integrity counter;
store data in one or more upstream, downstream, or analytics databases; and
continuously and dynamically update the knowledge graph metadata file, in real-time, based on the global session data and the integrity counter, permitting the system to proactively identify and mitigate data integrity breaches prior to a propagation in the one or more upstream, downstream, or analytics databases.
2. The system of claim 1 , further comprising splitting the global session data into multiple data entries stored in multiple databases.
3. The system of claim 2 , wherein the multiple data entries are linked via a single global session identification number.
4. The system of claim 1 , further comprising:
receive a purge request to delete one or more data entries of the global session data;
execute the purge request by updating a change log; and
verify integrity of the purge request.
5. The system of claim 4 , wherein verifying the integrity of the purge request comprises:
accessing the knowledge graph metadata file for the global session data; and
determining if an integrity counter for the one or more data entries is greater than zero.
6. The system of claim 5 , wherein the system is configured to rollback the purge request and restore the one or more data entries if the integrity counter for the one or more data entries is greater than zero.
7. The system of claim 5 , wherein the system is configured to generate a request to verify validity of the one or more data entries if the integrity counter is greater than zero to determine if a rollback of data deletion is necessary.
8. A computer program product for multi-stream data integrity and analysis, the computer program product comprising a non-transitory computer-readable medium comprising code causing an apparatus to:
receive global session data from a streaming or batch data source;
based on one or more data characteristics of the global session data, determine one or more impacted databases;
generate a knowledge graph metadata file for the global session data, wherein the knowledge graph metadata file comprises nodes representing data entries, relationships representing data dependencies, and one or more predictive analytics markers;
assign a global session reference number, a unique graph key, and an integrity counter to the metadata file, wherein the integrity counter triggers an automated rollback process upon detection of compromised data integrity;
generate a unique hash value based on the global session reference number, the unique graph key, and the integrity counter;
store data in one or more upstream, downstream, or analytics databases; and
continuously and dynamically update the knowledge graph metadata file, in real-time, based on the global session data and the integrity counter, permitting the system to proactively identify and mitigate data integrity breaches prior to a propagation in the one or more upstream, downstream, or analytics databases.
9. The computer program product of claim 8 , further comprising splitting the global session data into multiple data entries stored in multiple databases.
10. The computer program product of claim 9 , wherein the multiple data entries are linked via a single global session identification number.
11. The computer program product of claim 8 , further comprising:
receive a purge request to delete one or more data entries of the global session data;
execute the purge request by updating a change log; and
verify integrity of the purge request.
12. The computer program product of claim 11 , wherein verifying the integrity of the purge request comprises:
accessing the knowledge graph metadata file for the global session data; and
determining if an integrity counter for the one or more data entries is greater than zero.
13. The computer program product of claim 12 , wherein the system is configured to rollback the purge request and restore the one or more data entries if the integrity counter for the one or more data entries is greater than zero.
14. The computer program product of claim 12 , wherein the system is configured to generate a request to verify validity of the one or more data entries if the integrity counter is greater than zero to determine if a rollback of data deletion is necessary.
15. A method for multi-stream data integrity and analysis, the method comprising:
receiving global session data from a streaming or batch data source;
based on one or more data characteristics of the global session data, determining one or more impacted databases;
generating a knowledge graph metadata file for the global session data, wherein the knowledge graph metadata file comprises nodes representing data entries, relationships representing data dependencies, and one or more predictive analytics markers;
assigning a global session reference number, a unique graph key, and an integrity counter to the metadata file, wherein the integrity counter triggers an automated rollback process upon detection of compromised data integrity;
generating a unique hash value based on the global session reference number, the unique graph key, and the integrity counter;
storing data in one or more upstream, downstream, or analytics databases; and
continuously and dynamically update the knowledge graph metadata file, in real-time, based on the global session data and the integrity counter, permitting the system to proactively identify and mitigate data integrity breaches prior to a propagation in the one or more upstream, downstream, or analytics databases.
16. The method of claim 15 , further comprising splitting the global session data into multiple data entries stored in multiple databases.
17. The method of claim 16 , wherein the multiple data entries are linked via a single global session identification number.
18. The method of claim 15 , further comprising:
receive a purge request to delete one or more data entries of the global session data;
execute the purge request by updating a change log; and
verify integrity of the purge request.
19. The method of claim 18 , wherein verifying the integrity of the purge request comprises:
accessing the knowledge graph metadata file for the global session data; and
determining if an integrity counter for the one or more data entries is greater than zero.
20. The method of claim 19 , wherein the system is configured to rollback the purge request and restore the one or more data entries if the integrity counter for the one or more data entries is greater than zero.