Systems and methods of modeling and querying dynamic temporal graph on massive parallel graph processing and storage engine
A temporal graph model is provided that can handle all six graph updates cases: vertex created; vertex deleted; vertex property updated; edge created; edge deleted; and edge property updated. A memory efficient temporal graph storage solution stores the delta change of graph evolution at each update (e.g., instead of storing the whole graph snapshot at each time). Temporal graph querying techniques are described that can determine what the graph looks like at a given time in the past and can perform any graph query on that graph's snapshot at that time as if time traveling back to that time. Also provided are event sourcing techniques of recording and tracing temporal graph updates that make the evolution of the temporal graph explainable. Further provided is a seamless integration with a massive parallel graph processing and storage engine that fully leverages the multicore and multinode cluster computation and storage resources.
1. A method comprising:
receiving a graph update event for a temporal graph at a massive parallel graph processing engine, wherein the temporal graph comprises a topology comprising at least one vertex and at least one edge, wherein each of the at least one vertex and at least one edge comprises at least one property respectively, wherein the topology and the at least one properties evolve with time;
updating the temporal graph using the graph update event, wherein updating the temporal graph comprises storing at a massive parallel graph storage engine a delta change of graph evolution for each graph update event;
receiving a query for the temporal graph, wherein the query is for a previous time in the temporal graph;
processing the query using the temporal graph, wherein processing the query comprises performing the query on a snapshot of the graph at a previous point in time; and
outputting the results of the query processed using the temporal graph.
2. The method of claim 1 , wherein the graph update event comprises at least one of a topology change or a property change of at least one vertex or at least one edge of the temporal graph.
3. The method of claim 1 , wherein the graph update event comprises at least one of vertex created; vertex deleted; vertex property updated; edge created; edge deleted; or edge property updated.
4. The method of claim 1 , further comprising receiving, generating, or maintaining the temporal graph prior to receiving the graph update.
5. A system comprising:
a processor;
a memory;
an event streaming injector configured to receive a graph update for a temporal graph, wherein the temporal graph comprises a topology comprising at least one vertex and at least one edge, wherein each of the at least one vertex and at least one edge comprises at least one property respectively, wherein the topology and the at least one properties evolve with time;
a massive parallel graph processing engine configured to receive the graph update from the event streaming injector, process the graph update, and update the temporal graph, wherein updating the temporal graph comprises storing a delta change of graph evolution for the graph update; and
a massive parallel graph storage engine configured to store an initial snapshot of a graph topology and properties of the temporal graph and delta changes at each time along as time elapses, and an event associated with each time when the temporal graph has an update,
wherein the massive parallel graph processing engine is further configured to receive a query and provide a temporal storage request to the massive parallel storage engine, wherein the query is for a previous time in the temporal graph, and is further configured to provide a query response as an output to a computing device, wherein the query response is based on a snapshot of the graph at a previous point in time.
6. The system of claim 5 , wherein the massive parallel graph storage engine is further configured to provide a temporal storage response to the massive parallel graph processing engine.
7. The system of claim 5 , wherein the massive parallel graph storage engine is further configured to receive an updated temporal graph from the massive parallel graph processing engine.
8. The system of claim 5 , wherein the graph update comprises at least one of a topology change or a property change of at least one vertex or at least one edge of the temporal graph.
9. The system of claim 5 , wherein the graph update comprises at least one of vertex created; vertex deleted; vertex property updated; edge created; edge deleted; or edge property updated.
10. A system comprising:
at least one processor; and
a non-transitory computer readable medium comprising instructions that, when executed by the at least one processor, cause the system to:
receive a graph update event for a temporal graph at a massive parallel graph processing engine, wherein the temporal graph comprises: a graph schema comprising a plurality of generic vertices, wherein each generic vertex is configured to hold historical versions of the vertex in the temporal graph; and a plurality of generic edges, wherein each generic edge is configured to hold a plurality of connections between the generic vertices;
update the temporal graph using the graph update event, wherein updating the temporal graph comprises storing at a massive parallel graph storage engine a delta change of graph evolution for each graph update event;
receive a query for the temporal graph, wherein the query is for a previous time in the temporal graph;
process the query using the temporal graph, wherein processing the query comprises performing the query on a snapshot of the graph at a previous point in time; and
output the results of the query processed using the temporal graph.
11. The system of claim 10 , wherein each generic vertex comprises: a timestamp property that holds a timestamp when a specific version of the vertex is created, an identifier, and a status indicator.
12. The system of claim 11 , wherein each generic vertex further comprises a version identifier, a label, and a properties indicator.
13. The system of claim 10 , wherein each generic edge comprises: information pertaining to the endpoints of the generic edge, an identifier, and an identifier indicating whether the generic edge is directed or undirected.
14. The system of claim 13 , wherein each generic edge further comprises an edge label and a properties indicator.