Methods and systems for distributed computation of graph data
View Patent ↗Methods and systems for distributed computation of graph data permit edge collection and vertex collection, each to be partitioned among a plurality of computational units. In one embodiment, the methods employ a two-phase computational cycle, which is repeated until the computation is complete. In a first phase, processing units process each active edge and vertex by doing the following: reading their current attribute values, executing programmed computational functions, updating edge attributes and sending data messages to vertices. In a second phase, each vertex update processor processes each of its active vertices by doing the following: reading its current attribute values and received data messages, executing a programmed computational function, and updating the vertex's attribute values.
1. A computer-implemented method for distributed computation of a graph having one or more edges, each of the one or more edges connecting a source vertex and an endpoint vertex, the method comprising:
reading attribute values of the one or more edges by at least one edge processing unit and of the source vertices of the one or more edges by at least one vertex processing unit, the read attribute values of the one or more edges being stored in an edge list separate from a vertex list that stores the read attribute values of the source vertices of the one or more edges;
performing calculations on the read attribute values; and
updating the attribute values of the one or more edges maintained in the edge list and of the endpoint vertices of the one or more edges maintained in the vertex list, said updating based on the performed calculations,
wherein said performing is an independent process from said updating and does not modify any of the read attribute values, and said at least one edge processing unit and said at least one vertex processing unit are computer-programmable computational devices.
2. The computer-implemented method of claim 1 , further comprising maintaining a list of dependencies between read attribute values.
3. The computer-implemented method of claim 2 , wherein said updating the attribute values occurs based on the maintained list of dependencies such that there are no conflicts during said performing calculations.
4. The computer-implemented method of claim 1 , further comprising partitioning the one or more edges to one or more edge processing units and partitioning the source vertices and the endpoint vertices to one or more vertex processing units.
5. The computer-implemented method of claim 4 , wherein the one or more edge processing units work independently of each other and the one or more vertex processing units work independently of each other.
6. The computer-implemented method of claim 1 , further comprising transmitting a data message from a selected edge to the endpoint vertex of the selected edge,
wherein the data message includes at least one of an identification code of the source vertex of the selected edge and an attribute value of the selected edge.
7. The computer-implemented method of claim 6 , wherein said updating the attribute values of the endpoint vertices is also based on the transmitted data message.
8. The computer-implemented method of claim 1 , wherein each of the one or more edges is one of active or inactive edges, and said reading, performing, and updating are not performed for the inactive edges.
9. The computer-implemented method of claim 1 , wherein each source vertex or each endpoint vertex is one of active or inactive vertices, and said reading, performing, and updating are not performed for the inactive vertices.
10. The computer-implemented method of claim 1 , wherein the attribute values of the one or more edges further are stored in an edge data structure that includes at least one of an identification of the source vertex of an edge, a memory address of the source vertex of the edge, an identification of the endpoint vertex of the edge, and an edge type of the edge.
11. A system for distributed computation of a graph having one or more edges, each of the one or more edges connecting a source vertex and an endpoint vertex, the system comprising:
one or more edge processing units for reading attribute values of the one or more edges;
one or more vertex processing units for reading attribute values of the source vertices of the one or more edges; and
a graph storage unit for maintaining the attribute values of the endpoint vertices of the one or more edges, wherein the graph storage unit comprises an electronic data storage device, the read attribute values of the one or more edges are maintained in an edge list of the graph storage unit that is separate from a vertex list of the graph storage unit that stores the read attribute values of the source vertices of the one or more edges,
wherein said one or more edge processing units further perform calculations that do not modify any of the read attribute values of the one or more edges and update the attribute values of the one or more edges and of the endpoint vertices of the one or more edges based on the performed calculations, and said one or more vertex processing units further perform calculations on the attribute values of the endpoint vertices, said one or more edge processing units and said one or more vertex processing units are computer-programmable computational devices, wherein said performing calculations is an independent process from said updating.
12. The system of claim 11 , further comprising a master processor for maintaining a list of dependencies between read attribute values.
13. The system of claim 12 , wherein said one or more edge processing units further update the attribute values based on the maintained list of dependencies such that there are no conflicts during the performing of calculations.
14. The system of claim 12 , wherein said master processor further partitions the one or more edges to said one or more edge processing units and partitioning the source vertices and the endpoint vertices to said one or more vertex processing units.
15. The system of claim 14 , wherein the one or more edge processing units work independently of each other and the one or more vertex processing units work independently of each other.
16. The system of claim 11 , wherein said one or more edge processing units further transmit a data message from a selected edge to the endpoint vertex of the selected edge, and
the data message includes at least one of an identification code of the source vertex of the selected edge and an attribute value of the selected edge.
17. The system of claim 16 , wherein said one or more vertex processing units update the attribute values of the endpoint vertices based on the transmitted data message.
18. The system of claim 11 , wherein each of the one or more edges is one of active or inactive edges, and said one or more edge processing units do not perform calculations for the inactive edges.
19. The system of claim 11 , wherein each source vertex or each endpoint vertex is one of active or inactive vertices, and said one or more vertex processing units do not perform calculations for the inactive vertices.
20. The system of claim 11 , further comprising an edge data structure for maintaining the attribute values of the one or more edges that includes at least one of an identification of the source vertex of an edge, a memory address of the source vertex of the edge, an identification of the endpoint vertex of the edge, and an edge type of the edge.