System and method for parallel search on explicitly represented graphs
One embodiment of the present invention provides a system for partitioning a graph representing customer purchases to facilitate parallel computations. During operation, the system initially receives graph data indicating vertices and edges of the graph, wherein the vertices represent customers and products and the edges represent purchases. Next, the system partitions edges of the graph to generate a partitioned graph such that each edge of the graph is a member of a respective partition. The system may then perform parallel computations on the graph data in one or more partitions to determine product recommendations.
1. A computer-executable method for recommending a product to a consumer, comprising:
copying, by one or more processors to a memory of one or more computing devices, data representing source vertices, destination vertices, and edges of a graph, wherein a respective edge pointing from a source vertex representing a respective consumer to a destination vertex representing a respective product indicates that the respective consumer consumes the respective product;
partitioning the graph edges based on destination vertices to generate a plurality of successor-partitioned graphs;
partitioning the graph edges based on source vertices to generate a plurality of predecessor-partitioned graphs;
determining, by a plurality of processors operating in parallel, one or more products that the consumer consumes, which includes traversing, by a respective processor, a successor-partitioned graph from a source vertex representing the consumer to one or more destination vertices, wherein a respective destination vertex represents a product the consumer consumes;
determining, by the plurality of processors operating in parallel, one or more other consumers that consumes at least one product that the consumer also consumes, which includes traversing, by a respective processor, a predecessor-partitioned graph from a respective destination vertex to a respective source vertex, wherein a respective source vertex represents another consumer that consumes a product that the consumer also consumes;
determining one or more other products that the one or more other consumers consume; and
providing a recommendation to the consumer to consume one of the one or more other products.
2. The method of claim 1 , wherein the parallel computations comprise executing an n-bit parallel breadth-first search where n bits represent a degree of separation or a vertex and n<8.
3. The method of claim 1 , further comprising:
assigning a first successor-partitioned graph with vertices in a first range to a first processor; and
assigning a second successor-partitioned graph with vertices in a second range to a second processor, wherein the first range and second range are different ranges.
4. The method of claim 1 , wherein no two processors of the plurality of processors operating in parallel operate on the same vertex.
5. The method of claim 1 , wherein partitioning the graph edges based on destination vertices further comprises:
determining a number of partitions for dividing the destination vertices; and
dividing a range of the destination vertices among the number of partitions.
6. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for recommending a product to a consumer, comprising:
copying, by one or more processors to a memory of one or more computing devices, data representing source vertices, destination vertices, and edges of a graph, wherein a respective edge pointing from a source vertex representing a respective consumer to a destination vertex representing a respective product indicates that the respective consumer consumes the respective product;
partitioning the graph edges based on destination vertices to generate a plurality of successor-partitioned graphs;
partitioning the graph edges based on source vertices to generate a plurality of predecessor-partitioned graphs;
determining, by a plurality of processors operating in parallel, one or more products that the consumer consumes, which includes traversing, by a respective processor, a successor-partitioned graph from a source vertex representing the consumer to one or more destination vertices, wherein a respective destination vertex represents a product the consumer consumes;
determining, by the plurality of processors operating in parallel, one or more other consumers that consumes at least one product that the consumer also consumes, which includes traversing, by a respective processor, a predecessor-partitioned graph from a respective destination vertex to a respective source vertex, wherein a respective source vertex represents another consumer that consumes a product that the consumer also consumes;
determining one or more other products that the one or more other consumers consume; and
providing a recommendation to the consumer to consume one of the one or more other products.
7. The non-transitory computer-readable storage medium of claim 6 , wherein the parallel computations comprise executing an n-bit parallel breadth-first search where n bits represent a degree of separation or a vertex and n<8.
8. The non-transitory computer-readable storage medium of claim 6 , wherein the method further comprises:
assigning a first successor-partitioned graph with vertices in a first range to a first processor; and
assigning a second successor-partitioned graph with vertices in a second range to a second processor, wherein the first range and second range are different ranges.
9. The non-transitory computer-readable storage medium of claim 6 , wherein no two processors of the plurality of processors operating in parallel operate on the same vertex.
10. The non-transitory computer-readable storage medium of claim 6 , wherein partitioning the graph edges based on destination vertices further comprises:
determining a number of partitions for dividing the destination vertices; and
dividing a range of the destination vertices among the number of partitions.
11. A computing system for recommending a product to a consumer, the system comprising:
one or more processors,
a non-transitory computer-readable medium coupled to the one or more processors having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to perform a method comprising:
copying, by one or more processors to a memory of one or more computing devices, data representing source vertices, destination vertices, and edges of a graph, wherein a respective edge pointing from a source vertex representing a respective consumer to a destination vertex representing a respective product indicates that the respective consumer consumes the respective product;
partitioning the graph edges based on destination vertices to generate a plurality of successor-partitioned graphs;
partitioning the graph edges based on source vertices to generate a plurality of predecessor-partitioned graphs;
determining, by a plurality of processors operating in parallel, one or more products that the consumer consumes, which includes traversing, by a respective processor, a successor-partitioned graph from a source vertex representing the consumer to one or more destination vertices, wherein a respective destination vertex represents a product the consumer consumes;
determining, by the plurality of processors operating in parallel, one or more other consumers that consumes at least one product that the consumer also consumes, which includes traversing, by a respective processor, a predecessor-partitioned graph from a respective destination vertex to a respective source vertex, wherein a respective source vertex represents another consumer that consumes a product that the consumer also consumes;
determining one or more other products that the one or more other consumers consume; and
providing a recommendation to the consumer to consume one of the one or more other products.
12. The computing system of claim 11 , wherein the parallel computations comprise executing an n-bit parallel breadth-first search where n bits represent a degree of separation or a vertex and n<8.
13. The computing system of claim 11 , wherein the method further comprises:
assigning a first successor-partitioned graph with vertices in a first range to a first processor, and
assigning a second successor-partitioned graph with vertices in a second range to a second processor, wherein the first range and second range are different ranges.
14. The computing system of claim 11 , wherein no two processors of the plurality of processors operating in parallel operate on the same vertex.
15. The computing system of claim 11 , wherein partitioning the graph edges based on destination vertices further comprises:
determining a number of partitions for dividing the destination vertices; and
dividing a range of the destination vertices among the number of partitions.