Aggregated embeddings for a corpus graph
Systems and methods for generating embeddings for nodes of a corpus graph are presented. More particularly, operations for generation of an aggregated embedding vector for a target node is efficiently divided among operations on a central processing unit and operations on a graphic processing unit. With regard to a target node within a corpus graph, processing by one or more central processing units (CPUs) is conducted to identify the target node's relevant neighborhood (of nodes) within the corpus graph. This information is prepared and passed to one or more graphic processing units (GPUs) that determines the aggregated embedding vector for the target node according to data of the relevant neighborhood of the target node.
1 . A computer-implemented method, comprising:
determining, using a trained neural network, a target node embedding for a target node of a corpus graph maintained as a graph convolution network (GCN), wherein the target node is a node of a plurality of nodes of the corpus graph and the plurality of nodes are associated with a plurality of content items;
performing a plurality of random walks, beginning from the target node, to determine a relevant neighborhood of the target node within the corpus graph, wherein the neighborhood of the target node includes a plurality of neighbor nodes visited during a random walk of the plurality of random walks;
determining a respective importance value for the one or more neighbor nodes of the plurality of neighbor nodes, wherein determining the importance value for the one or more neighbor node comprises determining a count of visits to the respective neighbor nodes during the plurality of random walks;
selecting a subgroup of one or more nodes from the neighbor nodes based on the respective importance values;
generating aggregated neighborhood embedding information corresponding to the target node, the generating comprising:
applying a weight to the one or more neighbor nodes of the subgroup based on the count of visits to the respective neighbor nodes, and
selecting one or more embedding values from one or more of the one or more neighbor nodes of the subgroup, wherein the likelihood of selecting an embedding from a given neighbor node is based on the corresponding weight, to generate the aggregated neighborhood embedding information;
combining the target node embedding and the aggregated neighborhood embedding information to generate an aggregated embedding vector for the target node;
in response to a query corresponding to the target node, evaluating the aggregated embedding vector to identify a content item corresponding to a neighbor node using the aggregated neighborhood embedding information; and
providing the content item to a user device.
2 . The computer-implemented method of claim 1 , further comprising:
determining, for one or more nodes of the plurality of nodes, a corresponding embedding;
determining, for one or more nodes of the plurality of nodes, a corresponding aggregated neighborhood embedding information based at least in part on a respective relevant neighborhood; and
combining, for one or more nodes of the plurality of nodes, the corresponding embedding and the corresponding aggregated neighborhood embedding information to generate a corresponding aggregated embedding vector.
3 . The computer-implemented method of claim 1 , wherein the target embedding is determined by the trained neural network based at least in part on a first content item of the plurality of content items that is associated with the target node and metadata associated with the first content item.
4 . The computer-implemented method of claim 1 , wherein the relevant neighborhood is a subgraph of the corpus graph that represents a set of most relevant nodes from the plurality of nodes to the target node.
5 . The computer-implemented method of claim 1 , wherein the aggregated embedding vector for the target node is generated without processing one or more of the plurality of nodes.
6 . A computer-implemented method, comprising:
determining, for a target node from a plurality of nodes associated with a corpus graph, a target node embedding, wherein the nodes of the plurality of nodes are associated with a respective content item and a respective embedding vector that is representative of the respective content item and is generated by a deep neural network;
determining, for the target node, a plurality of neighbor nodes that form a relevant neighborhood of the target node, the determining comprising:
performing a plurality of random walks, beginning from the target node, to determine a neighborhood of the target node within the corpus graph, wherein the neighborhood of the target node includes a plurality of neighbor nodes visited during a random walk of the plurality of random walks;
selecting a relevant neighborhood of the target node from the neighborhood of the target node based on a count of visits to respective neighbor nodes during the plurality of random walks;
determining, based at least in part on the plurality of neighbor nodes, aggregated neighborhood embedding information, the determining comprising:
weighting one or more neighbor nodes of the relevant neighbor nodes based on the count of visits to respective neighbor nodes, and
selecting one or more embedding values for the aggregated neighborhood embedding information from embeddings for the relevant neighbor nodes, wherein the likelihood of selecting an embedding from a given relevant neighbor node is based on the respective weights of the relevant neighbor nodes;
combining the target node embedding and aggregated neighborhood information to generate an aggregated embedding vector for the target node;
in response to a query corresponding to the target node, evaluating the aggregated embedding vector to identify a content item corresponding to a neighbor node using the aggregated neighborhood information; and
providing the content item to a user device.
7 . The computer-implemented method of claim 6 , wherein determining the plurality of neighbor nodes that form the relevant neighborhood of the target node includes:
performing a random walk starting from the target node to the plurality of neighbor nodes.
8 . The computer-implemented method of claim 7 , wherein performing the random walk includes determining a respective number of visits to one or more of the plurality of neighbor nodes.
9 . The computer-implemented method of claim 8 , further comprising:
determining, based on the respective number of visits, a respective importance value for one or more neighbor node of the plurality of neighbor nodes; and
aggregating, based at least in part on the respective importance values, respective aggregated neighbor node embedding vectors associated with the plurality of neighbor nodes to generate the aggregated neighborhood embedding information.
10 . The computer-implemented method of claim 9 , wherein one or more of the aggregated neighbor node embedding vectors are determined based at least in part on a corresponding relevant neighborhood.
11 . The computer-implemented method of claim 9 , wherein aggregating the respective aggregated neighbor node embedding vectors associated with the plurality of neighbor nodes to generate the aggregated neighborhood embedding information includes normalizing the respective number of visits to one or more of the plurality of neighbor nodes.
12 . The computer-implemented method of claim 8 , wherein performing the random walk includes randomly selecting an option from a plurality of options to determine whether to reset the random walk to the target node.
13 . The computer-implemented method of claim 12 , further comprising: determining a ratio of the plurality of options corresponding to a number of the plurality of options that cause resetting of the random walk to the target node.
14 . The computer-implemented method of claim 8 , wherein the random walk is performed a predetermined number of iterations.
15 . The computer-implemented method of claim 6 , wherein the computer-implemented method is performed recursively.
16 . A computing system, comprising:
one or more processors; and
a memory storing program instructions that, when executed by the one or more processors, cause the one or more processors to at least:
determine a target node from a plurality of nodes associated with a corpus graph, wherein the nodes of the plurality of nodes are associated with a respective content item;
determine, for the target node, a plurality of relevant neighbor nodes of the target node comprising:
performing a plurality of random walks, beginning from the target node, to determine the plurality of relevant neighbor nodes associated with the target node, wherein performing the plurality of random walks includes performing a plurality of random traversals to a plurality of neighbor nodes and incrementing a count associated with each of the plurality of random traversals to the plurality of neighbor nodes;
determine the plurality of relevant neighbor nodes from the plurality of neighbor nodes based on the counts;
determine, based on the plurality of relevant neighbor nodes, aggregated neighborhood information, the determining comprising:
weighting one or more neighbor nodes of the relevant neighbor nodes based on a count of visits to respective neighbor nodes during the plurality of random walks, and
selecting one or more embedding values for the aggregated neighborhood information from embeddings for the relevant neighbor nodes, wherein the likelihood of selecting an embedding from a given relevant neighbor node is based on the respective weights of the relevant neighbor nodes; and
generate, based at least in part on the aggregated neighborhood information and a target embedding associated with the target node, an aggregated embedding for the target node;
in response to a query corresponding to the target node, evaluating the aggregated embedding to identify a content item corresponding to a neighbor node using the aggregated neighborhood information; and
providing the content item to a user device.
17 . The computing system of claim 16 , wherein:
the program instruction that, when executed by the one or more processors, further cause the one or more processors to at least determine an importance value for one or more neighbor nodes of the plurality of relevant neighbor nodes based at least in part on the counts; and
the target embedding associated with the target node is generated based at least in part on the importance values.
18 . The computing system of claim 16 , wherein the program instruction that, when executed by the one or more processors, further cause the one or more processors to at least:
determine an aggregated embedding vector for one or more neighbor nodes of the plurality of relevant neighbor nodes;
aggregate the aggregated embedding vector for the one or more neighbor nodes of the plurality of relevant neighbor nodes to generate aggregated neighborhood embedding information associated with the plurality of relevant neighbor nodes; and
combine the aggregated neighborhood embedding information with a target node embedding associated with the target node to generate the aggregated embedding for the target node.
19 . The computing system of claim 16 , wherein:
the one or more processors includes at least one central processing unit (CPU) and at least one graphical processing unit (GPU); and
the computing system further comprises:
a coordination component configured to allocate tasks between the at least one CPU and the at least one GPU.