Iterative deep graph learning for graph neural networks
View Patent ↗An initial noisy graph topology is obtained and an initial adjacency matrix is generated by a similarity learning component using similarity learning and a similarity metric function. An updated adjacency matrix with node embeddings is produced from the initial adjacency matrix using a graph neural network (GNN). The node embeddings are fed back to revise the similarity learning component. The generating, producing, and feeding back operations are repeated for a plurality of iterations.
1 . A method for jointly learning a graph structure and graph embeddings, the method comprising:
obtaining an initial noisy graph topology;
generating, using a similarity learning component, an initial adjacency matrix using similarity learning and a similarity metric function;
producing, from said initial adjacency matrix, using a graph neural network (GNN), an updated adjacency matrix with node embeddings;
feeding back the node embeddings to revise the similarity learning component;
repeating the generating, producing, and feeding back operations for a plurality of iterations;
updating weights of the graph neural network only after a convergence of the updated adjacency matrices generated by the plurality of iterations;
performing natural language processing on input from a human subject using a selected iteration of the updated adjacency matrix; and
reconfiguring at least one network-based asset in accordance with a result of the natural language processing.
2 . The method of claim 1 , further comprising controlling, by a graph regularization component, a smoothness, a connectivity and a sparsity of a corresponding graph structure of the updated adjacency matrix using an adapted graph regularization technique.
3 . The method of claim 1 , wherein the similarity metric function comprises a weighted cosine similarity metric applied to obtain the updated adjacency matrix for all pairs of nodes of a corresponding graph structure.
4 . The method of claim 3 , wherein the weighted cosine similarity metric is defined using m weight vectors w, each weight vector w representing one perspective, to compute m independent similarity matrices s, an average of the m independent similarity matrices s, and a final similarity s based on:
s
i
j
k
=
cos
(
w
k
⊙
v
i
,
w
k
⊙
v
j
)
,
s
i
j
=
1
m
∑
k
=
1
m
s
i
j
k
where s ij k computes a cosine similarity between input vectors v i and v j for a k-th perspective, where each perspective considers one part of semantics captured in the input vectors v i and v j .
5 . The method of claim 4 , wherein the adjacency matrix is designated as A (t) and is extracted from S by considering elements in S which are smaller than a non-negative threshold ε in an ε-neighborhood for each node, where:
A
ij
=
{
s
i
j
s
i
j
>
ε
0
otherwise
.
6 . The method of claim 3 , wherein the weighted cosine similarity metric is defined as:
s ij =cos( w⊙v i ,w⊙v j )
where ⊙ denotes a Hadamard product and w comprises a learnable weight vector which has a same dimension as input vectors v i and v j .
7 . The method of claim 6 , wherein each input vector v i and v j comprises one of raw node features and computed node embeddings.
8 . The method of claim 1 , wherein the updated adjacency matrix is learned by minimizing a joint loss function based on = pred + , where pred is a prediction loss based on a downstream task and is a graph regularization loss.
9 . The method of claim 8 , wherein the graph regularization loss is defined by:
=αΩ( A,X )+ f ( A )
where α is a non-negative hyperparameter, A is an adjacency matrix, X is a feature matrix, and Ω(A,X) is a smoothness loss.
10 . The method of claim 8 , wherein is based on a Frobenius norm of the updated adjacency matrix.
11 . The method of claim 1 , wherein the initial noisy graph topology is obtained from one of an original data graph and a graph constructed using a k-nearest neighbors (kNN) strategy, the graph is based on sequential data or a feature matrix of a corresponding application.
12 . The method of claim 11 , further comprising selecting an iteration of the updated adjacency matrix as a replacement for the initial noisy graph topology.
13 . The method of claim 12 , wherein the natural language processing is performed for a call center task.
14 . The method of claim 1 , wherein the repeating the generating, producing, and feeding back operations continues until a corresponding updated adjacency matrix designated as A (t) is sufficiently similar to a corresponding optimized graph for use in prediction based on a threshold ε to derive a final graph à (t) .
15 . The method of claim 14 , wherein the final graph à (t) is derived based on:
à (t) =λL (0) +(1−λ){η f ( A (t) )+(1−η) f ( A (1) )}
where η is a hyperparameter, λ is a hyperparameter used to balance a trade-off between the updated adjacency matrix A (t) and the initial noisy graph topology designated as A (0) , L (0) is a normalized adjacency matrix of the initial noisy graph topology A (0) , defined as L (0) =D (0) −1/2 A (0) D (0) −1/2 , and where D (0) is a corresponding degree matrix.
16 . The method of claim 1 , wherein the repeating the generating, producing, and feeding back operations continues for a predefined number of iterations.
17 . A non-transitory computer readable medium comprising computer executable instructions which when executed by a computer cause the computer to perform the method of:
implementing a similarity learning component and a graph neural network (GNN);
obtaining an initial noisy graph topology;
generating, using the similarity learning component, an initial adjacency matrix using similarity learning and a similarity metric function;
producing, from said initial adjacency matrix, using the graph neural network (GNN), an updated adjacency matrix with node embeddings;
feeding back the node embeddings to revise the similarity learning component;
repeating the generating, producing, and feeding back operations for a plurality of iterations;
updating weights of the graph neural network only after a convergence of the updated adjacency matrices generated by the plurality of iterations;
performing natural language processing on input from a human subject using a selected iteration of the updated adjacency matrix; and
reconfiguring at least one network-based asset in accordance with a result of the natural language processing.
18 . An apparatus comprising:
a memory; and
at least one processor, coupled to said memory, and operative to perform operations comprising:
implementing a similarity learning component and a graph neural network (GNN);
obtaining an initial noisy graph topology;
generating, using the similarity learning component, an initial adjacency matrix using similarity learning and a similarity metric function;
producing, from said initial adjacency matrix, using the graph neural network (GNN), an updated adjacency matrix with node embeddings;
feeding back the node embeddings to revise the similarity learning component;
repeating the generating, producing, and feeding back operations for a plurality of iterations;
updating weights of the graph neural network only after a convergence of the updated adjacency matrices generated by the plurality of iterations;
performing natural language processing on input from a human subject using a selected iteration of the updated adjacency matrix; and
reconfiguring at least one network-based asset in accordance with a result of the natural language processing.
19 . The apparatus of claim 18 , the operations further comprising implementing a graph regularization component, and controlling, by the graph regularization component, a smoothness, a connectivity and a sparsity of a corresponding graph structure of the updated adjacency matrix using an adapted graph regularization technique.
20 . The apparatus of claim 18 , wherein the repeating the generating, producing, and feeding back operations continues until a corresponding updated adjacency matrix designated as A (t) is sufficiently similar to a corresponding optimized graph for use in prediction based on a threshold ε to derive a final graph à (t) .