Information-aware graph contrastive learning
A method for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework is presented. The method includes obtaining two semantically similar views of a graph coupled with a label for training by employing a view augmentation component, feeding the two semantically similar views into respective encoder networks to extract latent representations preserving both structure and attribute information in the two views, optimizing a contrastive loss based on a contrastive mode by maximizing feature consistency between the latent representations, training a neural network with the optimized contrastive loss, and predicting a new graph label or a new node label in the graph.
1 . A method for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework, the method comprising:
obtaining two semantically similar views of a graph coupled with a label for training
by employing a view augmentation component to generate views of the graph that do not affect a semantic label of the graph;
selecting a contrastive mode that is characterized by an aggregation operation by solving an optimization problem:
(
c
i
*
,
c
j
*
)
=
arg
min
(
c
i
,
c
j
)
-
I
(
c
i
(
z
i
*
)
;
c
j
(
z
j
*
)
)
where c*i and c*j are aggregation operations applied to latent representations z*i and z*j from respective separate instances of encoder networks and where I(.;.) is a mutual information;
minimizing a contrastive loss based on the contrastive mode by maximizing feature consistency between the latent representations; and
training a document classification model by performing graph contrastive learning with the minimized contrastive loss;
predicting a new node label of a new technical paper based on citation information and textual content of the new technical paper using the document classification model trained by performing graph contrastive learning, the graph representing a citation network indicating citation between technical papers with nodes each indicating the technical papers, the label indicating a research area, the document classification model being trained in such a way that the document classification model predicts the new node label of the new technical paper based on the citation information and the textual content of the new technical paper, wherein the predicted new node label is used to support a decision making process.
2 . The method of claim 1 , further comprising: feeding the two semantically similar views into the respective separate instances of encoder networks to extract latent representations preserving both structure and attribute information in the two semantically similar views.
3 . The method of claim 2 , wherein each of the respective separate instances of encoder networks includes a graph neural network (GNN) backbone and a projection multilayer perceptron (MLP).
4 . The method of claim 1 , wherein the contrastive mode is selected from the group consisting of a global-global mode, a local-global mode, a local-local mode, a multi-scale mode, and a hybrid-mode.
5 . The method of claim 4 , wherein, in the global-global mode, graph representations from the two semantically similar views are contrasted.
6 . The method of claim 4 , wherein, in the local-global mode, node representations from the first view of the two semantically similar views are contrasted with graph representations from the second view of the two semantically similar views.
7 . The method of claim 4 , wherein, in the local-local mode, node representations from the two semantically similar views are contrasted.
8 . The method of claim 1 , wherein the new node label is classified as either positive or negative.
9 . A non-transitory computer-readable storage medium comprising a computer-readable program for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework, wherein the computer-readable program when executed on a computer causes the computer to perform the steps of:
obtaining two semantically similar views of a graph coupled with a label for training
by employing a view augmentation component to generate views of the graph that do not affect a semantic label of the graph;
selecting a contrastive mode that is characterized by an aggregation operation by solving an optimization problem:
(
c
i
*
,
c
j
*
)
=
arg
min
(
c
i
,
c
j
)
-
I
(
c
i
(
z
i
*
)
;
c
j
(
z
j
*
)
)
where c*i and c*j are aggregation operations applied to latent representations z*i and z*j from respective separate instances of encoder networks and where I(.;.) is a mutual information;
minimizing a contrastive loss based on the contrastive mode by maximizing feature consistency between the latent representations; and
training a document classification model by performing graph contrastive learning with the minimized contrastive loss;
predicting a new node label of a new technical paper based on citation information and textual content of the new technical paper using the document classification model trained by performing graph contrastive learning, the graph representing a citation network indicating citation between technical papers with nodes each indicating the technical papers, the label indicating a research area, the document classification model being trained in such a way that the document classification model predicts the new node label of the new technical paper based on the citation information and the textual content of the new technical paper, wherein the predicted new node label is used to support a decision making process.
10 . A system for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework, the system comprising: a memory; and one or more processors in communication with the memory configured to:
obtain two semantically similar views of a graph coupled with a label for training by employing a view augmentation component to generate views of the graph that do not affect a semantic label of the graph;
select a contrastive mode that is characterized by an aggregation operation by solving an optimization problem:
(
c
i
*
,
c
j
*
)
=
arg
min
(
c
i
,
c
j
)
-
I
(
c
i
(
z
i
*
)
;
c
j
(
z
j
*
)
)
where c*i and c*j are aggregation operations applied to latent representations z*i and z*j from respective separate instances of encoder networks and where I(.;.) is a mutual information;
minimize a contrastive loss based on the contrastive mode by maximizing feature consistency between the latent representations; and
train a document classification model by performing graph contrastive learning with the minimized contrastive loss;
predict a new node label of a new technical paper based on citation information and textual content of the new technical paper using the document classification model trained by performing graph contrastive learning, the graph representing a citation network indicating citation between technical papers with nodes each indicating the technical papers, the label indicating a research area, the document classification model being trained in such a way that the document classification model predicts the new node label of the new technical paper based on the citation information and the textual content of the new technical paper, wherein the predicted new node label is used to support a decision making process.