Information-aware graph contrastive learning
A method for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework is presented. The method includes obtaining two semantically similar views of a graph coupled with a label for training by employing a view augmentation component, feeding the two semantically similar views into respective encoder networks to extract latent representations preserving both structure and attribute information in the two views, optimizing a contrastive loss based on a contrastive mode by maximizing feature consistency between the latent representations, training a neural network with the optimized contrastive loss, and predicting a new graph label or a new node label in the graph.
1 . A method for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework, the method comprising (lump with selecting a contrastive mode):
obtaining two semantically similar views of a graph coupled with a label for training by employing a view augmentation component to generate views of the graph that do not affect a semantic label of the graph, wherein the graph represents a citation network indicating citation between technical papers and includes nodes each indicating the technical papers, and the labels indicates a research area;
feeding the two semantically similar views into respective separate instances of encoder networks to extract latent representations preserving both structure and attribute information in the two semantically similar views;
selecting a contrastive mode that is characterized by an aggregation operation by solving an optimization problem:
(
c
i
*
,
c
j
*
)
=
arg
min
(
c
i
,
c
j
)
-
I
(
c
i
(
z
i
*
)
;
c
j
(
z
j
*
)
)
where c*i and c*j are aggregation operations applied to latent representations z*i and z*j from the respective separate instances of encoder networks and where I(.;.) is a mutual information;
minimizing a contrastive loss based on the contrastive mode by maximizing feature consistency between the latent representations; and
training a document classification model by performing graph contrastive learning with the minimized contrastive loss, the trained document classification model being configured to predict a new node label of a new technical paper based on citation information and textual content of the new technical paper, wherein the predicted new node label is used to support a decision making process.
2 . The method of claim 1 , comprising
predicting the new label of the new technical paper based on the citation information and the textual content of the new technical paper using the document classification model.
3 . The method of claim 1 , wherein each of the separate instances of encoder networks includes a graph neural network (GNN) backbone and a projection multilayer perceptron (MLP).
4 . The method of claim 1 , wherein the contrastive mode is selected from the group consisting of a global-global mode, a local-global mode, a local-local mode, a multi-scale mode, and a hybrid-mode.
5 . The method of claim 4 , wherein, in the global-global mode, graph representations from the two semantically similar views are contrasted.
6 . The method of claim 4 , wherein, in the local-global mode, node representations from the first view of the two semantically similar views are contrasted with graph representations from the second view of the two semantically similar views.
7 . The method of claim 4 , wherein, in the local-local mode, node representations from the two semantically similar views are contrasted.
8 . The method of claim 1 , wherein the new node label is classified as either positive or negative.
9 . A non-transitory computer-readable storage medium comprising a computer-readable program for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework, wherein the computer-readable program when executed on a computer causes the computer to perform the steps of:
obtaining two semantically similar views of a graph coupled with a label for training by employing a view augmentation component to generate views of the graph that do not affect a semantic label of the graph, wherein the graph represents a citation network indicating citation between technical papers and includes nodes each indicating the technical papers, and the labels indicates a research area;
feeding the two semantically similar views into respective separate instances of encoder networks to extract latent representations preserving both structure and attribute information in the two semantically similar views;
selecting a contrastive mode that is characterized by an aggregation operation by solving an optimization problem:
(
c
i
*
,
c
j
*
)
=
arg
min
(
c
i
,
c
j
)
-
I
(
c
i
(
z
i
*
)
;
c
j
(
z
j
*
)
)
where c*i and c*j are aggregation operations applied to latent representations z*i and z*j from the respective separate instances of encoder networks and where I(.;.) is a mutual information;
minimizing a contrastive loss based on the contrastive mode by maximizing feature consistency between the latent representations; and
training a document classification model by performing graph contrastive learning with the minimized contrastive loss, the trained document classification model being configured to predict a new node label of a new technical paper based on citation information and textual content of the new technical paper, wherein the predicted new node label is used to support a decision making process.
10 . A system for performing contrastive learning for graph tasks and datasets by employing an information-aware graph contrastive learning framework, the system comprising:
a memory; and one or more processors in communication with the memory configured to:
obtain two semantically similar views of a graph coupled with a label for training by employing a view augmentation component, wherein the graph represents a citation network indicating citation between technical papers and includes nodes each indicating the technical papers, and the labels indicates a research area;
feed the two semantically similar views into respective separate instances of encoder networks to extract latent representations preserving both structure and attribute information in the two semantically similar views;
select a contrastive mode that is characterized by an aggregation operation by solving an optimization problem:
(
c
i
*
,
c
j
*
)
=
arg
min
(
c
i
,
c
j
)
-
I
(
c
i
(
z
i
*
)
;
c
j
(
z
j
*
)
)
where c*i and c*j are aggregation operations applied to latent representations z*i and z*j from the respective separate instances of encoder networks and where I(.;.) is a mutual information;
minimize a contrastive loss based on the contrastive mode by maximizing feature consistency between the latent representations; and
train a document classification model by performing graph contrastive learning with the minimized contrastive loss, the trained document classification model being configured to predict a new node label of a new technical paper based on citation information and textual content of the new technical paper, wherein the predicted new node label is used to support a decision making process.