System and method for molecular property prediction using edge-conditioned graph attention neural network
This disclosure relates generally to a system and method for molecular property prediction. The conventional methods for molecular property prediction suffer from inherent limitation to effectively encapsulate the characteristics of the molecular graph. Moreover, the known methods are computationally intensive, thereby leading to non-performance in real-time scenarios. The disclosed method includes performing self-attention on the nodes of a molecular graph of different sized neighborhood, and further performing a shared attention mechanism across the nodes of the molecular graphs to compute attention coefficients using and Edge-condition graph attention neural network (EC-GAT). The EC-GAT effectively utilizes the edge characteristics in the molecular graph for molecular property prediction.
1 . A processor-implemented method for molecular property prediction using edge-conditioned graph attention neural networks, comprising:
accessing, via one or more hardware processors, a database comprising a plurality of molecular graphs associated with a plurality of molecules and a plurality of labels indicative of chemical properties of the plurality of molecular graphs, wherein each molecular graph of the plurality of molecular graphs comprises a plurality of sink nodes, each sink node of the plurality of sink nodes connected to a plurality of source nodes for passing neural messages through a plurality of edges, wherein the edge-conditioned graph attention neural networks allow stacking of layers in which each node in a molecular graph aggregate information over local-graph neighbors attributes based on molecular graph's connectivity and adaptively adjust to weight nodes of importance;
computing, via the one or more hardware processors, attention coefficients (α i,j ) indicative of influence of the plurality of source nodes (j) on a sink node (i) of the plurality of sink nodes based on (1) a softmax output of a matrix product of a learnable attention vector (α) and a concatenated matrix (Θ 2 ,h i ) of a linearly transformed hidden state of the sink node (i), and (2) a matrix product of a linearly transformed edge-information ( ) associated with an edge connecting a source node from amongst the plurality of source nodes (j) with the sink node (i), and a relative difference of hidden state vectors of the source node and the sink node by a learnable weight matrix as
α
i
,
j
=
exp
(
LeakyReLU
(
a
⊤
[
Θ
2
h
i
e
j
,
i
ℳ
·
Θ
2
(
h
i
-
h
j
)
]
)
)
∑
k
∈
𝒩
(
i
)
⋃
{
i
}
exp
(
LeakyReLU
(
a
⊤
[
Θ
2
h
i
e
k
,
i
ℳ
·
Θ
2
(
h
i
-
h
k
)
]
)
)
LeakyReLU is a non-linear activation function,
(i) represents a plurality of neighboring source nodes perceived by the sink node (i),
α is a learnable attention vector,
Θ 2 h i is concatenated matrix of the linearly transformed hidden state of the sink node (i)
is the edge information of the edge connecting the source node (i) with the sink node (i),
· T denotes transposition, and ∥ represents concatenation,
(h i −h j ) is linearly transformed, relative difference of the hidden state of the source node and the sink node,
Θ 2 is a learnable weight matrix,
wherein the edge-conditioned graph attention neural networks perform self-attention on the nodes of the molecular graph of varied sized neighborhood and involve a shared attentional mechanism across the nodes of the molecular graphs to compute attention coefficients, wherein edge-conditioned graph attention neural networks is agnostic to selection of attention mechanism and operates on variable-sized graph structured inputs;
computing, via the one or more hardware processors, a plurality of neural messages sent from the plurality of source nodes to the sink node as a product of summation of a set of hidden state vectors of the plurality of source nodes and the linearly transformed edge-information associated with the edge connecting the source node from amongst the plurality of source nodes with the sink node;
determining, via the one or more hardware processors, a weighted sum of the plurality of neural messages from the plurality of source nodes perceived by the sink node to obtain an aggregated single-message vector from the plurality of source nodes, the weighted sum determined through the linearly transformed edge-information using the attention coefficients;
transforming, via the one or more hardware processors, the hidden state vector of the sink node to obtain a node level embedding of the molecular graph, wherein transforming the hidden state vector of the sink node comprises computing a summation of (1) the linearly transformed hidden state vector of the sink node parameterized by the product of the attention coefficients and the learnable weight parameter, and (2) the aggregated single-message vector from the plurality of source nodes;
determining, via the one or more hardware processors, graph level embedding of the molecular graph using a read-out function from the node level embedding of the molecular graph, wherein the read-out function takes node attributes transformed molecular graph as an input and computes a graph-level representation; and
computing, via the one or more hardware processors, the one or more molecular properties by feeding the graph level embedding of the molecular graph to a linear layer, wherein the edge-conditioned graph attention neural networks is resilient to noise and adapt on a task-relevant fragment of the molecular graphs at varying receptive fields, locality, depth to augment discriminative power of node and graph-level embeddings, the learned discriminative node and the graph-level embeddings aids in the molecular property prediction with a reduced computational complexity,
wherein the edge-conditioned graph attention neural networks provide a supervised learning on drug-like potential molecules for use in effective drug discovery and provide substantial prospects in diminishing the computational complexity which is a key desideratum for prognostication of molecular properties and aid in developing a potential drug for treatment.
2 . The method of claim 1 , further comprising:
operating on a molecular graph, a spatial-dynamic neighborhood aggregation-based message passing phase to exchange the plurality of neural messages resulting in transformation and updating of each node level embedding; and
performing statistical average of the corresponding node level embedding obtained from the edge-conditioned graph attention neural network and the spatial-dynamic neighborhood aggregation-based message passing phase.
3 . The method of claim 1 , further comprising:
operating on a molecular graph, a spatial Identity Mapping graph Convolution Networks based message passing phase to exchange the plurality of neural messages resulting in transformation and updating of each node level embedding; and
performing statistical average of the corresponding node level embedding obtained from the edge-conditioned graph attention neural network and the spatial Identity Mapping graph Convolution Networks based message passing phase.
4 . The method of claim 1 , further comprising:
operating on a molecular graph, a spatial-graph attentional propagation based message passing phase to exchange the plurality of neural messages resulting in transformation and updating of each node level embedding; and
performing statistical average of the corresponding node level embedding obtained from the edge-conditioned graph attention neural network and the spatial-graph attentional propagation based message passing phase.
5 . The method of claim 1 , wherein the edge information associated with the edge of the molecular graph comprises edge-type characteristics of the edge and a spatial distance between atoms of the molecular graph that are represented as nodes.
6 . A system for molecular property prediction using edge-conditioned graph attention neural networks, comprising:
a memory storing instructions;
one or more communication interfaces; and
one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
access a database comprising a plurality of molecular graphs associated with a plurality of molecules and a plurality of labels indicative of chemical properties of the plurality of molecular graphs, wherein each molecular graph of the plurality of molecular graphs comprises a plurality of sink nodes, each sink node of the plurality of sink nodes connected to a plurality of source nodes for passing neural messages through a plurality of edges, wherein the edge-conditioned graph attention neural networks allow stacking of layers in which each node in a molecular graph aggregate information over local-graph neighbors attributes based on molecular graph's connectivity and adaptively adjust to weight nodes of importance;
compute attention coefficients (α i,j ) indicative of influence of the plurality of source nodes (j) on a sink node (i) of the plurality of sink nodes based on (1) a softmax output of a matrix product of a learnable attention vector (α) and a concatenated matrix (Θ 2 h i ) of a linearly transformed hidden state of the sink node (i) and (2) a matrix product of a linearly transformed edge-information ( ) associated with an edge connecting a source node from amongst the plurality of source nodes (j) with the sink node (i), relative difference of hidden state vectors of the source node and the sink node by a learnable weight matrix as
α
i
,
j
=
exp
(
LeakyReLU
(
a
⊤
[
Θ
2
h
i
e
j
,
i
ℳ
·
Θ
2
(
h
i
-
h
j
)
]
)
)
∑
k
∈
𝒩
(
i
)
⋃
{
i
}
exp
(
LeakyReLU
(
a
⊤
[
Θ
2
h
i
e
k
,
i
ℳ
·
Θ
2
(
h
i
-
h
k
)
]
)
)
LeakyReLU is a non-linear activation function,
(i) represents a plurality of neighboring source nodes perceived by the sink node (i),
α is a learnable attention vector,
Θ 2 h i is concatenated matrix of the linearly transformed hidden state of the sink node (i),
is the edge information of the edge connecting the source node (j) with the sink node (i),
· T denotes transposition, and ∥ represents concatenation,
(h i -h j ) is linearly transformed, relative difference of the hidden state of the source node and the sink node,
Θ 2 is a learnable weight matrix,
wherein the edge-conditioned graph attention neural networks perform self-attention on the nodes of the molecular graph of varied sized neighborhood and involve a shared attentional mechanism across the nodes of the molecular graphs to compute attention coefficients, wherein edge-conditioned graph attention neural networks is agnostic to selection of attention mechanism and operates on variable-sized graph structured inputs;
compute a plurality of neural messages sent from the plurality of source nodes to the sink node as a product of a summation of a set of hidden state vectors of the plurality of source nodes and the linearly transformed edge-information associated with the edge connecting the source node from amongst the plurality of source nodes with the sink node;
determine a weighted sum of the plurality of neural messages from the plurality of source nodes perceived by the sink node to obtain an aggregated single-message vector from the plurality of source nodes, the weighted sum determined through the linearly transformed edge-information using the attention coefficients;
transform the hidden state vector of the sink node to obtain a node level embedding of the molecular graph, wherein to transform the hidden state vector of the sink node, the one or more hardware processors are configured by the instructions to compute a summation of (1) the linearly transformed hidden state vector of the sink node parameterized by the product of the attention coefficients and the learnable weight parameter, and (2) the aggregated single-message vector from the plurality of source nodes;
determine a graph level embedding of the molecular graph using a read-out function from the node level embedding of the molecular graph, wherein the read-out function takes node attributes transformed molecular graph as an input and computes a graph-level representation; and
compute the one or more molecular properties by feeding the graph level embedding of the molecular graph to a linear layer, wherein the edge-conditioned graph attention neural networks is resilient to noise and adapt on a task-relevant fragment of the molecular graphs at varying receptive fields, locality, depth to augment discriminative power of node and graph-level embeddings, the learned discriminative node and the graph-level embeddings aids in the molecular property prediction with a reduced computational complexity,
wherein the edge-conditioned graph attention neural networks provide a supervised learning on drug-like potential molecules for use in effective drug discovery and provide substantial prospects in diminishing the computational complexity which is a key desideratum for prognostication of molecular properties and aid in developing a potential drug for treatment.
7 . The system of claim 6 , wherein the one or more hardware processors are further configured by the instructions to:
operate on a molecular graph, a spatial-dynamic neighborhood aggregation-based message passing phase to exchange the plurality of neural messages resulting in transformation and updating of each node level embedding; and
perform statistical average of the corresponding node level embedding obtained from the edge-conditioned graph attention neural network and the spatial-dynamic neighborhood aggregation-based message passing phase.
8 . The system of claim 6 , wherein the one or more hardware processors are further configured by the instructions to:
operate on a molecular graph, a spatial Identity Mapping graph Convolution Networks based message passing phase to exchange the plurality of neural messages resulting in transformation and updating of each node level embedding; and
perform statistical average of the corresponding node level embedding obtained from the] edge-conditioned graph attention neural network and the spatial Identity Mapping graph Convolution Networks based message passing phase.
9 . The system of claim 6 , wherein the one or more hardware processors are further configured by the instructions to:
operate on a molecular graph, a spatial-graph attentional propagation based message passing phase to exchange the plurality of neural messages resulting in transformation and updating of each node level embedding; and
perform statistical average of the corresponding node level embedding obtained from the edge-conditioned graph attention neural network and the spatial-graph attentional propagation based message passing phase.
10 . The system of claim 6 , wherein the edge information associated with the edge of the] molecular graph comprises edge-type characteristics of the edge and a spatial distance between atoms of the molecular graph that are represented as nodes.
11 . One or more non-transitory machine-readable information storage mediums comprising one or more instructions for molecular property prediction using edge-conditioned graph attention neural networks which when executed by one or more hardware processors cause:
accessing a database comprising a plurality of molecular graphs associated with a plurality of molecules and a plurality of labels indicative of chemical properties of the plurality of molecular graphs, wherein each molecular graph of the plurality of molecular graphs comprises a plurality of sink nodes, each sink node of the plurality of sink nodes connected to a plurality of source nodes for passing neural messages through a plurality of edges, wherein the edge-conditioned graph attention neural networks allow stacking of layers in which each node in a molecular graph aggregate information over local-graph neighbors attributes based on molecular graph's connectivity and adaptively adjust to weight nodes of importance;
computing attention coefficients (α i,j ) indicative of influence of the plurality of source nodes (j) on a sink node (i) of the plurality of sink nodes based on (1) a softmax output of a matrix product of a learnable attention vector and a concatenated matrix (Θ 2 h i ) of a linearly transformed hidden state of the sink node (i), and (2) a matrix product of a linearly transformed edge-information ( ) associated with an edge connecting a source node from amongst the plurality of source nodes (j) with the sink node (i), and a relative difference of hidden state vectors of the source node and the sink node by a learnable weight matrix as
α
i
,
j
=
exp
(
LeakyReLU
(
a
⊤
[
Θ
2
h
i
e
j
,
i
ℳ
·
Θ
2
(
h
i
-
h
j
)
]
)
)
∑
k
∈
𝒩
(
i
)
⋃
{
i
}
exp
(
LeakyReLU
(
a
⊤
[
Θ
2
h
i
e
k
,
i
ℳ
·
Θ
2
(
h
i
-
h
k
)
]
)
)
LeakyReLU is a non-linear activation function,
(i) represents a plurality of neighboring source nodes perceived by the sink node (i),
α is a learnable attention vector,
Θ 2 h i is concatenated matrix of the linearly transformed hidden state of the sink node (i)
is the edge information of the edge connecting the source node (j) with the sink node (i),
· T denotes transposition, and ∥ represents concatenation,
(h i −h j ) is linearly transformed, relative difference of the hidden state of the source node and the sink node,
Θ 2 is a learnable weight matrix,
wherein the edge-conditioned graph attention neural networks perform self-attention on the nodes of the molecular graph of varied sized neighborhood and involve a shared attentional mechanism across the nodes of the molecular graphs to compute attention coefficients, wherein edge-conditioned graph attention neural networks is agnostic to selection of attention mechanism and operates on variable-sized graph structured inputs;
computing a plurality of neural messages sent from the plurality of source nodes to the sink node as a product of summation of a set of hidden state vectors of the plurality of source nodes and the linearly transformed edge-information associated with the edge connecting the source node from amongst the plurality of source nodes with the sink node;
determining a weighted sum of the plurality of neural messages from the plurality of source nodes perceived by the sink node to obtain an aggregated single-message vector from the plurality of source nodes, the weighted sum determined through the linearly transformed edge-information using the attention coefficients;
transforming the hidden state vector of the sink node to obtain a node level embedding of the molecular graph, wherein transforming the hidden state vector of the sink node comprises computing a summation of (1) the linearly transformed hidden state vector of the sink node parameterized by the product of the attention coefficients and the learnable weight parameter, and (2) the aggregated single-message vector from the plurality of source nodes;
determining graph level embedding of the molecular graph using a read-out function from the node level embedding of the molecular graph, wherein the read-out function takes node attributes transformed molecular graph as an input and computes a graph-level representation; and
computing the one or more molecular properties by feeding the graph level embedding of the molecular graph to a linear layer, wherein the edge-conditioned graph attention neural networks is resilient to noise and adapt on a task-relevant fragment of the molecular graphs at varying receptive fields, locality, depth to augment discriminative power of node and graph-level embeddings, the learned discriminative node and the graph-level embeddings aids in the molecular property prediction with a reduced computational complexity,
wherein the edge-conditioned graph attention neural networks provide a supervised learning on drug-like potential molecules for use in effective drug discovery and provide substantial prospects in diminishing the computational complexity which is a key desideratum for prognostication of molecular properties and aid in developing a potential drug for treatment.
12 . The one or more non-transitory machine readable information storage mediums of claim 11 , further comprising:
operating on a molecular graph, a spatial-dynamic neighborhood aggregation-based message passing phase to exchange the plurality of neural messages resulting in transformation and updating of each node level embedding; and
performing statistical average of the corresponding node level embedding obtained from the edge-conditioned graph attention neural network and the spatial-dynamic neighborhood aggregation-based message passing phase.
13 . The one or more non-transitory machine readable information storage mediums of claim 11 , further comprising:
operating on a molecular graph, a spatial Identity Mapping graph Convolution Networks based message passing phase to exchange the plurality of neural messages resulting in transformation and updating of each node level embedding; and
performing statistical average of the corresponding node level embedding obtained from the edge-conditioned graph attention neural network and the spatial Identity Mapping graph Convolution Networks based message passing phase.
14 . The one or more non-transitory machine readable information storage mediums of claim 11 , further comprising:
operating on a molecular graph, a spatial-graph attentional propagation based message passing phase to exchange the plurality of neural messages resulting in transformation and updating of each node level embedding; and
performing statistical average of the corresponding node level embedding obtained from the edge-conditioned graph attention neural network and the spatial-graph attentional propagation based message passing phase.
15 . The one or more non-transitory machine readable information storage mediums of claim 11 , wherein the edge information associated with the edge of the molecular graph comprises edge-type characteristics of the edge and a spatial distance between atoms of the molecular graph that are represented as nodes.