Method and system for extracting information from graphs

1. A method of training a model, said model being adapted to map a first graphical data structure representation to a second graphical data structure representation, the first graphical data structure representation comprising nodes, with at least one of a plurality of different types of relations between said nodes, the second graphical data structure representation comprising nodes, the mapping comprises using an attention mechanism, wherein the mapping comprises using a projection kernel to map the nodes of the first graphical structure to nodes of an intermediate representation, wherein the intermediate representation is produced for each node and each type of relation and using an attention kernel to enact the attention mechanism on the intermediate representation, wherein said attention mechanism establishes the importance of specific nodes dependent on their neighbourhood and the different types of relations between the nodes,

the method comprising:

receiving a training data set comprising an output layer and a corresponding input layer; and

training the parameters of the projection kernel and the attention kernel using the training data set.

2. The method of claim 1 , wherein the attention mechanism is enacted by attention coefficients and the attention coefficients across a neighbourhood are normalised.

3. The method of claim 1 , wherein the attention mechanism is enacted by attention coefficients and the attention coefficients across a neighbourhood of nodes for one relation are normalised.

4. The method of claim 1 , wherein the method further comprises using dropout to train parameters of the projection kernel and the attention kernel.

5. The method of claim 4 , wherein two dropout stages are used, one to dropout features from the nodes of the intermediate representation prior to the introduction of the attention mechanism and the second to dropout features from the nodes of the intermediate representation prior to the application of the attention kernel.

6. The method of claim 1 , wherein the training comprises determining a loss function, wherein said loss function comprises the addition of an error determined from the prediction loss and the L2 norm of the attention kernel and the projection kernel.

7. The method of claim 1 , wherein the training comprises using batch normalisation.

8. The method of claim 7 , wherein two batch normalisation stages are used, wherein the first stage is after the attention mechanism has been applied to the intermediate node representation and the second stage is after the application of the attention kernel.

9. The method of claim 1 , wherein the nodes of the first graphical data structure representation each comprise a feature vector and the nodes of the second representation also each comprise a feature vector, the mapping converting the features of the feature vectors of the first graphical data structure representation into transformed feature vectors of the second graphical data structure representation.

10. The method of claim 9 , wherein the feature vectors of the first graphical data structure representation are transformed into feature vectors of the second graphical data structure representation via a linear transformation and an attention mechanism.

11. The method of claim 10 , wherein the attention mechanism is configured such that after the linear transformation, the relation between the linearly transformed feature vectors are independent of other relations.

12. The method of claim 11 , wherein the attention mechanism comprises attention coefficients derived from a product of a relation dependent vector with a concatenation of two related nodes where the feature vectors from the two related nodes have been transformed via said linear transformation.

13. The method of claim 12 , wherein the attention coefficients are derived from the said product using an activation function that allows normalisation over the neighbourhood of nodes.

14. The method of claim 10 , wherein matrices that define the linear transformation and the attention mechanism are decomposed.

15. The method of claim 1 , wherein the attention mechanism is a multi-head attention mechanism.

16. The method of claim 1 , wherein the attention mechanism is enacted by attention coefficients and there is a coupling between attention coefficients across different relations.

17. A system for training a model, said model for mapping a first graphical data structure representation to a second graphical data structure representation, the system comprising a processor and a memory, the processor being configured to:

receive a first graphical data structure representation comprising nodes, with at least one of a plurality of different types of relations between said nodes,

retrieve from memory a pre-trained model adapted to map said first graphical data structure representation to a second graphical data structure representation, wherein the pre-trained model uses an attention mechanism, wherein said attention mechanism establishes the importance of specific nodes dependent on their neighbourhood and the different types of relations between the nodes; and

map the first graphical data structure representation to a the second graphical data structure representation,

wherein the mapping comprises using a projection kernel to map the nodes of the first graphical structure to nodes of an intermediate representation, wherein the intermediate representation is produced for each node and each type of relation and using an attention kernel to enact the attention mechanism, on the intermediate representation, wherein said attention mechanism establishes the importance of specific nodes dependent on their neighbourhood and the different types of relations between the nodes,

the processor being further adapted to:

receive a training data set comprising an output layer of the second graphical structure and a corresponding input layer of the first graphical structure; and

train the parameters of the projection kernel and the attention kernel using the training data set.

18. The system of claim 17 , wherein the first processor comprises a GPU.

19. A non-transitory computer readable medium carrying computer readable instructions which when executed on a computer cause the computer to perform a method of training a model, said model being adapted to map a first graphical data structure representation to a second graphical data structure representation, the first graphical data structure representation comprising nodes, with at least one of a plurality of different types of relations between said nodes, the second graphical data structure representation comprising nodes, the mapping comprises using an attention mechanism, wherein the mapping comprising using a projection kernel to map the nodes of the first graphical structure to nodes of an intermediate representation wherein the intermediate representation is produced for each node and each type of relation and using an attention kernel to enact the attention mechanism on the intermediate representation, wherein said attention mechanism establishes the importance of specific nodes dependent on their neighbourhood and the different types of relations between the nodes,

the method comprising:

receiving a training data set comprising an output layer and a corresponding input layer; and

training the parameters of the projection kernel and the attention kernel using the training data set.