End-to-end memory networks

1. A computing device, comprising:

a feature extraction component including:

a first memory embedding matrix to convert a set of knowledge entries into a first set of memory vectors,

a second memory embedding matrix to convert the set of knowledge entries into a second set of memory vectors, and

an inquiry embedding matrix to convert an inquiry into an input vector;

a memory hop component to conduct one or more hop operations, each individual hop operation includes:

determining an input vector for the individual hop operation,

generating a probability vector based on the input vector and the first set of memory vectors, and

generating an output vector as a weighted combination of the second set of memory vectors, weights of the weighted combination being elements of the probability vector; and

a prediction component to determine a machine-generated response to the inquiry based on the output vector of a last hop operation among the one or more hop operations.

2. The computing device of claim 1 , wherein the set of knowledge entries are a set of sentences, the inquiry is a question seeking an answer based on the sentences, and the machine-generated response is a machine-predicted answer to the question.

3. The computing device of claim 1 , wherein the set of knowledge entries are words within a text sequence, the inquiry is to seek a prediction of a next word for the text sequence, and the machine-generated response is a machine-prediction of the next word.

4. The computing device of claim 1 , further comprising:

a training component to train matrices of the feature extraction component and the prediction component using a training data set including sample knowledge entries, sample inquiry and sample responses.

5. The computing device of claim 4 , wherein the training component needs no supervision on picking knowledge entries or memory vectors that are relevant to the inquiry, during the training of the matrices of the feature extraction component and the prediction component.

6. The computing device of claim 1 , wherein the prediction component determines the machine-generated response to the inquiry, by steps of:

combining the output vector and the input vector of the last hop operation;

feeding the combination of the output vector and the input vector into a prediction matrix; and

calculating a continuous weighting function of a result of the prediction matrix to produce the machine-generated response to the inquiry.

7. The computing device of claim 1 , wherein the individual hop operation generates the probability vector by:

feeding a dot product of the input vector and each memory vector of the first set of memory vectors into a continuous weighting function to produce an element of the probability vector; and

determining the probability vector including the elements produced by the continuous weighting function corresponding to the memory vectors of the first set.

8. The computing device of claim 7 , wherein each element of the probability vector indicates a level of relevancy between the inquiry and a knowledge entry corresponding to the element.

9. The computing device of claim 7 , wherein the continuous weighting function is a softmax function, the softmax function allows back-propagation of training error signal through the softmax function to the inquiry and the knowledge entries during training of the feature extraction component and the prediction component.

10. The computing device of claim 1 , wherein the memory hop component determines the input vector for the individual hop operation by:

in an event that the individual hop operation is a first hop operation, determining that the input vector converted from the inquiry is the input vector for the first hop operation; and

in an event that the individual hop operation is not a first hop operation, determining the input vector for the individual hop operation as a combination of an input vector and an output vector for another hop operation that is immediately prior to the individual hop operation.

11. The computing device of claim 1 , wherein the first memory embedding matrix for the one or more hop operations share the same parameters, and the second memory embedding matrix for the one or more hop operations share the same parameters.

12. The computing device of claim 11 , wherein in an event that the individual hop operation is not a first hop operation, determining the input vector for the individual hop operation as a sum of an output vector of another hop operation and an input vector for the other hop operation updated by a linear mapping matrix, the other hop operation being immediately prior to the individual hop operation.

13. The computing device of claim 1 , wherein instances of the first memory embedding matrix for each of the hop operations have different parameters, instances of the second memory embedding matrix for each of the hop operations have different parameters; and

wherein the individual hop operation further includes:

converting the set of knowledge entries into a first set of memory vectors for the individual hop operation, using the instance of the first memory embedding matrix corresponding to the individual hop operation; and

converting the set of knowledge entries into a second set of memory vectors for the individual hop operation, using the instance of the second memory embedding matrix corresponding to the individual hop operation.

14. The computing device of claim 13 , wherein the instance of the first memory embedding matrix for the individual hop operation share the same parameters as an instance of the second memory embedding matrix for another hop operation that is immediately prior to the individual hop operation.

15. The computing device of claim 14 , wherein

for a first hop operation, the instance of the first memory embedding matrix share the same parameters with the inquiry embedding matrix; and

for a last hop operation, the instance of the second memory embedding matrix share the same parameters with a prediction matrix used by the prediction component to determine the machine-generate response.