Computing numeric representations of words in a high-dimensional space

1. A system comprising:

a classifier implemented in one or more computers, comprising:

an embedding function layer configured to receive an input comprising a plurality of words that surround an unknown word in a sequence of words and map the plurality of words into a numeric representation in a high-dimensional space; and

a classifier layer configured to process the numeric representation of the input to generate a respective word score for each word in a pre-determined set of words, wherein each of the respective word scores represents a predicted likelihood that the corresponding word is the unknown word; and

instructions to process each word in a vocabulary of words using the embedding function layer to obtain a respective numeric representation of each word in the vocabulary in the high-dimensional space and to associate each word in the vocabulary with the respective numeric representation of the word in the high-dimensional space.

2. The system of claim 1 , wherein the numeric representations are continuous representations represented using floating-point numbers.

3. The system of claim 1 , wherein positions of representations in the high-dimensional space reflect semantic similarities, syntactic similarities, or both, between words represented by the representations.

4. The system of claim 1 , wherein the embedding layer maps each of the plurality of words to a respective floating point vector and outputs a single merged vector that is a combination of the respective floating point vectors.

5. A system comprising:

a classifier implemented in one or more computers, comprising:

an embedding function layer configured to receive an input comprising an input word and map the input word into a numeric representation in a high-dimensional space, and

a plurality of classifiers, wherein each of the classifiers corresponds to a respective position surrounding the input word in a sequence of words, and wherein each of the plurality of classifiers is configured to: process the numeric representation of the input word to generate a respective word score for each word in a pre-determined set of words, wherein each of the respective word scores represents a predicted likelihood that the corresponding word will be found in the corresponding position relative to the input word; and

instructions to process each word in a vocabulary of words using the embedding function layer to obtain a respective numeric representation of each word in the vocabulary in the high-dimensional space and to associate each word in the vocabulary with the respective numeric representation of the word in the high-dimensional space.

6. The system of claim 5 , wherein the numeric representations are continuous representations represented using floating-point numbers.

7. The system of claim 5 , wherein positions of representations in the high-dimensional space reflect semantic similarities, syntactic similarities, or both, between words represented by the representations.

8. The system of claim 5 , wherein the embedding layer maps the input word to a floating point vector.

9. A method for assigning a respective point in a high-dimensional space to each word in a vocabulary of words, the method comprising:

obtaining a set of training data, wherein the set of training data comprises sequences of words;

training a plurality of classifiers and an embedding function on the set of training data, wherein the embedding function receives an input word and maps the input word to a numeric representation in the high-dimensional space in accordance with a set of embedding function parameters, wherein each of the classifiers corresponds to a respective position surrounding the input word in a sequence of words, and wherein each of the classifiers processes the numeric representation of the input word to generate a respective word score for each word in a pre-determined set of words, wherein each of the respective word scores represents a predicted likelihood that the corresponding word will be found in the corresponding position relative to the input word, and wherein training the embedding function comprises obtaining trained values of the embedding function parameters;

processing each word in the vocabulary using the embedding function in accordance with the trained values of the embedding function parameters to generate a respective numerical representation of each word in the vocabulary; and

associating each word in the vocabulary with the respective numeric representation of the word in the high-dimensional space.

10. The method of claim 9 , wherein the numeric representations are continuous representations represented using floating-point numbers.

11. The method of claim 9 , wherein positions of representations in the high-dimensional space reflect semantic similarities, syntactic similarities, or both, between words represented by the representations.

12. The method of claim 9 , wherein the embedding function maps the input word to a floating point vector.

13. The method of claim 9 , wherein training the plurality of classifiers and the embedding function comprises preforming a backpropagation training technique to obtain the trained values of the embedding function parameters.

14. A method for assigning points in a high dimensional space to each word in a vocabulary of words, the method comprising:

obtaining a set of training data, wherein the set of training data comprises sequences of words;

training a classifier and an embedding function on the set of training data, wherein the embedding function receives a plurality of words surrounding an unknown word in a sequence of words and maps the plurality of words into a numeric representation in accordance with a set of embedding function parameters, wherein the classifier processes the numeric representation of the sequence of words to generate a respective word score for each word in a pre-determined set of words, and wherein each of the respective word scores measure a predicted likelihood that the corresponding word is the unknown word, and wherein training the embedding function comprises obtained trained values of the embedding function parameters;

processing each word in the vocabulary using the embedding function in accordance with the trained values of the embedding function parameters to generate a respective numerical representation of each word in the vocabulary in the high-dimensional space; and

associating each word in the vocabulary with the respective numeric representation of the word in the high-dimensional space.

15. The method of claim 14 , wherein the numeric representations are continuous representations represented using floating-point numbers.

16. The method of claim 14 , wherein positions of representations in the high-dimensional space reflect semantic similarities, syntactic similarities, or both, between words represented by the representations.

17. The method of claim 14 , wherein the embedding function maps each of the plurality of words to a respective floating point vector and outputs a single merged vector that is a combination of the respective floating point vectors.

18. The method of claim 14 , wherein training the classifier and the embedding function comprises preforming a backpropagation training technique to obtain the trained values of the embedding function parameters.