Deep neural networks and methods for using same
View Patent ↗A method and system for labeling a selected word of a sentence using a deep neural network includes, in one exemplary embodiment, determining an index term corresponding to each feature of the word, transforming the index term or terms of the word into a vector, and predicting a label for the word using the vector. The method and system, in another exemplary embodiment, includes determining, for each word in the sentence, an index term corresponding to each feature of the word, transforming the index term or terms of each word in the sentence into a vector, applying a convolution operation to the vector of the selected word and at least one of the vectors of the other words in the sentence, to transform the vectors into a matrix of vectors, each of the vectors in the matrix including a plurality of row values, constructing a single vector from the vectors in the matrix, and predicting a label for the selected word using the single vector.
1. A method for labeling a selected word of a sentence, the method comprising the steps of:
providing a deep neural network including a first deep layer for extracting and indexing word features of the sentence and extracting and indexing selected predicate-to-selected word distance features that are relevant to a given natural language processing (NLP) task, and a second deep layer for converting feature indices to vectors using one or more look-up tables, the deep neural network being trained using a multiple NLP task learning process, the multiple NLP task learning process sharing look-up tables during training;
determining, with the first layer of the deep neural network in a computer process, an index term for each extracted word feature of the selected word;
transforming, with the second layer of the deep neural network in a computer process, each of the index terms of the selected word into a vector;
determining, with the first layer of the deep neural network in a computer process, an index term for distance data between a selected predicate and the selected word;
transforming, with the second layer of the deep neural network in a computer process, the index term for the distance data into another vector;
concatenating, in a computer process, the vectors to generate a single vector; and
predicting, with another layer of the deep neural network in a computer process, a label for the word using the single vector.
2. The method of claim 1 , wherein the labeling comprises at least two different natural language processing tasks and further comprising the step of performing the determining, transforming and predicting steps for each task simultaneously.
3. A method for labeling a selected word of a sentence, the method comprising the steps of:
providing a deep convolutional neural network including a first deep layer for extracting and indexing word features of the sentence and extracting and indexing selected predicate-to-selected word distance features that are relevant to a given natural language processing (NLP) task, and a second deep layer for converting feature indices to vectors using one or more look-up tables, the deep convolutional neural network being trained using a multiple NLP task learning process, the multiple NLP task learning process sharing look-up tables during training;
determining with the first layer of the deep convolutional neural network in a computer process, for each word in the sentence, an index term for each extracted word feature of the word;
transforming with the second layer of the deep convolutional neural network in a computer process, each of the index terms of each of the words into a vector;
determining, with the first layer of the deep convolutional neural network in a computer process, for each word in the sentence, an index term for distance data between a selected predicate and the word;
transforming, with the second layer of the deep convolutional neural network in a computer process, each of the index terms for the distance data into another vector;
concatenating, in a computer process, the vectors of each of the words to generate a single vector for each of the words;
applying, with another layer of the deep convolutional neural network in a computer process, a convolution operation to the vector of the selected word in the sentence and at least one of the vectors of the other words in the sentence to transform the vectors into a matrix of vectors, each of the vectors in the matrix including a plurality of row values;
constructing, with another layer of the deep convolutional neural network in a computer process, a single vector from the vectors in the matrix; and
predicting, with another layer of the deep convolutional neural network in a computer process, a label for the selected word using the single vector.
4. The method of claim 3 , wherein the labeling comprises at least two different natural language processing tasks and further comprising the step of performing the determining, transforming, applying, constructing and predicting steps for each task simultaneously.
5. A system comprising:
a central processing unit; and
a memory communicating with the central processing unit, the memory comprising instructions executable by the processor for labeling a selected word of a sentence by:
providing a deep neural network including a first deep layer for extracting and indexing word features of the sentence and extracting and indexing selected predicate-to-selected word distance features that are relevant to a given natural language processing (NLP) task, and a second deep layer for converting feature indices to vectors using one or more look-up tables, the deep neural network being trained using a multiple NLP task learning process, the multiple NLP task learning process sharing look-up tables during training;
determining, with the first layer of the deep neural network in a computer process, an index term for each extracted word feature of the selected word;
transforming, with the second layer of the deep neural network in a computer process, each of the index terms of the selected word into a vector;
determining, with the first layer of the deep neural network in a computer process, an index term for distance data between a selected predicate and the selected word;
transforming, with the second layer of the deep neural network in a computer process, the index term for the distance data into another vector;
concatenating, in a computer process, the vectors to generate a single vector; and
predicting, with another layer of the deep neural network in a computer process, a label for the word using the single vector.
6. The system of claim 5 , wherein the labeling comprises at least two different natural language processing tasks and further comprising performing the determining, transforming and predicting steps for each task simultaneously.
7. A system comprising:
a central processing unit; and
a memory communicating with the central processing unit, the memory comprising instructions executable by the processor for labeling a selected word of a sentence by:
providing a deep convolutional neural network including a first deep layer for extracting and indexing word features of the sentence and extracting and indexing selected predicate-to-selected word distance features that are relevant to a given natural language processing (NLP) task, and a second deep layer for converting feature indices to vectors using one or more look-up tables, the deep convolutional neural network being trained using a multiple NLP task learning process, the multiple NLP task learning process sharing look-up tables during training;
determining with the first layer of the deep convolutional neural network in a computer process, for each word in the sentence, an index term for each extracted word feature of the word;
transforming with the second layer of the deep convolutional neural network in a computer process, each of the index terms of each of the words into a vector;
determining, with the first layer of the deep convolutional neural network in a computer process, for each word in the sentence, an index term for distance data between a selected predicate and the word;
transforming, with the second layer of the deep convolutional neural network in a computer process, each of the index terms for the distance data into another vector;
concatenating, in a computer process, the vectors of each of the words to generate a single vector for each of the words;
applying, with another layer of the deep convolutional neural network in a computer process, a convolution operation to the vector of the selected word in the sentence and at least one of the vectors of the other words in the sentence to transform the vectors into a matrix of vectors, each of the vectors in the matrix including a plurality of row values;
constructing, with another layer of the deep convolutional neural network in a computer process, a single vector from the vectors in the matrix; and
predicting, with another layer of the deep convolutional neural network in a computer process, a label for the selected word using the single vector.
8. The system of claim 7 , wherein the labeling comprises at least two different natural language processing tasks and further comprising performing the determining, transforming, applying, constructing and predicting steps for each task simultaneously.