Natural language processing with k-NN
A system for natural language processing includes a memory array and a processor. The memory array is divided into a similarity section storing a plurality of feature vectors, a SoftMax section in which to determine probabilities of occurrence of the feature vectors, a value section storing a plurality of modified feature vectors, and a marker section. The processor activates the array to perform parallel operations in each column indicated by the marker section: a similarity operation in the similarity section between a vector question and feature vectors stored in indicated columns; a SoftMax operation in the SoftMax section to determine an associated SoftMax probability value for indicated feature vectors; a multiplication operation in the value section to multiply the associated SoftMax value by modified feature vectors stored in indicated columns; and a vector sum in the value section to accumulate an attention vector of output of the multiplication operation.
1. A system for natural language processing, the system comprising:
a memory array having rows and columns, said memory array being divided into a similarity section initially storing a plurality of feature or key vectors in columns thereof, wherein each of said vectors has a fixed size, a SoftMax section in which to determine probabilities of occurrence of said feature or key vectors, a value section initially storing a plurality of modified feature vectors in columns thereof, and a marker section storing a marker vector in a row thereof specifying columns to be operated upon, wherein said sections being contiguous to one another such that a column of one section of said sections is contiguous with a column of a neighboring section and operations in one or more columns of said memory array are associated with one feature vector to be processed, and wherein said memory array comprises a bit line processor per column of each said section, each said bit line processor operating on one bit of data of its associated section; and
an in-memory processor operating in a constant time as a function of said fixed size and irrespective of the number of said vectors, said processor to activate said memory array to perform the following operations in parallel in each column indicated by said marker vector:
a similarity operation in said similarity section between a vector question and each said feature vector stored in each said indicated column to generate a similarity output in each said indicated column;
a SoftMax operation in said SoftMax section on each said similarity output in said similarity section to determine an associated SoftMax value for each said indicated feature vector, wherein an intermediate output of exponential operations of said SoftMax operation is stored in said bit-line processor of said SoftMax section of each said indicated column; and
a multiplication operation in said value section to multiply each said associated SoftMax value in said SoftMax section by each said modified feature vector stored in each said indicated column to generate a multiplication output in each said indicated column;
said in-memory processor to also perform a horizontal vector sum in said value section of said multiplication output in each said indicated column to accumulate an attention vector sum, said vector sum to be used to generate a new vector question for a further iteration or to generate an output value in a final iteration.
2. The system according to claim 1 wherein said memory array comprises operational portions, one portion per iteration of a natural language processing operation, each portion being divided into said similarity, SoftMax, and value sections.
3. The system according to claim 1 wherein said memory array is one of: an SRAM, a non-volatile, a volatile, and a non-destructive array.
4. The system according to claim 1 and also comprising a neural network feature extractor to generate said feature and modified feature vectors.
5. The system according to claim 1 and wherein said feature vectors comprise features of a word, a sentence, or a document.
6. The system according to claim 1 wherein said feature vectors are the output of a pre-trained neural network.
7. The system according to claim 1 and also comprising a pre-trained neural network to generate an initial vector question.
8. The system according to claim 7 and also comprising a question generator to generate a further question from said initial vector question and said attention vector sum.
9. The system according to claim 8 wherein said question generator is a neural network.
10. The system according to claim 8 and wherein said question generator is implemented as a matrix multiplier on bit lines of said memory array.
11. A method for natural language processing, the method comprising:
having a memory array having rows and columns, said memory array being divided into a similarity section initially storing a plurality of feature or key vectors in columns thereof, wherein each of said vectors has a fixed size, a SoftMax section in which to determine probabilities of occurrence of said feature or key vectors, a value section initially storing a plurality of modified feature vectors in columns thereof, and a marker section storing a marker vector in a row thereof specifying columns to be operated upon, said sections being contiguous to one another such that a column of one section of said sections is contiguous with a column of a neighboring section and operations in one or more columns of said memory array are associated with one feature vector to be processed, and wherein said memory array comprises a bit line processor per column of each said section, each said bit line processor operating on one bit of data of its associated section; and
activating said memory array to operate in a constant time as a function of said fixed size and irrespective of the number of said vectors to perform the following operations in parallel in each column indicated by said marker vector:
performing a similarity operation in said similarity section between a vector question and each said feature vector stored in each said indicated column to generate a similarity output in each said indicated column;
performing a SoftMax operation in said SoftMax section on each said similarity output in said similarity section to determine an associated SoftMax value for each said indicated feature vector, wherein an intermediate output of exponential operations of said SoftMax operation is stored in said bit-line processor of said SoftMax section of each said indicated column; and
performing a multiplication operation in said value section to multiply each said associated SoftMax value in said SoftMax section by each said modified feature vector stored in each said indicated column to generate a multiplication output in each said indicated column;
performing a horizontal vector sum operation in said value section of said multiplication output in each said indicated column to accumulate an attention vector sum, said vector sum to be used to generate a new vector question for a further iteration or to generate an output value in a final iteration.
12. The method according to claim 11 and also comprising generating said feature and modified feature vectors with a neural network and storing them into said similarity and value sections, respectively.
13. The method according to claim 11 and wherein said feature vectors comprise features of a word, a sentence, or a document.
14. The method according to claim 11 and also comprising generating an initial vector question using a pre-trained neural network.
15. The method according to claim 14 and also comprising generating a further question from said initial vector question and said attention vector sum.
16. The method according to claim 15 wherein generating a further question utilizes a neural network.
17. The method according to claim 15 and wherein said generating a further question comprises performing matrix multiplication on bit lines of said memory array.