Intelligent short text information retrieve based on deep learning
Text based searching can return results based on the system determining the searched text includes keywords or search terms. The present solution can return results based on a semantic analysis. The solutions described herein can provide high accuracy compared against the full-text or keyword-based retrieval algorithms. The solution can sort the results by semantic relevance based on the user's input search request. The present solution can provide meaningful results to the user even when the search text does not include the exact search keywords or phrases entered by the user.
1. A method to retrieve content based on text input, comprising:
receiving, by a data processing system, a request comprising a plurality of terms;
determining, by a vector generator executed by the data processing system, an average of a plurality of word vectors, the plurality of word vectors including a word vector retrieved for each term of the plurality of terms of the request, the plurality of word vectors generated by multiplying an encoded vector for a respective term of the plurality of terms by a matrix of weights provided by at least one intermediate layer of a neural network;
generating, by the vector generator using the average of the plurality of word vectors, a sentence vector to map the request to a first vector space;
retrieving, from a database by the vector generator, a plurality of trained sentence vectors corresponding to a plurality of candidate electronic documents, wherein each of the plurality of trained sentence vectors map a respective sentence of each of the plurality of candidate electronic documents to the first vector space;
determining, by a scoring engine executed by the data processing system, a distance in the first vector space between the sentence vector and each trained sentence vector of the plurality of trained sentence vectors;
generating, by the scoring engine, a similarity score for each of the plurality of trained sentence vectors based on the respective one of the plurality of trained sentence vectors and the sentence vector and the distance in the first vector space between the sentence vector and each trained sentence vector of the plurality of trained sentence vectors;
selecting, by the scoring engine, an electronic document from the plurality of candidate electronic documents based on a ranking of the similarity score of each of the plurality of trained sentence vectors; and
providing, by the data processing system, the electronic document.
2. The method of claim 1 , further comprising generating, by the vector generator, a word vector for each of the plurality of terms, wherein the word vector maps a respective term of the plurality of terms to a second vector space.
3. The method of claim 2 , wherein the word vector for each of the plurality of terms comprise a vector of weights indicating a probability of one of the plurality of terms occurring.
4. The method of claim 2 , further comprising generating, by the vector generator, the word vector for each of the plurality of terms with one of a Continuous Bag-of-Words neural network model or a Skip-Gram neural network model.
5. The method of claim 1 , further comprising generating, by the vector generator, a trained sentence vector based on an average of candidate word vectors of terms in a sentence.
6. The method of claim 1 , further comprising generating, by the scoring engine, the similarity score for each of the plurality of trained sentence vectors using a Pearson Similarity Calculation.
7. The method of claim 1 , further comprising:
generating, by the scoring engine, a return list comprising a subset of the plurality of candidate electronic documents corresponding to one of the plurality of trained sentence vectors having the similarity score above a predetermined threshold; and
providing, by the data processing system, the return list.
8. The method of claim 1 , further comprising:
calculating, by the vector generator, the sentence vector based on a difference between an inner product of each of a plurality of word vectors in a sentence and a common sentence vector.
9. The method of claim 8 , further comprising calculating, by the vector generator, a common sentence vector by averaging each of the plurality of trained sentence vectors.
10. The method of claim 1 , wherein the plurality of candidate electronic documents comprise web pages, text files, log files, forum questions, or forum answers.
11. The method of claim 1 , further comprising one hot encoding, by the vector generator, each of the plurality of terms to generate a binary array for each of the plurality of terms.
12. A system to retrieve content based on text input, the system comprising a memory storing processor executable instructions and one or more processors to:
receive a request comprising a plurality of terms;
determine, by a vector generator executed by the data processing system, an average of a plurality of word vectors, the plurality of word vectors including a word vector retrieved for each term of the plurality of terms of the request, the plurality of word vectors generated by multiplying an encoded vector for a respective term of the plurality of terms by a matrix of weights provided by at least one intermediate layer of a neural network;
generate, by the vector generator using the average of the plurality of word vectors, a sentence vector to map the request to a first vector space;
retrieve, from a database by the vector generator, a plurality of trained sentence vectors corresponding to a plurality of candidate electronic documents, wherein each of the plurality of trained sentence vectors map a respective sentence of each of the plurality of candidate electronic documents to the first vector space;
determine, by a scoring engine executed by the data processing system, a distance in the first vector space between the sentence vector and each trained sentence vector of the plurality of trained sentence vectors;
generate, by the scoring engine, a similarity score for each of the plurality of trained sentence vectors based on the respective one of the plurality of trained sentence vectors and the sentence vector and the distance in the first vector space between the sentence vector and each trained sentence vector of the plurality of trained sentence vectors;
select, by the scoring engine, an electronic document from the plurality of candidate electronic documents based on a ranking of the similarity score of each of the plurality of trained sentence vectors; and
provide the electronic document.
13. The system of claim 12 , further comprising the one or more processors to generate, by the vector generator, a word vector for each of the plurality of terms, wherein the word vector maps a respective term of the plurality of terms to a second vector space.
14. The system of claim 13 , wherein word vector for each of the plurality of terms comprises a vector of weights indicating a probability of one of the plurality of terms occurring.
15. The system of claim 13 , further comprising the one or more processors to generate, by the vector generator, the word vector for each of the plurality of terms with one of a Continuous Bag-of-Words neural network model or a Skip-Gram neural network model.
16. The system of claim 12 , further comprising the one or more processors to generate, by the vector generator, a trained sentence vector based on an average of candidate word vectors of terms in a sentence.
17. The system of claim 12 , further comprising the one or more processors to generate, by the scoring engine, the similarity score for each of the plurality of trained sentence vectors using a Pearson Similarity Calculation.
18. The system of claim 12 , further comprising the one or more processors to:
generate, by the scoring engine, a return list comprising a subset of the plurality of candidate electronic documents corresponding to one of the plurality of trained sentence vectors having the similarity score above a predetermined threshold; and
provide the return list.
19. The system of claim 12 , further comprising the one or more processors to calculate, by the vector generator, a common sentence vector by averaging each of the plurality of trained sentence vectors.
20. The system of claim 12 , wherein the plurality of candidate electronic documents comprise web pages, text files, log files, forum questions, or forum answers.