Systems and methods for interpreting natural language search queries
Systems and methods are described herein for interpreting natural language search queries that account for contextual relevance of words of the search query that would ordinarily not be processed, including, for example, processing each word of the query. Each term or phrase is associated with a respective part of speech, and a frequency of occurrence of a combination of adjacent terms or phrases public domain is determined. A relevance of each term is then determined based on its respective type of term and frequency of occurrence in the public domain. The natural language search query is then interpreted based on the importance or relevance of each term.
1. A computer-implemented method for interpreting a natural language search query, the method comprising using processing circuitry for:
receiving the natural language search query;
determining, using natural language processing, that the natural language search query comprises a plurality of terms or phrases, each term or phrase of the plurality of terms or phrases is associated with a respective term type;
determining, based on a grammatical structure, a first term type associated with a first term or phrase of the plurality of terms or phrases and a second term type associated with a second term or phrase of the plurality of terms or phrases, wherein the first term type is different than the second term type;
performing a first search for the first term or phrase in a public domain and a second search for the second term or phrase in the public domain;
determining a relation score between the first term or phrase and the second term or phrase based on the first search and the second search;
based on the determined relation score, interpreting the natural language search query including the first term or phrase and the second term or phrase, in a context of the first term type;
retrieving search results based on the interpreted natural language search query; and
generating for display the search results.
2. The method of claim 1 , wherein the natural language search query is received from an input device.
3. The method of claim 1 , wherein the natural language search query is received as audio data, then the method further comprises transcribing the natural language search query into a plurality of words.
4. The method of claim 1 , wherein determining that the natural language search query comprises a plurality of terms or phrases, further comprises:
splitting the natural language search query into a plurality of words;
analyzing a first word of the plurality of words;
determining, based on analyzing the first word, whether the first word can be part of the first term type;
in response to determining that the first word can be part of the first term type, analyzing the first word together with a second word that immediately follows the first word;
determining, based on analyzing the first word together with the second word, whether the first word and the second word can be analyzed using the first term type;
in response to determining that the first word and the second word can be analyzed using the first term type, identifying the first and second word as associated with the first term type.
5. The method of claim 1 , further comprises:
determining a respective frequency with which the second term or phrase immediately follows the first term or phrase of the plurality of terms or phrases in metadata describing a plurality of content items;
determining a relevance for each term or phrase of the plurality of terms or phrases based on its respective term type and frequency; and
interpreting the natural language search query based on the relevance of each term or phrase.
6. The method of claim 5 , wherein determining the respective frequency with which the second term or phrase immediately follows the first term or phrase of the plurality of terms or phrases in metadata describing a plurality of content items comprises:
retrieving the metadata describing the plurality of content items; and
counting the total occurrences of the second term or phrase immediately following the first term or phrase contained in the metadata.
7. The method of claim 1 , wherein the term type is selected from a keyword, a genre, and content type.
8. The method of claim 1 , further comprises:
generating a respective vector for each term of the plurality of terms;
accessing a knowledge graph associated with content metadata;
identifying a plurality of terms or phrases to which each term or phrase of the plurality of terms or phrases connects in the knowledge graph;
calculating a distance between each respective term or phrase and each term or phrase connected to the respective term or phrase; and
generating the vector for each term or phrase based on connections of each respective term or phrase and the distance between each respective term or phrase and each term or phrase to which each respective term or phrase is connected.
9. A system for interpreting a natural language search query, the system comprising control circuitry configured to:
receive the natural language search query;
determine, using natural language processing, that the natural language search query comprises a plurality of terms or phrases, each term or phrase of the plurality of terms or phrases is associated with a respective term type;
determine, based on a grammatical structure, a first term type associated with a first term or phrase of the plurality of terms or phrases and a second term type associated with a second term or phrase of the plurality of terms or phrases, wherein the first term type is different than the second term type;
perform a first search for the first term or phrase in a public domain and a second search for the second term or phrase in the public domain;
determine a relation score between the first term or phrase and the second term or phrase based on the first search and the second search;
based on the determined relation score, interpret the natural language search query including the first term or phrase and the second term or phrase, in a context of the first term type;
retrieve search results based on the interpreted the natural language search query; and generate for display the search results.
10. The system of claim 9 , wherein the natural language search query is received from an input device.
11. The system of claim 9 , wherein the natural language search query is received as audio data, then the method further comprises transcribing the natural language search query into a plurality of words.
12. The system of claim 9 , wherein the control circuitry configured to determine that the natural language search query comprises a plurality of terms, is further configured to:
split the natural language search query into a plurality of words;
analyze a first word of the plurality of words;
determine, based on analyzing the first word, whether the first word can be part of the first term type;
in response to determining that the first word can be part of the first term type, analyze the first word together with a second word that immediately follows the first word;
determine, based on analyzing the first word together with the second word, whether the first word and the second word can be analyzed using the first term type;
in response to determining that the first word and the second word can be analyzed using the first term type, identify the first and second word as associated with the first term type.
13. The system of claim 9 , the control circuitry is further configured to:
determine a respective frequency with which the second term or phrase immediately follows the first term or phrase of the plurality of terms or phrases in metadata describing a plurality of content items;
determine a relevance for each term or phrase of the plurality of terms or phrases based on its respective term type and frequency; and
interpret the natural language search query based on the relevance of each term or phrase.
14. The system of claim 13 , wherein the control circuitry configured to determine the respective frequency with which the second term immediately follows the first term or phrase of the plurality of terms or phrases in metadata describing a plurality of content items is configured to:
retrieve the metadata describing the plurality of content items; and
count the total occurrences of the second term or phrase immediately following the first term or phrase contained in the metadata.
15. The system of claim 9 , wherein the term type is selected from a keyword, a genre, and content type.
16. The system of claim 9 , the control circuitry is further configured to:
generate a respective vector for each term or phrase of the plurality of terms or phrases;
access a knowledge graph associated with content metadata;
identify a plurality of terms or phrases to which each term or phrase of the plurality of terms or phrase connects in the knowledge graph;
calculate a distance between each respective term or phrase and each term or phrase connected to the respective term or phrase; and
generate the vector for each term or phrase based on connections of each respective term or phrase and the distance between each respective term or phrase and each term or phrase to which each respective term or phrase is connected.
17. A non-transitory computer-readable medium having non-transitory computer-readable instructions encoded thereon for interpreting a natural language search query that, when executed by control circuitry, cause the control circuitry to:
receive the natural language search query;
determine, using natural language processing, that the natural language search query comprises a plurality of terms or phrases, each term or phrase of the plurality of terms or phrases is associated with a respective term type;
determine, based on a grammatical structure, a first term type associated with a first term or phrase of the plurality of terms or phrases and a second term type associated with a second term or phrase of the plurality of terms or phrase, wherein the first term type is different than the second term type;
perform a first search for the first term or phrase in a public domain and a second search for the second term or phrase in the public domain;
determine a relation score between the first term or phrase and the second term or phrase based on the first search and the second search;
based on the determined relation score, interpret the natural language search query including the first term or phrase and the second term or phrase, in a context of the first term type;
retrieve search results based on the interpreted the natural language search query; and generate for display the search results.
18. The non-transitory computer-readable medium of claim 17 , wherein the natural language search query is received from an input device.
19. The non-transitory computer-readable medium of claim 17 , wherein the natural language search query is received as audio data, wherein the execution of the instructions further causes the control circuitry to transcribe the natural language search query into a plurality of words.
20. The non-transitory computer-readable medium of claim 17 , wherein execution of the instruction to determine that the natural language search query comprises a plurality of terms or phrase, is further configured to:
split the natural language search query into a plurality of words;
analyze a first word of the plurality of words;
determine, based on analyzing the first word, whether the first word can be part of the first term type;
in response to determining that the first word can be part of the first term type, analyze the first word together with a second word that immediately follows the first word;
determine, based on analyzing the first word together with the second word, whether the first word and the second word can be analyzed using the first term type;
in response to determining that the first word and the second word can be analyzed using the first term type, identify the first and second word as associated with the first term type.