System and method for identifying relevant search results via an index
A system and method for identifying relevant search results via an index is provided. A search query is received. A semantic representation of query substructures and a list of key terms is generated for the query. Each key term includes a term in the search query or a term related to the query. An inverted index having key terms each associated with a semantic representation and a link to a source reference is accessed. The inverted index is queried using a subset of key terms. A result set for the subset key terms is identified within the inverted index. Each result is scored and a subset of the result set is identified as retrieval candidates based on the scoring. One or more of the retrieval candidates are selected based on a comparison of the query semantic representation with the semantic representations for the retrieval candidates.
1. A system for identifying relevant search results via an index, comprising:
a query module to receive a search query and to generate a semantic representation for the query comprising a plurality of substructures as a semantic analysis of the search query and a list of key terms, wherein the key terms each comprise a term in the search query or a term related to one of the terms in the search query;
an inverted index comprising a set of key terms each associated with a semantic representation for each of a plurality of passages comprising a plurality of substructures and further associated with a link to a reference document for the passage;
a candidate module to identify one or more of the passages as retrieval candidates by comparing the semantic representations associated with the passages with the semantic representation of the search query, comprising:
a term selection module to select a subset of the key terms from the search query and to query the inverted index with the key terms in the subset;
a result module to identify within the inverted index a result set for each of the key terms in the subset; and
a scoring module to score each of the results in the set based on a distance of relationships of the substructures in the semantic representation of the search query and the semantic representation of the passages and to identify a subset of the result sets as the retrieval candidates based on the scoring; and
a candidate selection module to select one or more of the retrieval candidates based on a comparison of the semantic representation of the search query with the semantic representations for each of the retrieval candidates,
wherein the modules are executed via a computer processor.
2. A system according to claim 1 , comprising:
a reference result module to select the reference documents associated each of the selected retrieval candidates as results of the search query.
3. A system according to claim 1 , comprising:
a refinement module to refine the identification of the retrieval candidates based upon one or more heuristically tuned thresholds.
4. A system according to claim 1 , comprising:
a scoring comparison module to score the comparisons of the search query semantic representation with the semantic representations of each retrieval candidate.
5. A system according to claim 1 , comprising:
a transform module to identify the key terms for the query via a set of transform rules.
6. A system according to claim 5 , wherein the transform rules comprise at least one of rules to identify and index particular terms in the search query, identify and extract synonyms to the terms in the search query, identify and extract hypernyms to the terms in the search query, identify and associate the grammatical role of the terms in the search query, identify and associate a word sense of the terms in the search query, and associate a linguistic identifier.
7. A system according to claim 1 , comprising:
a retrieval goal module to define search retrieval goals for the retrieval candidates comprising one or more of retrieving certain search terms, minimum relevance scores, inclusion of one or more search terms identified as necessary or significant, and identifying one of a minimum and maximum number of retrieval candidates.
8. A system according to claim 7 , comprising:
a success determination module to determine a success of the retrieval candidates against one or more of the search retrieval goals.
9. A system according to claim 1 , wherein each related term comprises one of a synonym, hypernym, hyponym, and an identifier of the term from the search query.
10. A method for identifying relevant search results via an index, comprising steps of:
receiving a search query and generating a semantic representation for the query comprising a plurality of substructures as a semantic analysis of the search query and a list of key terms, wherein the key terms each comprise a term in the search query or a term related to one of the terms in the search query;
accessing an inverted index comprising a set of key terms each associated with a semantic representation for each of a plurality of passages comprising a plurality of substructures and further associated with a link to a reference document for the passage;
identifying one or more of the passages as retrieval candidates by comparing the semantic representations associated with the passages with the semantic representation of the search query, comprising:
selecting a subset of the key terms from the search query and querying the inverted index with the key terms in the subset;
identifying within the inverted index a result set for each of the key terms in the subset; and
scoring each of the results in the set based on a distance of relationships of the substructures in the semantic representation of the search query and the semantic representation of the passages and identifying a subset of the result sets as the retrieval candidates based on the scoring; and
selecting one or more of the retrieval candidates based on a comparison of the semantic representation of the search query with the semantic representations for each of the retrieval candidates.
11. A method according to claim 10 , comprising:
selecting the reference documents associated each of the selected retrieval candidates as results of the search query.
12. A method according to claim 10 , comprising:
refining the identification of the retrieval candidates based upon one or more heuristically tuned thresholds.
13. A method according to claim 10 , comprising:
scoring the comparisons of the search query semantic representation with the semantic representations of each retrieval candidate.
14. A method according to claim 10 , comprising:
identifying the key terms for the query via a set of transform rules.
15. A method according to claim 14 , wherein the transform rules comprise at least one of rules to identify and index particular terms in the search query, identify and extract synonyms to the terms in the search query, identify and extract hypernyms to the terms in the search query, identify and associate the grammatical role of the terms in the search query, identify and associate a word sense of the terms in the search query, and associate a linguistic identifier.
16. A method according to claim 10 , comprising:
defining search retrieval goals for the retrieval candidates comprising one or more of retrieving certain search terms, minimum relevance scores, inclusion of one or more search terms identified as necessary or significant, and identifying one of a minimum and maximum number of retrieval candidates.
17. A method according to claim 16 , comprising:
determining a success of the retrieval candidates against one or more of the search retrieval goals.
18. A method according to claim 10 , wherein each related term comprises one of a synonym, hypernym, hyponym, and an identifier of the term from the search query.