Inverted table for storing and querying conceptual indices
According to an aspect, storing and querying conceptual indices (CIs) includes creating a conceptual inverted index (CII) from the CIs. The CII includes CII entries, each of which corresponds to a concept in a concept graph. Creating the CII includes populating each entry with pointers to documents selected from the CIs having likelihoods of being related to the concept that are greater than a threshold value, and the corresponding likelihoods. An aspect also includes receiving a query that includes a concept in the concept graph, and generating query results from a search that include the row at least a subset of the pointers to documents. Each of the CIs is associated with a corresponding document and includes a CI entry for each concept in the concept graph, and each of the CI entries specifies a value indicating a likelihood that the document is related to the concept in the concept graph.
1. A method, comprising:
creating a conceptual inverted index (CII) based on conceptual indices (CIs), the CII including CII entries, each of which corresponds to a separate concept in a concept graph, the creating including for each CII entry:
with respect to the concept corresponding to the CII entry, populating the CII entry with:
pointers to documents selected from the CIs having likelihoods of being related to the concept that are greater than a threshold value; and
the corresponding likelihoods of the documents;
receiving a query that includes one of the concepts in the concept graph as a search term;
searching the CII for the search term; and
generating query results from the searching, the query results including at least a subset of the pointers to documents;
creating an explanations index that indicates for each of the documents pointed to by the pointers included in the query results, a likelihood that each of one or more concepts extracted from the document is related to the search term; and
generating an explanation of the query results based on the explanations index, the explanation including a summary of a relevance of the query results to the query,
wherein each of the CIs is associated with a corresponding one of the documents and includes a CI entry for each concept in the concept graph, and each of the CI entries specifies a value indicating a likelihood that the one of the documents is related to the concept in the concept graph.
2. The method of claim 1 , wherein the query results include a pointer to a document that does not explicitly mention the search term.
3. The method of claim 1 , wherein the at least one CI entry indicates a document having a likelihood of being related to a concept in the concept graph that is not mentioned in the document.
4. The method of claim 3 , wherein the concept not mentioned in the document is related to the concept included in the query.
5. The method of claim 1 , wherein the query results only include pointers to documents having likelihoods greater than a second threshold of being related to the concept included in the search term.
6. The method of claim 1 , wherein the query further includes one or more additional concepts as the search term.
7. The method of claim 1 , wherein each document is associated with a valid version number and the generating query results includes verifying that any pointers to documents correspond to documents that match the associated valid version numbers.
8. The method of claim 7 , wherein the method further comprises removing pointers to documents that do not match the associated valid version numbers.
9. The method of claim 1 , wherein a document is related to a concept in the concept graph if a concept extracted from the document is connected to the concept in the concept graph via a path in the concept graph.
10. The method of claim 1 , wherein the summary includes an excerpt of text.
11. The method of claim 1 , wherein the summary includes contents of the explanations index.