Search system and method
A search system and method are provided that uses taxonomies, entities, facets, and ontologies to provide a user with a more comprehensive set of search results in response to a query. The search system has an indexing engine that performs one or more indexing steps that permit the search engine to return a comprehensive set of search results. For example, the indexing engine may index a document according to a set of synsets so that the search engine may use the synsets, during retrieval of results to a query, to return a more comprehensive set of search results.
1. An indexing system for linking formally expressed knowledge with a corpus of content having a plurality of pieces of content, the system comprising:
a knowledge base containing a plurality of pieces of formally expressed knowledge, the formally represented knowledge further comprising one or more synsets wherein each synset contains a group of terms that have a similar meaning, one or more taxonomies wherein each taxonomy contains one or more synsets in a subject matter area that are organized from a synset having a general meaning to a synset having a specific meaning, one or more ontologies wherein each ontology contains one or more synsets associated with an area of interest and one or more facets wherein each facet is associated with a particular ontology and wherein a document is associated with the facet when the document contains the one or more synsets associated with the facet;
a computer system having one or more software pieces each having a plurality of lines of computer instructions wherein the computer instructions are executed by the computer system, the software pieces further comprising an index engine that indexes each piece of content in a corpus to generate one or more indexes for each piece of content, the index engine further comprising an assignment engine that assigns an index to each piece of content based on the formally expressed knowledge contained in the knowledge base.
2. The system of claim 1 , wherein the assignment engine further comprises a synset assignment engine that assigns a particular synset to a particular piece of content when a term in the particular piece of content matches a term in the synset.
3. The system of claim 2 , wherein the assignment engine disambiguates a particular piece of content to determine an interpretation of the particular piece of content based on the one or more synsets assigned to the particular piece of content.
4. The system of claim 1 , wherein the assignment engine further comprises a facet assignment engine that assigns a particular facet to a particular piece of content when a term in the piece of content is contained in the synsets of the facet.
5. The system of claim 4 , wherein the assignment engine further comprises a synset assignment engine that assigns a particular synset to a particular piece of content when a term in the particular piece of content matches a term in the synset.
6. The system of claim 1 , wherein the formally expressed knowledge further comprises one or more entity lists wherein each entity list contains a list of one or more entities associated with other entities contained in the entity list, and wherein the index engine further comprises an entity extraction engine that extracts an entity contained in an entity list from a particular piece of content in the corpus and associates a particular entity list to the particular piece of content.
7. The system of claim 6 , wherein the assignment engine further comprises a facet assignment engine that assigns a particular facet to a particular piece of content when a term in the piece of content is contained in the synsets of the facet.
8. The system of claim 7 , wherein the assignment engine further comprises a synset assignment engine that assigns a particular synset to a particular piece of content when a term in the particular piece of content matches a term in the synset.
9. The system of claim 1 , wherein the index engine further comprises an authority determining engine that determines an authority score associated with each piece of content, the authority score being based on one or more factors including a reputation of the author of the piece of content and a reliability of the source of the piece of content.
10. The system of claim 9 , wherein the formally expressed knowledge further comprises one or more entity lists wherein each entity list contains a list of one or more entities associated with other entities contained in the entity list, and wherein the index engine further comprises an entity extraction engine that extracts an entity contained in an entity list from a particular piece of content in the corpus and associates a particular entity list to the particular piece of content.
11. The system of claim 10 , wherein the assignment engine further comprises a facet assignment engine that assigns a particular facet to a particular piece of content when a term in the piece of content is contained in the synsets of the facet.
12. The system of claim 11 , wherein the assignment engine further comprises a synset assignment engine that assigns a particular synset to a particular piece of content when a term in the particular piece of content matches a term in the synset.
13. A computer implemented indexing method for linking formally represented knowledge with a corpus of content having a plurality of pieces of content text, image and video artifacts using a knowledge base containing a plurality of pieces of formally represented knowledge, the formally represented knowledge further comprising one or more synsets wherein each synset contains a group of terms that have a similar meaning, one or more taxonomies wherein each taxonomy contains one or more synsets in a subject matter area that are organized from a synset having a general meaning to a synset having a specific meaning, one or more ontologies wherein each ontology contains one or more synsets associated with an area of interest and one or more facets wherein each facet is associated with a particular ontology and wherein a document is associated with the facet when the document contains the one or more synsets associated with the facet, the method comprising:
indexing using a computer implemented index engine, each piece of content in a corpus to generate one or more indexes for each piece of content, wherein the indexing further comprises assigning an index to each piece of content based on the formally represented knowledge contained in the knowledge base.
14. The method of claim 13 , wherein assigning an index further comprises assigning a particular synset to a particular piece of content when a term in the particular piece of content matches a term in the synset.
15. The method of claim 14 , wherein assigning an index further comprises disambiguating a particular piece of content to determine an interpretation of the particular piece of content based on the one or more synsets assigned to the particular piece of content.
16. The method of claim 13 , wherein assigning an index further comprises assigning a particular facet to a particular piece of content when a term in the piece of content is contained in the synsets of the facet.
17. The method of claim 16 , wherein assigning an index further comprises assigning a particular synset to a particular piece of content when a term in the particular piece of content matches a term in the synset.
18. The method of claim 13 , wherein the formally represented knowledge further comprises one or more entity lists wherein each entity list contains a list of one or more entities associated with other entities contained in the entity list, and wherein assigning the index further comprises extracting an entity contained in an entity list from a particular piece of content in the corpus and associating a particular entity list to the particular piece of content.
19. The method of claim 18 , wherein assigning an index further comprises assigning a particular facet to a particular piece of content when a term in the piece of content is contained in the synsets of the facet.
20. The method of claim 19 , wherein assigning an index further comprises assigning a particular synset to a particular piece of content when a term in the particular piece of content matches a term in the synset.
21. The method of claim 13 , wherein the indexing further comprises determining an authority score associated with each piece of content, the authority score being based on one or more factors including a reputation of the author of the piece of content and a reliability of the source of the piece of content.
22. The method of claim 21 , wherein the formally represented knowledge further comprises one or more entity lists wherein each entity list contains a list of one or more entities associated with other entities contained in the entity list, and wherein assigning the index further comprises extracting an entity contained in an entity list from a particular piece of content in the corpus and associating a particular entity list to the particular piece of content.
23. The method of claim 22 , wherein assigning an index further comprises assigning a particular facet to a particular piece of content when a term in the piece of content is contained in the synsets of the facet.
24. The method of claim 23 , wherein assigning an index further comprises assigning a particular synset to a particular piece of content when a term in the particular piece of content matches a term in the synset.
25. An indexing system capable of linking formally expressed knowledge to a corpus of content, the system comprising:
a corpus of content having a plurality of pieces of content;
a knowledge base containing a plurality of pieces of formally expressed knowledge, the formally represented knowledge further comprising one or more synsets wherein each synset contains a group of terms that have the same meaning, one or more taxonomies wherein each taxonomy contains one or more synsets in a subject matter area that are organized from a synset having a general meaning to a synset having a specific meaning, one or more ontologies wherein each ontology contains one or more synsets associated with an area of interest and one or more facets wherein each facet is associated with a particular ontology and wherein a document is associated with the facet when the document contains the one or more synsets associated with the facet; and
an indexing engine, coupled to the knowledge base and the corpus of content, that performs a first level of indexing to generate one or more features for each piece of content in the corpus of content and performs a second level of indexing, using the one or more features for each piece of content, to classify a piece of content to a particular facet based on the one or more facets in the knowledge base and to determine an authority score associated with each piece of content, the authority score being based on one or more factors including a reputation of the author of the piece of content and a reliability of the source of the piece of content.
26. A computer implemented indexing method for linking formally represented knowledge with a corpus of content comprising a plurality of pieces of content using a knowledge base containing a plurality of pieces of formally represented knowledge, the formally represented knowledge further comprising one or more synsets wherein each synset contains a group of terms that have the same meaning, one or more taxonomies wherein each taxonomy contains one or more synsets in a subject matter area that are organized from a synset having a general meaning to a synset having a specific meaning, one or more ontologies wherein each ontology contains one or more synsets associated with an area of interest and one or more facets wherein each facet is associated with a particular ontology and wherein a document is associated with the facet when the document contains the one or more synsets associated with the facet, the method comprising:
performing, using a computer implemented indexing engine, a first level of indexing to generate one or more features for each piece of content in the corpus of content; and
performing, using the indexing engine, a second level of indexing, using the one or more features for each piece of content, to classify a piece of content to a particular facet based on the one or more facets in the knowledge base and to determine an authority score associated with each piece of content, the authority score being based on one or more factors including a reputation of the author of the piece of content and a reliability of the source of the piece of content.
27. The method of claim 26 , wherein performing the first level of indexing further comprises identifying one or more keywords for each piece of content in the corpus of content and identifying one of one or more synsets and one or more entities for each piece of content in the corpus of content to generate the one or more features for each piece of content in the corpus of content.
28. The method of claim 27 , wherein performing the first level of indexing further comprises identifying one or more orphan words in each piece of content in the corpus of content.