Systems and methods for the creation, update and use of models in finding and analyzing content
Embodiments of a search system that provides knowledge based searching of content based on a knowledge model created from the content being searched are disclosed. Embodiments of such search systems may build a model of entitles and relationships representing the collective knowledge contained in a set of content analyzed. When a search is performed on content the model may be leveraged to improve the accuracy, relevance and recall of the search.
1. A non-transitory computer readable medium to store instructions that, when executed by a processor, cause the processor to:
access a data store, the data store storing content comprising a set of articles, an index to the set of articles, and an ontology including entities and relationships between the entities;
determine a first entity and a second entity based on an article in the set of articles;
determine proximity data for the first entity and the second entity in the article,
determine an entity score for the first entity;
save the entity score in the data store, wherein the entity score is associated with the article and with the first entity;
determine a relationship between the first entity and the second entity based on the article;
determine a strength value for the relationship, the strength value based on the proximity data, frequency data, and time data;
update the ontology with the strength value for the relationship;
receive a search entity through a search interface, the search entity corresponding to the first entity;
determine an initial set of articles based on the index, the search entity, and the ontology, the initial set of articles including a first search article and a second search article, wherein the first search article and the second search article reference the first entity;
determine a first score for the first search article based on a frequency of appearance of the first entity in the first search article and a frequency of appearance of the second entity in the first search article multiplied by the strength value for the relationship between the first entity and the second entity;
determine a second score for the second search article based on a frequency of appearance of the first entity in the second search article and a frequency of appearance of the second entity in the first search article multiplied by the strength value for the relationship between the first entity and the second entity;
rank the initial set of articles based on the first score and the second score; and
return the ranked initial set of articles through the search interface.
2. The computer readable medium of claim 1 , wherein to determine the initial set of articles includes:
to access the ontology to determine a second search entity based on the search entity and a modeled relationship between the search entity and the second search entity; and
to include a third article in the initial set of articles based on the second search entity, the third article referencing the second search entity.
3. The computer readable medium of claim 2 , wherein a modeled strength value of the modeled relationship meets or exceeds a threshold relationship strength.
4. The computer readable medium of claim 2 , wherein the receipt of the search entity through the search interface comprises to:
refine the search interface based on the determination of the second search entity;
allow a user of the search interface to select the second search entity; and
determine the initial set of articles based on the second search entity.
5. The computer readable medium of claim 1 , wherein the first entity is associated with a search plan specific to the first entity.
6. The computer readable medium of claim 5 , wherein the search plan includes a disambiguation array determined from the content.
7. The computer readable medium of claim 6 , wherein the disambiguation array includes a set of terms and counter terms.
8. The computer readable medium of claim 7 , wherein the determination of the initial set of articles comprises to perform a search of the content based on the disambiguation array using the index.
9. A method for identifying articles in response to a search entity, the method comprising:
accessing a data store, the data store storing content comprising a set of articles, an index to the set of articles, and an ontology including entities and relationships between the entities;
determining a first entity and a second entity based on an article in the set of articles;
determining proximity data for the first entity and the second entity in the article;
determining an entity score for the first entity;
saving the entity score in the data store, wherein the entity score is associated with the article and with the first entity;
determining a relationship between the first entity and the second entity based on the article;
determining a strength value for the relationship, the strength value based on the proximity data, frequency data, and time data;
updating the ontology with the strength value for the relationship;
receiving a search entity through a search interface, the search entity corresponding to the first entity;
determining an initial set of articles based on the index, the search entity, and the ontology, the initial set of articles including a first search article and a second search article, wherein the first search article and the second search article reference the first entity;
determining a first score for the first search article based on a frequency of appearance of the first entity in the first search article and a frequency of appearance of the second entity in the first search article multiplied by the strength value for the relationship between the first entity and the second entity;
determining a second score for the second search article based on a frequency of appearance of the first entity in the second search article and a frequency of appearance of the second entity in the first search article multiplied by the strength value for the relationship between the first entity and the second entity;
ranking the initial set of articles based on the first score and the second score; and
returning the ranked initial set of articles through the search interface.
10. The method of claim 9 , wherein the step of determining the initial set of articles comprises:
accessing the ontology to determine a second search entity based on the search entity and a modeled relationship between the search entity and the second search entity; and
including a third article in the initial set of articles based on the second search entity, the third article referencing the second search entity.
11. The method of claim 10 , wherein a modeled strength value of the modeled relationship meets or exceeds a threshold relationship strength.
12. The method of claim 10 , wherein the receipt of the search entity through the search interface comprises:
refining the search interface based on the determination of the second search entity;
allowing a user of the search interface to select the second search entity; and
determining the initial set of articles based on the second search entity.
13. The method of claim 9 , wherein the first entity is associated with a search plan specific to the first entity.
14. The method of claim 13 , wherein the search plan includes a disambiguation array determined from the content.
15. The method of claim 14 , wherein the disambiguation array includes a set of terms and counter terms.
16. The method of claim 15 , wherein the determination of the initial set of articles comprises to perform a search of the content based on the disambiguation array using the index.