Generating hypotheses and recognizing events in data sets
A hypotheses generation and event recognition system that enables event recognition by analyzing documents to construct one or more qualitative metrics (e.g., frequency of keywords, changes in sentiment, occurrence of ontological terms, evolution of topics, etc.), establishing a baseline for the qualitative metric(s), and outputting changes to that baseline for display. In aggregate, those qualitative metrics comprise temporal and/or spatial signals that, when combined, define signatures of events of interest. Accordingly, the user and/or the system may identify an event of interest based on the change in baseline. The system may further provide functionality to generate hypotheses by coding data according to an ontology, populating an ontology space, and using an optimization algorithm to rank points or neighborhoods in the coded ontology space. The system may further store links between ontological terms and qualitative metrics to provide functionality to test generated hypotheses that include those linked ontological terms.
1 . An automated method for recognizing an event in a geographic region, the method comprising:
receiving an ontology including ontological terms related to a domain of interest;
receiving documents by a server;
analyzing the documents, by the server, to identify the ontological terms and construct one or more temporal metrics indicative of events of interest in the domain of interest;
identifying, by the server, a relevant geographic region for each of at least some of the documents;
identifying, by the server, a baseline for each of the one or more temporal metrics;
analyzing additional documents, by the server, to identify the ontological terms in the additional documents, identify a relevant geographic region for each of at least some of the additional documents, and construct the one or more temporal metrics indicative of events of interest in the domain of interest;
identifying, by the server based on the analysis of one or more of the additional documents, a deviation from at least one baseline identified for at least one of the one or more temporal metrics; and
identifying a geographic region of interest, by the server, by identifying the relevant geographic region of the one or more additional documents indicative of the deviation from the at least one baseline.
2 . The method of claim 1 , further comprising:
identifying a potential event of interest, by the server, based on the deviation from the at least one baseline.
3 . The method of claim 1 , wherein identifying the relevant geographic region of each document or additional document comprises:
identifying, in the document or additional document, one or more named places in the relevant geographic region; or
determining that the document or additional document is from a publication from or focused on the relevant geographic region.
4 . The method of claim 1 , wherein identifying the geographic region of interest comprises identifying an emergence of documents associated with the geographic region of interest expressing themes related to the domain of interest.
5 . The method of claim 1 , wherein the one or more temporal metrics include:
a frequency of ontological terms indicative of the events of interest in the domain of interest; or
a sentimentality of the documents or the additional documents that include the ontological terms.
6 . The method of claim 1 , wherein the ontological terms related to the domain of interest comprise keywords of interest received from a user.
7 . The method of claim 1 , wherein the additional documents comprise documents generated or received by the server after the documents.
8 . The method of claim 1 , wherein identifying the deviation from the at least one baseline comprises identifying an emergence of documents clustered in time expressing themes related to the domain of interest.
9 . The method of claim 1 , wherein the ontology comprises a plurality of elements, each of the elements comprise a plurality of ontological terms, the ontology defines ontological vectors comprising an ontological term from each of two or more of the plurality of elements, each of the ontological vectors describes a hypothesis, and the method further comprises:
forming an ontology space wherein each dimension of the ontology space comprises one or more of the elements of the ontology;
populating the ontology space by adding the ontological vectors identified in the documents such that a weight or each point in the ontology space is proportional to a number of ontological vectors associated with that point found in the documents;
using an optimization algorithm to rank points or clusters of points in the ontology space based on the weights of the points or the clusters of points; and
outputting the ranked list of points or clusters of points, each point or cluster of points corresponding to one or more ontological vectors describing a hypothesis.
10 . The method of claim 9 , further comprising:
storing links between ontological terms included in the ontology and one or more temporal metrics; and
constructing the one or more temporal metrics in response to a determination that a linked ontological term is included in an ontological vector in the ranked list of points or clusters of points.
11 . A system, comprising:
non-transitory storage media configured to store:
documents; and
an ontology including ontological terms related to a domain of interest; and
a server comprising at least one computer processor configured to:
analyze the documents to identify the ontological terms and construct one or more temporal metrics indicative of events of interest in the domain of interest;
identify a relevant geographic region for each of at least some of the documents;
identify a baseline for each of the one or more temporal metrics;
analyze additional documents to identify the ontological terms in the additional documents, identify a relevant geographic region for each of at least some of the additional documents, and construct the one or more temporal metrics indicative of events of interest in the domain of interest;
identify, based on the analysis of one or more of the additional documents, a deviation from at least one baseline identified for at least one of the one or more temporal metrics; and
identify a geographic region of interest by identifying the relevant geographic region of the one or more additional documents indicative of the deviation from the at least one baseline.
12 . The system of claim 11 , wherein the server is further configured to identify a potential event of interest based on the deviation from the at least one baseline.
13 . The system of claim 11 , wherein the server is configured to identify the relevant geographic region of each document or additional document by:
identifying, in the document or additional document, one or more named places in the relevant geographic region; or
determining that the document or additional document is from a publication from or focused on the relevant geographic region.
14 . The system of claim 11 , wherein the server is configured to identify the geographic region of interest by identifying an emergence of documents associated with the geographic region of interest expressing themes related to the domain of interest.
15 . The system of claim 11 , wherein the one or more temporal metrics include:
a frequency of ontological terms indicative of the events of interest in the domain of interest; or
a sentimentality of the documents or the additional documents that include the ontological terms.
16 . The system of claim 11 , wherein the ontological terms related to the domain of interest comprise keywords of interest received from a user.
17 . The system of claim 11 , wherein the additional documents comprise documents generated or received by the server after the documents.
18 . The system of claim 11 , wherein the server is configured to identify the deviation from the at least one baseline by identifying an emergence of documents clustered in time expressing themes related to the domain of interest.
19 . The system of claim 11 , wherein:
the ontology comprises a plurality of elements, each of the elements comprising a plurality of ontological terms;
the ontology defines ontological vectors comprising an ontological term from each of two or more of the plurality of elements, each of the ontological vectors describing a hypothesis; and
the server is further configured to:
form an ontology space wherein each dimension of the ontology space comprises one or more of the elements of the ontology;
populate the ontology space by adding the ontological vectors identified in the documents such that a weight or each point in the ontology space is proportional to a number of ontological vectors associated with that point found in the documents;
use an optimization algorithm to rank points or clusters of points in the ontology space based on the weights of the points or the clusters of points; and
output the ranked list of points or clusters of points, each point or cluster of points corresponding to one or more ontological vectors describing a hypothesis.
20 . The system of claim 19 , wherein:
the non-transitory storage media configured stores links between ontological terms included in the ontology and one or more temporal metrics; and
the server is configured to construct the one or more temporal metrics in response to a determination that a linked ontological term is included in an ontological vector in the ranked list of points or clusters of points.