Public event detection platform
Methods and systems for determining public events are described. Data is received from a plurality of heterogeneous data feeds each having a native format. Event data objects are generated based on the received data, the event data objects each including at least a human readable message, an event time, and an event location. The human readable message of each of the event data objects is parsed into keywords, and the event data objects are grouped into one or more public events based on the keywords, time, location, and a sentiment based on the keywords. The public events are filtered based on a filter score, and the event data objects are clustered based on sentiment, location, and time. A density of the clustered event data objects is determined, and the validity of the public events is determined by a comparison of the density to a predetermined density threshold.
1. A method of determining public events, the method comprising:
receiving data from a plurality of heterogeneous data feeds each having a native format;
generating a plurality of event data objects based on the received data from the plurality of heterogeneous data feeds, each of the plurality of data objects including at least a human readable message, an event time, and an event location;
parsing the at least one human readable message of each of the plurality of event data objects into keywords;
for each event data object:
determining a probability that the keywords classify the event data object as one or more of a set of predetermined tags;
assigning one or more of the set of predetermined tags to the event data object based on the determined probability; and
assigning a sentiment to the event data object based on a valence score determined by a comparison of the keywords and a predetermined set of sentiment words;
grouping each of the plurality of event data objects into at least one public event based on the assigned tags, the event time of each of the plurality of event data objects, the event location of each of the plurality of event data objects, and the sentiment of each of the event data objects;
filtering the at least one public event based on a filter score;
clustering the plurality of event data objects grouped within the at least one public event based on at least the sentiment, the event location, and the event time of each of the plurality of event data objects grouped within the at least one public event;
determining the density of the clustered event data objects; and
determining the validity of the at least one public event by comparing the determined density to a predetermined threshold density.
2. The method of claim 1 , wherein the plurality of heterogeneous data feeds include at least one of a social media data feed, a beacon data feed, a video data feed, a smart app data feed, a manual input data feed, and a third party data feed.
3. The method of claim 1 , wherein generating the plurality of event data objects based on the received data from the plurality of heterogeneous data feeds further comprises converting the format of the received data from the native format to a standard format including converting slang, emoji, shorthand text, or non-standard human native language message data to normalized human native language message data.
4. The method of claim 1 , wherein the determined probability is based on a training data set including at least a user input indicating validity of the at least one public event.
5. The method of claim 4 , wherein the determined probability is based on a Multinomial Naïve Bayes machine learning algorithm.
6. The method of claim 1 , wherein the assigned sentiment is a predetermined hazard level.
7. The method of claim 1 , wherein the filter score comprises a spam filter score based on:
a frequency of generating event data objects grouped within the at least one public event from a single data feed; and
a similarity of the human readable message of each of the plurality of event data objects grouped within the at least one public event to the human readable messages of each of the other plurality of event data objects grouped within the at least one public event.
8. The method of claim 7 , wherein the spam filter score is further based on a weighted credibility score determined by tracking each of the plurality of heterogeneous data feeds.
9. The method of claim 1 , wherein clustering the plurality of event data objects grouped within the at least one public event includes weighting one or more of the sentiment, the event location, and the event time of each of the plurality of event data objects grouped within the at least one public event.
10. A public event detection system comprising:
a programmable circuit;
a memory operatively connected to the programmable circuit, the memory storing a public event detection application comprising instructions which, when executed, cause the programmable circuit to:
receive data from a plurality of heterogeneous data feeds each having a native format;
generate a plurality of event data objects based on the received data from the plurality of heterogeneous data feeds, each of the plurality of data objects including at least a human readable message, an event time, and an event location;
parse the at least one human readable message of each of the plurality of event data objects into keywords;
for each event data object:
determine a probability that the keywords classify the event data object as one or more of a set of predetermined tags;
assign one or more of the set of predetermined tags to the event data object based on the determined probability; and
assign a sentiment to the event data object based on a valence score determined by a comparison of the keywords and a predetermined set of sentiment words;
group each of the plurality of event data objects into at least one public event based on the assigned tags, the event time of each of the plurality of event data objects, the event location of each of the plurality of event data objects, and the sentiment of each of the event data objects;
filter the at least one public event based on a filter score;
cluster the plurality of event data objects grouped within the at least one public event based on at least the sentiment, the event location, and the event time of each of the plurality of event data objects grouped within the at least one public event;
determine the density of the clustered event data objects; and
determine the validity of the at least one public event by comparing the determined density to a predetermined threshold density.
11. The public event detection system 10 , wherein the plurality of heterogeneous data feeds include at least one of a social media data feed, a beacon data feed, a video data feed, a smart app data feed, a manual input data feed, and a third party data feed.
12. The public event detection system of claim 10 , wherein the instructions further cause the programmable circuit to convert the native format of the received data to a standardized analysis format including converting slang, emoji, shorthand text, or non-standard human native language message data to normalized human native language message data.
13. The method of claim 10 , wherein the determined probability is based on a training data set including at least a user input indicating validity of the at least one public event.
14. The method of claim 13 , wherein the determined probability is based on a Multinomial Naïve Bayes machine learning algorithm.
15. The method of claim 10 , wherein the assigned sentiment is a predetermined hazard level.
16. The method of claim 10 , wherein the filter score comprises a spam filter score based on:
a frequency of generating event data objects grouped within the at least one public event from a single data feed; and
a similarity of the human readable message of each of the plurality of event data objects grouped within the at least one public event to the human readable messages of each of the other plurality of event data objects grouped within the at least one public event.
17. The method of claim 16 , wherein the spam filter score is further based on a weighted credibility score determined by tracking each of the plurality of heterogeneous data feeds.
18. The method of claim 10 , wherein clustering the plurality of event data objects grouped within the at least one public event includes weighting one or more of the sentiment, the event location, and the event time of each of the plurality of event data objects grouped within the at least one public event.
19. A method of determining public events, the method comprising:
receiving a trigger event including at least an event time and an event location;
retrieving data from a plurality of heterogeneous data feeds;
generating a plurality of event data objects based on the retrieved, each of the plurality of data objects including at least a human readable message, an event time, and an event location;
parsing the at least one human readable message of each of the plurality of event data objects into keywords;
for each event data object:
determining a probability that the keywords classify the event data object as one or more of a set of predetermined tags;
assigning one or more of the set of predetermined tags to the event data object based on the determined probability; and
assigning a sentiment to the event data object based on a valence score determined by a comparison of the keywords and a predetermined set of sentiment words;
grouping each of the plurality of event data objects into at least one public event based on the assigned tags, the event time of each of the plurality of event data objects, the event location of each of the plurality of event data objects, and the sentiment of each of the event data objects;
filtering the at least one public event based on a filter score;
clustering the plurality of event data objects grouped within the at least one public event based on at least the sentiment, the event location, and the event time of each of the plurality of event data objects grouped within the at least one public event;
determining the density of the clustered event data objects;
determining the validity of the at least one public event by comparing the determined density to a predetermined threshold density; and
generating at least one public event notification alerts.
20. The method of claim 19 , wherein the determined probability is based on a training data set including at least a user input indicating validity of the at least one public event.