Social media events detection and verification
Systems and techniques for detecting and verifying social media events are disclosed. The system and techniques allow for processing of social media data to extract potentially valuable information in a timely manner and determine the veracity of the detected information. One implementation of the disclosure relates to event detection. Event detection involves ingestion and processing of social media data. Another implementation of the disclosure relates to verification of a detected event and generating a verification score.
1. A system comprising:
an event detection server including a processor and memory storing instructions that, in response to receiving social media data from at least one data source, cause the processor to:
apply a set of filter modules to the social media data to generate a data store, the data store comprising a set of identified concepts and corresponding attributes of the social media data;
select one of the set of identified concepts from the data store using a corresponding threshold value associated with the attributes of the social media data, wherein the corresponding threshold value is associated with three or more distinct attributes of the social media data related to the identified concept and wherein one of the attributes of the social media data is an authorship value and the corresponding threshold value represents at least three or more similar identified concepts associated with different authorship values;
generate an event detected cluster using the selected identified concept;
generate a verification score for each item of social media data associated with the event detected cluster, the verification score being indicative of a veracity of the event detected cluster, wherein the verification score is determined by analyzing at least a user category, a social media category and an event features category relating to each item of social media data associated with the event detected cluster; and
present the event detected cluster and verification score.
2. The system of claim 1 , wherein the memory stores instructions that, in response to receiving the social media data, cause the processor to delete the selected identified concept from the data store.
3. The system of claim 1 , wherein one of the set of filter modules detects language of the social media data and deleting the social media data that is not in English.
4. The system of claim 1 , wherein one of the set of filter modules detects profanity used in the social media data and removes the social media data containing the detected profanity.
5. The system of claim 1 , wherein one of the set of filter modules detects at least one of spam, chat and advertisement in the social media data and removing the social media data that contains the at least one detected spam, chat and advertisement.
6. The system of claim 1 , wherein one of the set of filter modules applies Parts-Of-Speech tagging of the social media data.
7. The system of claim 1 , wherein one of the set of filter modules analyzes semantic and syntactic structures in the social media data to determine identified concepts in the social media data.
8. The system of claim 1 further comprising a topic categorization module configured to generate a topic classification for the event detected cluster.
9. The system of claim 1 further comprising a summary module configured to generate a summary for the event detected cluster.
10. The system of claim 1 further comprising a newsworthiness module configured to generate a newsworthy score for the event detected cluster.
11. The system of claim 1 further comprising an opinion module configured to identify opinion or fact for each item of social media data associated with the event detected cluster.
12. The system of claim 1 further comprising a credibility module configured to generate a credibility score for each item of social media data associated with the event detected cluster.
13. The system of claim 1 , further comprising an event processing server configured to present the event detected cluster and the verification score to the user on a graphical user interface.
14. The system of claim 1 , wherein the user category comprises at least one of name of author, description of author, URL of author, location of author, location of the author matching the location of the event, author being a witness to the event, protection level of the author's account, or verification of the author.
15. The system of claim 1 , wherein the social media category comprises at least one of multimedia, url, elongated word, url from news source, or word sentiment associated with the social medial data.
16. The system of claim 1 , wherein the event features category comprises at least one topic of the event and portion of the social media that deny, believe or question the event associated with each item of the social media data.
17. The system of claim 1 , wherein the social media category is twitter data and the event features category further comprises at least one of a count of the most retweeted tweets, a frequency of retweeted tweets or a frequency of hashtags associated with each item of the social media data.
18. A computer-implemented method for detecting an event in social media, the method comprising:
receiving, by an event detection server, social media data from at least one data source;
applying, by the event detection server, a set of filter modules to the social media data to generate a data store, the data store comprising a set of identified concepts and corresponding attributes of the social media data;
selecting, by the event detection server, one of the set of identified concepts from the data store using a corresponding threshold value associated with the attributes of the social media data, wherein the corresponding threshold value is associated with three or more distinct attributes of the social media data related to the identified concept and wherein one of the attributes of the social media data is an authorship value and the corresponding threshold value represents at least three or more similar identified concepts associated with different authorship values;
generating, by the event detection server, an event detected cluster using the selected identified concept;
generating a verification score for each item of social media data associated with the event detected cluster, the verification score being indicative of a veracity of the event detected cluster, wherein the verification score is determined by analyzing at least a user category, a social media category and an event features category relating to each item of social media data associated with the event detected cluster;
presenting the event detected cluster and verification score.
19. The method of claim 18 further comprises presenting the event detected cluster and the verification score to the user on a graphical user interface.
20. The method of claim 18 wherein the user category comprises at least one of name of author, description of author, URL of author, location of author, location of the author matching the location of the event, author being a witness to the event, protection level of the author's account, or verification of the author.
21. The method of claim 18 wherein the social media category comprises at least one of multimedia, url, elongated word, url from news source, or word sentiment associated with the social medial data.
22. The method of claim 18 wherein the event features category comprises at least one topic of the event and portion of the social media that deny, believe or question the event associated with each item of the social media data.
23. The method of claim 18 wherein the social media category is twitter data and the event features category further comprises at least one of a count of the most retweeted tweets, a frequency of retweeted tweets or a frequency of hashtags associated with each item of the social media data.