System and methods thereof for real-time detection of an hidden connection between phrases
A system for identifying hidden connections between non-sentiment phrases. The system comprises a network interface enabling an access to one or more data sources; a data warehouse storage for at least storing a plurality of phrases including sentiment phrases and non-sentiment phrases; an analysis unit for identifying hidden connections between non-sentiment phrases based on at least one proximity rule and for generating at least an association between at least two non-sentiment phrases having a hidden connection and a sentiment phrase, wherein an association between the at least two non-sentiment phrases having the hidden connection and the corresponding sentiment phrase is a term taxonomy.
1. A system for identifying hidden connections between non-sentiment phrases, comprising:
a network interface for enabling an access to one or more data sources accessible by a plurality of user nodes;
a non-transitory data warehouse storage medium for at least storing a plurality of phrases including sentiment phrases and the non-sentiment phrases received from the plurality of user nodes; and
an analysis unit for identifying hidden connections between the non-sentiment phrases based on at least one proximity rule applies on mentions of the non-sentiment phrases in collected textual content, wherein the at least one proximity rule is at least a distance measure based on at least one of a number of characters and words between two mentions of the non-sentiment phrases, wherein the analysis unit is further configured to generate at least an association between at least two non-sentiment phrases having a hidden connection and a sentiment phrase, wherein an association between the at least two non-sentiment phrases having the hidden connection and the corresponding sentiment phrase is a term taxonomy, wherein the hidden connection is any one of: a first degree of separation of the at least two non-sentiment phrases and at least a first degree of separation of the at least two non-sentiment phrases that are associated with a common non-sentiment phrase.
2. The system of claim 1 , further comprises:
a mining unit for collecting the textual content from the one or more sources and generating the sentiment phrases and the non-sentiment phrases.
3. The system of claim 2 , wherein the mining unit is connected to a phrase database containing identified non-sentiment phrases and sentiment phrases, wherein generating of the phrases further includes comparing phrases in the textual content to phrases stored in the phrase database and separating between the sentiment phrases and the non-sentiment phrases identified in the textual content.
4. The system of claim 1 , wherein the analysis unit is further configured to store the term taxonomies in the data warehouse storage connected to the network, wherein responsive to a query the analysis unit provides a sentiment to the non-sentiment phrase provided in the query.
5. The system of claim 1 , wherein the analysis unit is further configured to identify hidden connections between non-sentiment phrases by:
identifying all connections between at least two non-sentiment phrases, wherein at least two non-sentiment phrases are determined to have a hidden connection when they meet the at least one proximity rule;
identifying all direct connections among all the identified connections, wherein a direct connection includes at least two non-sentiment phrases that meet a predetermined correlation threshold; and
filtering out all identified direct connections from the identified connected non-sentiment phrases, thereby resulting with hidden connections of non-sentiment phrases, wherein each of the hidden connections includes at least two non-sentiment phrases.
6. The system of claim 5 , wherein the proximity rule further comprises at least one of: a number of mentions of the at least two non-sentiment phrases in a web page, a number of mentions of the at least two non-sentiment phrases in different web pages linked to each other, a number of mentions of the at least two non-sentiment phrases in a piece of collected textual content, and a number of web pages within a web site between the at least two mentions non-sentiment phrases.
7. The system of claim 1 , wherein one of the at least two non-sentiment phrases that are indirectly connected is a brand name, wherein the brand name is provided as an input by a user.
8. The system of claim 1 , wherein the at least two non-sentiment phrases that are correlative by nature include any one of: phrases that contain the same word, phrases that contain derivative of the same word, and similar phrases.
9. The system of claim 1 , wherein the data source is at least one of: a social network, a blog, a news feed, and a web page.
10. A method for identifying hidden connections between non-sentiment phrases, comprising:
receiving at least one proximity rule;
identifying by an analysis unit all connections between each of at least two non-sentiment phrases stored in a non-transitory data warehouse storage medium, wherein the data warehouse storage medium contains a plurality of non-sentiment and a plurality of sentiment phrases, wherein at least two non-sentiment phrases are determined to be connected when they meet the at least one proximity rule applies on mentions of the non-sentiment phrases in collected textual content, wherein the at least one proximity rule is at least a distance measure based on at least one of a number of characters and words between two mentions of the non-sentiment phrases;
identifying all direct connections among all the identified connections, wherein non-sentiment phrases of a direct connection are determined to meet a predetermined correlation;
filtering out all the identified direct connections from the connected non-sentiment phrases, thereby resulting with hidden connections of the non-sentiment phrases, wherein each of the hidden connections includes at least two non-sentiment phrases, wherein the hidden connection is any one of: a first degree of separation of the at least two non-sentiment phrases and at least a first degree of separation of the at least two non-sentiment phrases that are associated with a common non-sentiment phrase; and
associating between at least two non-sentiment phrases of each of the hidden connections and a sentiment phrase, wherein an association between the at least two non-sentiment phrases and the corresponding sentiment phrase is a term taxonomy.
11. The method of claim 10 , further comprises:
crawling one or more data sources by an agent operative on a computing device to collect the textual content from at least one data source accessible by a plurality of user nodes;
performing phrase extraction from the textual content to generate phrases; and
identifying the plurality of sentiment phrases and the plurality of non-sentiment phrases from the generated phrases; and
storing the identified hidden connections and created term taxonomies in a data warehouse storage.
12. The method of claim 11 , wherein identifying the sentiment phrases and non-sentiment phrases further comprises:
comparing each of the generated phrases to sentiment phrases and non-sentiment phrases stored in a phrases database;
determining that a phrase is a sentiment phrase when a match is found between the phrase and at least a sentiment phrase in the phrase database; and
determining a phrase is a non-sentiment phrase if a match is found between the phrase and at least a non-sentiment phrase in the phrase database.
13. The method of claim 11 , wherein the data source is at least one of: a social network, a blog, a news feed, and a web page.
14. The method of claim 10 , wherein the proximity rule further comprises at least one of: a number of mentions of the at least two non-sentiment phrases in a web page, a number of mentions of the at least two non-sentiment phrases in different web pages, a number of mentions of the at least two non-sentiment phrases in a piece of collected textual content, and a number of web pages within a web site between the at least two mentions non-sentiment phrases.
15. The method of claim 10 , wherein one of the at least two non-sentiment phrases that are indirectly connected is a brand name, wherein the brand name is provided as an input by a user.
16. The method of claim 10 , wherein the at least two non-sentiment phrases that are correlative by nature include any one of: phrases that contain the same word, phrases that contain derivative of the same word, and similar phrases.
17. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute the method according to claim 10 .