System and methods thereof for detection of user demographic information
A system and method for detecting implicit information related to a user are disclosed. The system comprises a network interface for accessing one or more data sources; a plurality of agents for collecting textual content from the one or more data sources; a processing unit for extracting explicit demographic information related to the user and generating implicit demographic information related to the user respective of the explicit demographic information; and a data warehouse connected to the processing unit for storing the generated implicit demographic information.
1. A system for detecting hidden demographic information related to a user, comprising:
a network interface for accessing of one or more data sources;
a plurality of agents for collecting textual information from the one or more data sources;
a processing unit;
a data warehouse; and
a memory coupled to the processing unit that contains therein instructions that when executed by the processing unit configures the system to:
analyze the collected textual information to detect phrases in the textual information and separate between sentiment phrases and non-sentiment phrases in the detected phrases, wherein any sentiment phrase and any non-sentiment phrase that contains one or more proper name contains at least one word that is not part of the one or more proper name;
extract, from the detected phrases, explicit demographic information revealed by the user;
generate implicit demographic information related to the user based on the non-sentiment phrases and on a frequency that the non-sentiment phrases appear in the non-sentiment phrases from which is extracted explicit demographic information; and,
store the identified explicit and implicit demographic information in the data warehouse.
2. The system of claim 1 , wherein the plurality of agents are further configured to collect data respective of the user's common activity with respect of the one or more data sources; and, to store the data collected in the data warehouse.
3. The system of claim 2 , wherein the user's common activity is at least one of: a typing speed, language proficiency, time of the day the user is usually active in the one or more data sources.
4. The system of claim 1 , wherein the one or more data sources are at least one of: a social network, a blog, a news feed, and a web page.
5. The system of claim 1 , wherein the implicit demographic information is at least one of the following statistics characterizing the user: user age, gender, relationship status, location, home address, work, hobbies, contacts, and at least one parameter related to a behavior of the user.
6. The system of claim 1 , wherein the system is further configured to authenticate implicit demographic information revealed by the user.
7. The system of claim 1 , wherein the system is further configured to determine at least a probability of the implicit demographic information.
8. A method for detecting implicit demographic information related to a user, comprising:
collecting textual information from one or more data sources;
analyzing, by a data mining system, the collected textual information to detect phrases in the textual information and separate between sentiment phrases and non-sentiment phrases in the detected phrases, wherein any sentiment phrase and any non-sentiment phrase that contains one or more proper name contains at least one word that is not part of the one or more proper name;
extracting, from the detected phrases, explicit demographic information revealed by the user;
generating implicit demographic information related to the user based on the non-sentiment phrases and based on a frequency that the non-sentiment phrases appear in the non-sentiment phrases from which is extracted explicit demographic information; and
storing the explicit and implicit identified demographic information in a data warehouse.
9. The method of claim 8 , wherein the extraction further comprises:
collecting data respective of the user's common activity with respect of the one or more data sources; and
storing the data collected in the data warehouse.
10. The method of claim 9 , wherein the user's common activity is at least one of: a typing speed, language proficiency, time of the day the user is usually active in the one or more data sources.
11. The method of claim 8 , wherein each of the one or more data sources is at least one of: a social network, a blog, a news feed, and a web page.
12. The method of claim 8 , wherein the implicit demographic information includes at least one of the following statistics characterizing the user: an age, a gender, a relationship status, a location, a home address, an occupation, hobbies, contacts, and at least one parameter related to a behavior of the user.
13. The method of claim 8 , further comprising:
authenticating implicit demographic information revealed by the user.
14. The method of claim 8 , further comprising:
determining at least a probability of the implicit demographic information.
15. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to perform a process comprising:
collecting textual information from one or more data sources;
analyzing, by a data mining system, the collected textual information to detect phrases in the textual information and separate between sentiment phrases and non-sentiment phrases in the detected phrases, wherein any sentiment phrase and any non-sentiment phrase that contains one or more proper name contains at least one word that is not part of the one or more proper name;
extracting, from the detected phrases, explicit demographic information revealed by the user;
generating implicit demographic information related to the user based on the non-sentiment phrases and based on a frequency that the non-sentiment phrases appear in the non-sentiment phrases from which is extracted explicit demographic information; and
storing the explicit and implicit identified demographic information in a data warehouse.
16. The system of claim 1 , wherein the system generates at least one item of implicit demographic information related to the user based on the non-sentiment phrases and on a frequency that non-sentiment phrases related to each of the at least one item of the implicit demographic information appear in all of the non-sentiment phrases from which the explicit demographic information was extracted.
17. The method of claim 8 , wherein the generating generates at least one item of implicit demographic information related to the user based on the non-sentiment phrases and on a frequency that non-sentiment phrases related to each of the at least one item of the implicit demographic information appear in all of the non-sentiment phrases from which the explicit demographic information was extracted.
18. The non-transitory computer readable medium of claim 15 , wherein the non-transitory computer readable medium causes generating at least one item of implicit demographic information related to the user based on the non-sentiment phrases and on a frequency that non-sentiment phrases related to each of the at least one item of the implicit demographic information appear in all of the non-sentiment phrases from which the explicit demographic information was extracted.