Systems and methods for identifying and analyzing internet users
This disclosure describes systems, methods, and apparatus for generating reports enhancing an understanding of Internet users based on their generated content and actions taken by others in response to the generated content.
1. A server system comprising:
a network interface that receives product, service, or customer data from a client, receives a query from the client, and returns a report of social profiles to the client;
a memory storing a searchable social profile datastore having one or more social profiles;
a processor running an API, a crawler module, a parser, an analysis module, and a scoring module,
the API receives the product, service, or customer data from the client via the network interface, receives a query from the client via the network interface, and returns a report of social profiles to the client in response to the query via the network interface,
the crawler module collects content and raw data from the Internet based on the product, service, or customer data,
the parser parses the content and raw data into terms,
the analysis module computes the following for each term: a reach value and an impact value, and
the scoring module:
computes scores for one or more social profiles for each term based on the reach value and impact value; and
adds to or updates the one or more social profiles with the scores,
wherein the reach value represents a number of relationships between a user and other users as well as a quality of those relationships,
wherein the relationships are identified based on (1) explicit links between users and (2) user actions that imply relationships.
2. The server system of claim 1 , wherein the analysis module comprises:
a compute reach module; and
a compute impact module.
3. The server system of claim 2 , wherein the analysis module further comprises a graph builder module that maps relationships between social profiles in the searchable social profile datastore.
4. The server system of claim 3 , wherein the graph builder module creates and updates a graph datastore.
5. The server system of claim 4 , wherein the compute reach module computes reach values based on quantitative analysis of the graph datastore.
6. The server system of claim 4 , wherein the graph datastore includes nodes representing users, and includes edges between the nodes, where the edges represent relationships between the users, and wherein the edges are weighted based on a quality of each relationship.
7. The server system of claim 6 , wherein the graph builder module is further configured to merge nodes deemed to represent the same user by searching for edges that are common to at least two different nodes and further considering the weight of each of these edges.
8. The server system of claim 6 , wherein a weight assigned to an edge is greater for an edge generated based on an explicit link between users than an edge generated based on user actions that imply a relationship between users.
9. The server system of claim 2 , wherein reach and impact values as determined by the analysis module are used to calculate scores in the scoring module and the scores are assigned to social profiles in the searchable social profile datastore.
10. The server system of claim 1 , wherein the compute impact module computes a number and quality of actions taken in response to a user's publication of content on the Internet.
11. The server system of claim 1 , wherein the query includes one or more contexts describing one or more terms, and the report includes one or more social profiles having highest scores in the one or more contexts.
12. The server system of claim 1 , further comprising a profile enrichment module running on the processor, the profile enrichment module assigns actual user names to the one or more social profiles in the searchable profile datastore.
13. The server system of claim 1 , wherein the crawler module creates a node for every URL crawled.
14. A method for generating reports enhancing an understanding of Internet users based on their generated content and actions taken by others in response to the generated content, the method comprising:
collecting content or other raw data via a crawler module that accesses webpages from a network interface of a server system;
associating the content or other raw data with a social profile residing in or being added to a memory;
calculating scores of reach and impact for the social profile based on terms parsed from the content or other raw data, wherein the reach scores are based at least in part on a number and quality of relationships between a user associated with the social profile and other users, and where the relationships are identified based on (1) explicit links between users and (2) user actions that imply relationships;
receiving a query, via the network interface, for users fitting one or more contexts;
identifying social profiles fitting the one or more contexts; and
returning a report in response to the query and transmitted through the network interface, having the social profiles fitting the one or more contexts wherein an order of the social profiles is based on the scores.
15. The method of claim 14 , wherein the calculating scores is performed based on relevance scores for each of the terms parsed from the content or other raw data, wherein reach depends on a number and quality of relationships between users.
16. The method of claim 14 , further comprising:
receiving another query, via the network interface, for contexts fitting one or more users;
identifying the contexts fitting the one or more users; and
returning another report in response to the another query and transmitted through the network interface, having the contexts.
17. A method comprising:
seeding a crawler module;
crawling the Internet to find new users that can be used to create new social profiles, the first crawling based on the seeding;
crawling the Internet to further populate existing social profiles, the second crawling based on the seeding including at least consumer data from customer transactions, wherein the existing social profiles are based on existing customers of a marketing client;
parsing content or other raw data generated by the first and second crawling into terms;
associating the terms and the content or other raw data with the existing social profiles or the new social profiles;
computing scores for each term based on reach and impact values for each term, where the reach values represent a number of relationships between a user and other users as well as a quality of those relationships;
associating the scores for each term with the existing social profiles and the new social profiles;
returning a report to the marketing client via the network interface in response to a query received via the network interface, the report including one or more social profiles matching contexts found in the query.
18. The method of claim 17 , wherein computing scores for each term includes computing relevance values for each term and then computing scores for each term based on the reach, relevance, and impact values.
19. The method of claim 17 , where the report includes one or more contexts.
20. The method of claim 17 , wherein the marketing client provides the customer data.